Sales: dataon_sales@dataonstorage.com     |     Tech Support: Please visit our support portal     |     Office Support Hours: M-F 9:00AM ~ 6:00PM (PST)
Title
Contact Us
Sales: 888.726.8588

Nexenta High Availability ZFS Storage System Setup Using Cisco and DataON

"Author: Rolf Versluis @ Adcap Network Systems Blog"

 

In order to use a storage system beyond niche applications, it has to be able to scale up to full high availability with no single point of failure. As part of prototyping Adcap’s new storage product, we have set up and started testing a clustered high availability configuration. This article is an overview of the setup of the Nexenta High Availability Cluster plugin with some pictures of a few gotchas along the way.

 

 

Equipment:
  • Two Cisco C240 servers
    • Dual 8 core processors at 2.7 GHz
    • 132 GB memory
    • Qlogic QLE8152 10 Gbps 2 port CNA
    • LSI 9207-8e SAS2 HBA
    • LSI 9266-8i MegaRAID SAS HBA
  • DataON JBOD 1600
  • DataON JBOD 1640
  • Six Hitachi 3TB 7800 RPM drives
  • Eight Toshiba 300GB 10K RPM drives
  • Two sTec 400GB MLC and Two sTec 8GB ZeusRAM SSD
  • NexentaStor version 3.1.4 software 30 day trial

 

 

The setup I am using is two Cisco C240′s. It seems kind of a waste of hard drive real estate to get a 24 drive server and only use 2 hard drives in it, but that is part of a greater plan. I have to use at least a 2U rack mount server in order to get the number of PCIe slots that I need. Also, in many situations, our customers are not necessarily going to start with a Clustered configuration. They might start with a single C240 storage device. Then add a second one for storage redundancy. Then after getting familiar with the Nexenta storage environment, they may want to move more data to it, and grow to a high reliability configuration.

 

In that case I would anticipate that the C240′s are pretty much filled with hard drives. When the JBOD’s are added, the stored data is moved over to the drives on the JBOD’s because it is more reliable. At that point the drives on the C240′s are taken out of service and physically placed in the JBOD. It is of course easier if the drive trays on the servers and JBOD’s are the same, but it’s not a big deal to move the drives from one set of trays to another.

 

The C240 is also what Nexenta has put on the Hardware Compatibility List, which is important for support, so it all comes together into the C240 being the Cisco server of choice. The below configuration would be when all the drives have been moved out of the C240′s.

 

For Nexenta Clustered High Availability to work, the two servers have to be identical in version and PCIe expansion card configuration. The NexentaStor software will take advantage of all the memory available to it for the Adaptive Refresh Cache, and these servers both have 128GB of 1600 GHz DRAM in them, so they should have some good performance. The following two screen shots are of the Cisco server management GUI, showing processors speed and the PCIe adapter cards.

 

The Cisco servers were shipped to me with Cisco P81 Virtual Interface Cards installed, which is Cisco’s 10GB Converged Network Adapter. It is a great card to use on a regular server because it can be logically carved up for different virtual machines, but in this case we don’t need that functionality. There are also no drivers for the P81 available on the NexentaStor software, so I took them out. The Qlogic QLE8152 works for 10Gb connectivity, even though it is not officially supported by Nexenta, and I had some of those in my lab, so that’s what I used in this setup.

 

The LSI Mass storage controller listed in slot 3 is the LSI 9207-8e SAS2 external HBA. It is what we use to connect to the external JBOD’s. It would better to have two of these cards in a full production system.

 

The LSI 9266-8i MegaRAID SAS HBA is a hardware RAID card that is used for the 24 internal drives. We are just using it for the operating system discs. When we have a C240 full of drives, we end up setting up all the drives as individual RAID 0 groups, which is not optimal, but works. My preference is to use a LSI 9207-8i for internal drives.

 

This is a picture of the back of the servers and the DataON JBOD’s. Notice a problem? I had not put the external SAS HBA’s in the sames slots on each server. After taking this picture we corrected the issue before doing testing.

 

I like to make lots of mistakes when the equipment is in the lab, that way it gives us the ability to anticipate and avoid these issues when doing customer installations…

 

The DataON JBOD’s are great for a HA configuration. The power supplies are accessible from the back and are hot swappable. So are the SAS interface trays! Each of the SAS interface interface trays has two inputs, one for each server, and an output, for JBOD cascading.

 

Furthermore, there is an entire second SAS tray. In the recommended configuration when using a stack of multiple JBOD’s cascaded together, the dual trays allows for full connectivity by both servers at each end of the stack. That way if a JBOD in the middle has a hardware issue, all the rest of the JBOD’s are still accessible by the servers.

 

In order to use a storage system beyond niche applications, it has to be able to scale up to full high availability with no single point of failure. As part of prototyping Adcap’s new storage product, we have set up and started testing a clustered high availability configuration. This article is an overview of the setup of the Nexenta High Availability Cluster plugin with some pictures of a few gotchas along the way.

 

Nexenta High Availability ZFS Storage System Setup Using Cisco and DataON

 

Your are currently browsing this site with Internet Explorer 6 (IE6).

Your current web browser must be updated to version 7 of Internet Explorer (IE7) to take advantage of all of template's capabilities.

Why should I upgrade to Internet Explorer 7? Microsoft has redesigned Internet Explorer from the ground up, with better security, new capabilities, and a whole new interface. Many changes resulted from the feedback of millions of users who tested prerelease versions of the new browser. The most compelling reason to upgrade is the improved security. The Internet of today is not the Internet of five years ago. There are dangers that simply didn't exist back in 2001, when Internet Explorer 6 was released to the world. Internet Explorer 7 makes surfing the web fundamentally safer by offering greater protection against viruses, spyware, and other online risks.

Get free downloads for Internet Explorer 7, including recommended updates as they become available. To download Internet Explorer 7 in the language of your choice, please visit the Internet Explorer 7 worldwide page.