IBM BladeCenter S: A true out of the datacenter experience.

A couple of days ago I finally got a chance to play around with a IBM BladeCenter S chassis with a HL21 blade server. The BladeCenter S is positioned as an entry level blade solution for the SMB market and looked like the perfect solution for a small Lotus Domino based local consultant company that I was hired to perform the installation for. The usage scenario was a powerful, extensible and manageable solution that they could utilize in their development and testing work as well as allow for further growth. Initially it will be used to run VMware Server and run a few development VMs, but the plan is to migrate to a full VMware ESX based infrastructure as the company and their computing needs continue to grow.

But, this is supposed to be about the BladeCenter "experience", so I'll get right back on track. Last week we got the equipment and we spent a some time unpacking it and assembling all the components, fitting it in the rack on the serverroom etc. Then, the first disappointment hit us. The power cables that were delivered with the power supplies, were for use with a PDU only. In other words, they simply didn't fit into the power sockets in the server room. Remember, this is a pretty small firm that hasn't got a full datacenter setup with all the bells and whistles that bigger clients have. So, we had to order some new cables just to be able to supply some power to the thing in the first place. Of course the IBM site that explains what features the BladeCenter S has, only claims

BladeCenter S uses standard office power supplies, enabling you to build your IT outside of the data center

so we thought we got what we needed. Sadly that wasn't the case, but I admit we probably should have checked that part a bit better with our hardware vendor.

After this little setback, we had to wait a couple of days before we had the time to get the setup working and configured. So this afternoon we started fresh, armed with proper power cables. Initial setup of the chassis was a breeze, so was also the case for the Nortel Layer 2/3 10 Gigabit Uplink Ethernet Switch Module. That little gem of a switch really impressed me with easy setup and dynamic routing between different networks on different ports. Since we're doing virtualization here we had a need to connect it to both the DMZ and the LAN as there will be VMs running in both zones. The Nortel made that very easy to accomplish, as well as a very nice way to rewire the existing servers into the same segments. Good one!

Feeling very upbeat about the whole BladeCenter, we continued to setup the first (and so far only) HL21 Blade and get the basic Windows 2003 Server Enterprise x64 setup running before diving into VMware Server.

Thats when the problems really started. The SAS controller module inside the HL21 Blade could not see any of the SAS disks that were present in the SAS Connectivity Module.

That lead us to really digging into the documentation, without really finding any clues at all. The only thing we discovered by doing that was that the SAS Expansion card doesn't support Raid5. Come again? No Raid5? I seriously never anticipated that to become a problem. I still have a very hard time wrapping my head around the fact that a "state-of-the-art" Blade Server/Chassis, manufactured by IBM and introduced in 2007 (!!) doesn't support Raid5 out of the box.

2.4.4 RAID options
The SAS Expansion Card supports three RAID levels:
RAID-1 (called Integrated Mirroring)
RAID-1E (called Integrated Mirroring Enhanced)
RAID-0 (called Integrated Striping)

This is taken right out of the IBM RedBook "Implementing the IBM BladeCenter S Chassis", also known as redp4357. Can you believe it? According to a forum post somewhere, that I'm unable to find again right now, Raid5 support will be available in Q2 2008. It better just be a firmware upgrade when it's released, and not a replacement part.

In pursuit of the lost SAS drives we also ran into a blog post from IBM about the release of a new Storage and Configuration Management software (SCM) and in out of pure desperation we thought that might be the solution to our problems. A very slow 250MB download later I had the SCM installed on my laptop, but that did us no good at all. In fact, it gave us nothing at all. Simply put, it just didn't work. After installing it my browser tries to open http://localhost/Console/faces/protected/MainConsole.jsp and that doesn't exist. Going back through the documentation it seems like it should be using port 32100, so I simply tried adding that to the URL. Still nothing. A big blank nothing once again appeared in my browser, and that was that. I tried uninstalling, reinstalling, rebooting but I still couldn't get the thing to work. I checked firewall settings, ran netstat -a to view open ports and nothing was to be found. The SCM documentation mentions the IBM SCM Service, but that service is nowhere to be found on my machine.

After scratching our heads until we nearly started bleeding, it turns out that the problem with the missing SAS drives was twofold.
Firstly, the Options Rom on the blade doesn't have enough storage space for two SAS controllers and PXE boot on the network cards. As unbelievable as it might sound, it's actually true. A small hint IBM; Expand the size of the options rom when you add new features?

The second part of the problem was equally stupid. It turns out that the firmware revison of the SAS Connectivity Module we got wasn't compatible with the BladeCenter S. After all, we only ordered the BladeCenter as a complete pre-configured setup from IBM it's not like the components we get should work together.  Upgrading the firmware worked though, and we got the blade software installation running pretty quickly after all of the problems were sorted out.

Now, all that remains is configuring Windows 2003 Server, VMWare Server, creating some template VMs and setup the VMware virtual switches and whatnot. I'll leave that to another day though, I've had my share of BladeCenter experiences today.

Posted by Christian Mohn aka h0bbel