Yet another frequently asked question: My company is getting more serious about virtualization. Should we keep using our fibre channel SAN or switch to iSCSI or NFS?
My usual answer is a series of questions: What technology do you know best? How big is your SAN? Why are you thinking about switching? What’s your performance like? What will it cost?
The thing is, I never have an answer. Whereas some common virtualization questions have easy answers this one depends heavily on what you’re trying to do. iSCSI is a great way for small- and medium-sized organizations to get into cluster filesystems. With 10 Gbps Ethernet you can get SAN-like performance, too, but 10 Gbps NICs are as costly as fibre channel HBAs. If you already have a fibre channel SAN you may already have a lot of what you need. There are large iSCSI-based virtualization implementations, there are large NFS-based ones, and obviously large FC-based ones, and a lot of good reasons to choose any of them.
Sometimes there’s a massive savings with one of the technologies, or there’s a particular storage vendor that you like that only does iSCSI, NFS, or fibre channel. If there is not a compelling reason to go one direction or another, and everything you’re evaluating is on the Hardware Compatibility List, I’d make the call based on what I know and limit the number of new things I’m trying to do at once.
Update: Commenters are chiding me for not mentioning NFS, and the chiding is a good thing. I don’t use NFS, but a lot of people do, and they love it. The problem I’ve had with NFS is that VMware doesn’t seem to think it’s as cool as other options, because their certification of NFS-based arrays seems to lag or not exist in a number of cases (like for SRM). As such I tend not to suggest it, because I don’t want to see someone get trapped with it. However, with Storage VMotion, migration to a new storage technology is easy, so it’s not a problem anymore. As such I’ve updated the post. Thanks guys!
——————————–
This is the fourth post in my series of “what VMware questions do I hear most often?” The first three questions were:
- How much capacity should I have for VMware?
- Should I convert my old servers to ESX?
- What kind of servers should I buy for VMware?
If you think of a question you’d like me to answer please put it in the comments. Thanks!
You forget the third option: NFS.
NFS is a *great* storage option for ESX. It is really simple to setup; if you use vlans, security is no longer an issue and you do not have to provision the whole disk at once: it grows as needed (and with lvm in linux, you just plug in more disks if you run out of space). If you are an ‘appliance’ kind of guy, then you can give your bucks to NetApp, they will be more than willing to give you a very expensive (but very nice) NFS appliance.
I agree with Natxo. NFS is the bees knees. We have 6 esx 3.5 servers, clustered (getting ready to add 5 more), with 15-20 vm’s each. We’re running it all over NFS to our Netapp filer. We have had no performance issues and management is a breeze.
I’m curious why you didn’t mention this option…?
I added NFS to the post:
Update: Commenters are chiding me for not mentioning NFS, and the chiding is a good thing. I don’t use NFS, but a lot of people do, and they love it. The problem I’ve had with NFS is that VMware doesn’t seem to think it’s as cool as other options, because their certification of NFS-based arrays seems to lag or not exist in a number of cases (like for SRM). As such I tend not to suggest it, because I don’t want to see someone get trapped with it. However, with Storage VMotion, migration to a new storage technology is easy, so it’s not a problem anymore. As such I’ve updated the post. Thanks guys!
After several months of research, I am still convinced that FibreChannel is a legacy technology. That is, its primary purpose existed to encapsulate Storage in a Layer 2 frame so as to reduce CPU load required for the networking and its secondary purpose was create a quasi-proprietary network for storage vendors.
Ten years later and CPU is not the issue and convergence is all the rage, therefore FibreChannel is on the decline. Your post is yet more proof that this is true since you do not automatically consider Fibrechannel.
I discuss this in more detail in a post at http://etherealmind.com/2009/04/20/the-case-against-fcoe-fibrechannel/ for anyone who is interested.