How much disk I/O are you doing? Are your performance monitors watching everything they need to? I had occasion to dissect a storage setup recently, and it became obvious that the I/O to and from the arrays, including the remote mirroring traffic, wasn’t really being watched.
When your host writes to LUN across a SAN, you:
1) Generate load on the link between the host and the fibre channel switch.
2) Generate load on the ISLs between the fibre channel switches.
3) Generate load on the link between a fibre channel switch and the array.
4) Generate load back out of the array as it writes the I/O remotely. If this is synchronous the host will be waiting for it. So you get load on from the array to the switches, and…
5) load on the ISLs again to the remote site, and…
6) load between the remote switch and the remote array.
If you’re going to do SAN performance monitoring do it where it counts: on the ISLs and links to the arrays. Sure, a host might push its individual connections hard sometimes but if your ISLs are taxed or your array connections are taxed everybody will suffer. If you are doing synchronous mirroring you absolutely need to watch those ports, too. If they get overloaded the whole system will slow.
If I have to choose what I monitor first I choose the core infrastructure components. Make those fast and reliable and the hosts will figure themselves out. 🙂
