Datacore SanSymphony split brain scenario

Posted: March 25, 2012 in datacore, Sansymphony, virtualisation, VMWARE

Anyone looking at SanSymphony should read this article if they are considering stretching mirrored SanSymphony volumes across two data centers.

We’ve implemented this very scenario at the port with a slight difference around mitigating the impact if god forbid a full split brain disaster should occur. We’re currently using vSphere4 so we can’t utilize the new features of vSphere5 that allow stretched esx clusters either.

Oliver Krehan’s article deals with a vSphere5 stretched esx cluster scenario and is a must read. He does suggest one vmfs volume per vm which I’m always loathed to use, as it totally negates the point of vmfs. I would add that doing everything possible to avoid a split brain scenario at the design phase is critical. I’m putting together a post soon to cover stretched SamSymphony clusters.

SanSymphony implements a primary/secondary node structure that means that it will direct IO to the primary path, (Ignoring ALUA, which is a whole different discussion) unless that node’s paths are dead.

Effectively at the port we have created two esx clusters, one in each data center which are mapped to all the vmfs volumes available at both data centers. Virtual Machines are then run on the cluster that is in the same data center as the primary node volume. This means that in normal operation, IO stays within the data center and only crosses the ISL links in a failure. In a full split brain scenario where no LAN or SAN connectivity is available between data centers, we should have changes being made only at the primary node and not at the secondary.

There are a couple of limitations, one is that a complete failure of a data center requires manual intervention (or a partial or full automated script) to restart virtual machines on the other cluster. The other is that careful management of virtual machine placement is needed to ensure virtual machines are correctly aligned on the correct cluster and vmfs volume.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s