No, I’m not quoting the lyrics from the 80’s rock group Foreigner. Two companies have had server issues this week and its not fun trying to recover from server crashes. STAR BASE has had to help them recover their applications. It’s especially frustrating because both of these situations were rather unusual.
The first one had a system drive with a raid 1 mirror. This means that data that has written to one drive, is also written to a second disk drive. So, if one fails, the system is able to run off of the other drive. Apparently the raid controller was also having a problem, and the second drive was not being written and so the server crashed along with all of the applications it was running.
The second company was running a server and had a raid 5 array. In this configuration, the system would continue running if any one drive failed. But in their case, two drives failed, which caused them to lose all of their applications and data.
One of the reasons for sharing this is I can use one of my Jeffisms and that is Never happens quite a bit. Neither of these machines should have went down because of the strategies used to provide for redundancy. Both cases were very unusual and both had a good strategy for preventing server outages, but guess what? Disaster can strike at any time, and its usually never a good time.