Friday 5 February 2016

Server down

My Secure Server has been running rather slowly. I rummaged around a bit, and couldn't see why, so I rebooted it, just on general principles - a reboot sometimes fixes a problem, and it's very easy to do.

This time, it had the opposite effect. The computer wouldn't start up. Time to get busy - very busy.

The secure server is an important computer - it's the one that people use when they give stuff like credit cards to buy things. If there's no secure server, then there's no commerce taking place - bad news. Very bad news.

In fact, that news would be so bad, that I have another one standing by, for just such an eventuality. So I switched the load over to that, and was back in action in minutes. Then I looked at the failed computer.

It's a hard drive problem, and at this point, I don't know exactly what failed. It looks like it can't read the partition sector, but that might just mean it can't read anything. This drive isn't going to be used again. I'm giving it a thorough disk-wipe, and it might appear in a geocache one day.

So with the backup running nicely, it was time to think about another backup. Actually, I had two backup servers for the secure server, because it is just *so* important. I decided to build a new one to replace the one that failed, and at the same time reload the software on backup number 2. And while I was doing this, I documented what I had to do to set it up, so that next time this happens, I can just follow the recipe.

For the hard drive, I'm using a very old 25 gb (that tells you how old it is) IBM drive. Old, but it's been very reliable, as IBM kit often is. For the operating system, it's Red Hat Fedora Core 23 (I started using this before Core 1) which is the latest. I installed the latest OpenSSL crypto software, the latest Apache web server, and so on and so on. And now I have a sparkly new backup server humming quietly (and periodically copying what's on the main secure server) ready to go in case of failure.

And a second backup!


  1. What would be the contingency plan for this kind of situation should you be indisposed?

  2. Someone from my huge IT department would do what's needful.