There was a terrible smell in the Data Centre this morning. A smell I recognised - hydrogen sulphide. And that means - bad batteries in a UPS. I know it means that because I've had it many times before, and I know how to fix it.
I have five APC 3U 3000 KvA UPSes. They each use eight SLA (sealed lead acid) batteries, and when one (or more) of those batteries goes bad, you get the smell. And a red light on the front.
So we used the "nose test" to determine which one was bad, and the red light to confirm it. And those things are *heavy*, so we used the pump trolley (every Data Center should have one). We slid it onto the trolley, and trundled it to where there was space to work on it, then pumped the trolley up to a good working height..
I unscrewed the dozen or so bolts that hold the top plate on, and had a look. Sure enough, those batteries looked minging; bulging a bit, and distorted. So I levered them out (in extreme cases, I have a crowbar I use, but these came out with a bit of persuasion from a large screwdriver. I put the voltmeter on them, and they were severely duff. So they're going to the recycling center. Lead is useful.
I had a set of eight batteries that I'd taken out of another unit. I like to change these batteries every three years , but when these came out, they looked perfect, so I kept them. And yes - seven months after being removed from their old home, they were all still showing 12.5 volts.
So I removed the wiring harness from the old batteries, put it on the new (I always forget to make a diagram of how it's supposed to go, and I have to logic it out). I say "new" but they were dated August 2014, so they should be out of date by now. However, the voltmeter says OK, so I believe it. Put the top cover back on, gave it power, and it's sitting now, charging up quietly to itself. I think it's looking good.
If those batteries don't work, I'll order a new set from Ebay, it's only £80, and that even includes a new wiring harness!