I've been looking at big boxes on Ebay for a while. By big, I mean heavy on memory.
The boxes I use at the moment, can only take 2gb of memory - that's because I'm using mostly motherboards that I bought several years ago, because I decided then that it would be a great idea to pick a motherboard that has all the features I want, then buy a couple of dozen of them. I still have several boxed and unopened!
But back in those days, 300 gb was a big drive. Now 300gb is barely worth mentioning; I'm using 2000, 4000 and 8000 gb drives.
When you're accessing drives so large, you want the computer to do a lot of caching, especially of inodes, because reading the same bits of disk again and again really slows things down, compared with having those bits cached in memory. So, as everyone knows, having a lot of memory makes your computer faster.
I've been looking at 64gb, 128gb and 256 gb servers. These tend to cost £200, £400 and £1500. But I was wandering around Ebay a few days ago, and I noticed an auction for a 64gb server, plus a 16gb server of physical size 2U (that means 3 1/2 inches high), and the current bid for the two of them was £10!
Well, I couldn't ignore that. So I put in a bid, and wound up acquiring the two for £45.
How come there are such bargains on Ebay? Well, it's about supply and demand. The supply comes from bankrupcies and clearances, although this sale was an individual. The problem is demand. No big company is going to entrust its precious data to a server bought second hand from Ebay. For a start, you won't get any support from the vendor (Dell in this case). And the person making the purchase decisions isn't spending his own money, so it's *much* more important that his arse be covered than money be saved. And private individuals don't buy this sort of computer; it's not designed to sit on your desk or under it, its natural habitat is a 19 inch rack in a data center.
Along comes myself. I run a small internet company, and that needs equipment (and yes, I do have a couple of 19 inch racks). So I get Cisco Pix firewalls at a tenth of the price of a new one, and in this case, the two servers I bought would probably have been a few thousand when new.
But - no vendor support, no income tax, no guarantee, no money back (and no VAT either, the guy I bought it from isn't a trader, so doesn't need to register for VAT).
Before I parted with the cash, we started up both of the computers, and they worked, so I piled them into the motor and took them home.
First, the big 64gb box. It's a Dell Poweredge R805, so I went to the Dell web site and downloaded the hardware manuals. It came without a hard drive (and I suspect it's never had one, plus the date was completely wrong, so it might never have been actually used). I need a hard drive, obviously. But the expansion slots were PCIe, and all the stuff I have is PCI. But! I made a mistake several years ago, and bought three PCIe Sata raid cards, which have been sitting on the shelf ever since. So I got one of them out and installed it. I started up the computer, and got "PCIe training error", whatever that means.
Inside the box, there's one Sata connector, a USB, and an SD card slot. Aha, I thought, and dug out a 64gb USB drive that I bought a couple of years ago for no particular reason, other than it was really cheap. I plugged a DVD Rom drive into the Sata slot, powered it from an external power supply (the R805 doesn't seem to have any way of powering drives!) and tried to install Linux Fedora 23. That didn't work, it crashed very soon after I started, so I tried the other way. I put a 64gb SD card into the slot (again, bought for no particular reason a while ago). And I installed Linux - hurrah!
But I was only getting 56 gb, it was reporting two of the memory sticks as faulty. I tried reseating them, no luck. I tried various things, and eventually, by swapping the sticks in B1 and B2 with the sticks from B7 and B8, I got the full 64gb. Nice. I'm running an extensive memory test now; if any of the sticks fail, I'll replace them.
So at this point, I have a computer that works, has 64gb memory, running Linux Fedora 23 on a 64 Gb SD card. Looking good! But I need to attach hard drives.
First, I tried another PCIe Sata card, and that worked, so the problem was with the first card. This means that I now have the capability to attach five drives to the system; four on the PCIe card and one on the motherboard connector. But there's no space inside the box to put the drives (it seems to expect to have two 2 1/2 inch drives, puny) and no power to spin them.
So here's what I'm going to do. I'll let the five Sata cables run out of the rear of the box, and next to it, I'll have a 1U box, with no computer, but with a power supply which will power the drives inside it. And the drives will have Sata cables coming out of the back of that 1U box. So then I connect the five cables from the Dell to the five cables from the drive box, and that gives me anything up to 40 terabytes (five 8tb drives). And all I really need is 16 tb, so there's room for exansion in the future. To make the Sata connection, I'll need cables that have a male connector at one end and female at the other; I just ordered those from Ebay at £1 each.
On June 14, Fedora 24 comes out (free, of course), so I'm planning to do my final install using that. I've also ordered a 512gb memory card (£5) and a couple more 64gb cards (£2.68).
There were two servers - don't forget the 16 gb server, which is a Dell Poweredge 2950. The first thing I did for that was have a look inside at what sort of memory is uses. It has four free memory slots, so I ordered (from Ebay, of course) another 16gb, £18. It came with a plethora of hard drives; two 73gb and four 146 gb. It was running Windows Server, but I don't want that. First, I changed the configuration of the drives, so that instead of running the 146 gbs using Raid 5 (which is clever, and I don't trust it) it's running Raid 0 (which is simple, and I do trust it). But what happens if a drive fails, you might be thinking. Well, that's why I have backups. Some systems need Raid 5, because they can't have even a minute's downtime. But my needs aren't like that, so what I do is I run a pair of servers, one is customer-facing, the other is a hot backup for the first. And that's the role I'm planning for this Dell.
So I configured the Raid to look like two drives, one 146 gb (the two 73s) and one 584 gb (the four 146s). This 2950 has a built-in DVD, so installing Linux was straightforward.
Again, I'm planning to put in a PCIe card and that will mean I can attach six Sata drives (four from the card, and two off the motherboard). And those will likewise go into a separate drive box.
Another nice feature of these servers is that the 2950 has two gigabit ethernet ports, and the R805 has four. So I'll put up a gigabit switch, so that these two servers can talk to each other on a very fast link; ideal for doing the backups, and it won't saturate the rest of the network.
Computers are such fun.