A computer for Christmas It was April, and the elves were still sorting out the shambles from last Christmas. And a right shambles it was too; boys toys sent to girls, girls toys sent to boys, and some customers getting no presents at all. The reindeer schedule had been cobbled together at the last minute, the dwarves were given the wrong production dockets - a complete shambles. Everyone was fed up, and highly cheesed off with the boss. The dwarves were fed up at the amount of overtime they'd put in, the reindeer were seriously thinking about a strike, and the elves were working-to-rule. Rudolf "the Red", the reindeer's shop steward, called a meeting, and invited delegations from the dwarves and elves to attend as observers. "We can't go through another year like this. We've got to force management to invest some of their bloated profits to move forward into the future." he said. Everyone agreed, and so they formed the Ad Hoc Committee to Study Ways and Means. By the time the committee had reported back, it was July, and by then everyone had gone on holiday. So it was not until September, at the annual Conference that the report was examined. The Ad Hoc Committee's conclusions were simple - all aspects of ordering, production and distribution had to be computerised, and Conference endorsed the report and voted that it be made a non-negotiable demand at the next meeting with management. Meanwhile, in a penthouse deep under the North Pole, Sir Sangrethta Claus and the rest of the main board of Christmas PLC were having a meeting. "It worked a treat", said Sir Sangrethta. "They'll be practically begging us to put in computers." For some time now, Sir Santa had been trying to introduce modern technology throughout his plant, but it had to be done carefully, as a strike at the wrong time would have a disastrous impact on his bottom line. So in early October, Rudolf and his fellow negotiators sat down at the table with Sir Sangrethta Claus and his industrial relations director, expecting a tough fight. But after a token resistance, the threat of a December strike worked as usual, and management acceded to the worker's just demands. Perhaps it was as well that Rudolf didn't hear Sir Santa's "Ho ho ho" as he left the meeting. Naturally, at that point, they called in the best consultant they could find, so I packed my warmest woollies, and set off for the North Pole. They had written down their requirement, which made a change, so when I sat down for my first meeting with Santa, I knew what to expect. The problem was a little unusual. It was a standard ordering, stock control, production, distribution and accounts system they needed, but with a highly seasonal demand pattern, and with a rather high volume. About two billion customers must receive their presents exactly on time, and although some orders are received in good time, for the most part the customers don't do anything until the last minute. Santa gave me a guided tour of the system. "The orders come in here", he said, "and go down to the elves. They enter them in a ledger, with the name and address of the customer. Each order is given a number, and a production docket is filled in." We proceeded to the workshops. Almost drowned by the din of hammers, Santa shouted "The dockets are distributed from here". A dwarf rushed past, sneezing violently and carrying a big pile of boats. "We're already ramping up production for this year. Each docket is given to the dwarf that specialises in that particular product", he said. "When the toy is finished, the docket is pinned to it, and it goes back to the elves." We went back upstairs to see another band of elves. They were matching the docket numbers to the ledger entries, and tying a label with the delivery address to the goods. "From here it goes to dispatch", Santa said. "What about invoicing?", I asked. He took me into another large room, where yet more elves were working from huge directories of birth and marriage certificates. "We have to work out which parents to invoice; it isn't always as simple as it sounds. Some customers have several parents, some have none. But they all have to get presents, and it all has to be paid for." "I never realised you ran such a complex operation", I said. Santa turned and looked at me. "This is the biggest business in the world. There are two billion customers, and on the average they get twenty five pound's worth of toys. Our turnover is bigger than most governments. Over the years, we've gradually built it up from a non-commercial and amateurish thing into a proper big business." We went on to Distribution. Here, the reindeer were gift-wrapping the presents, sorting them according to post codes, and passing them down a chute to the loading bay, where they were piled up on the sleighs. "The trouble is, everything's done by hand, and takes forever. We have to service our two billion customers in an incredibly short space of time, so the work force is enormous, and grossly underutilised most of the year." I agreed. "Yes," I said, "I can see a lot of ways that computers would help. For example, looking up the customer's parents should obviously be done by a database, and the post code sort could be done at the same time." Santa beamed. "There's a lot of things that need doing; it's a big job," I continued. "Probably be able to get it done in a couple of years or so." Santa's beam turned into a blench. "Gulp", he gulped. "We've got to have it in time for the coming Christmas." Now, I'm used to tight deadlines, but this was ridiculous. "Not possible", I said. "Can't be done. You can't just wave a magic wand and expect a major system like this to appear out of thin air in no time at all. It needs careful analysis, specification, design and finally coding; at least two years." "What if we waved a million magic wands?", asked Santa. "?", I said. "We can use the fairies. The wishing level doesn't get very high until much later in the year, so they haven't got much to do. They can do the grunt work of actual coding and data entry, but we need you to design it. It must be done for this year, and money is no object". That last phrase silenced the objections forming in my throat. Little cash registers went ding in my head, and my eyes filled with pound signs. Maybe the job couldn't be done by ordinary means, but with a million wishes at my disposal, maybe, just maybe ... It took me two weeks to complete the fundamental design. I started by designing the data structure, in the best Wirth tradition. The main problem was the sheer size of it. Santa had said two bilion customers, but I knew that you have to allow for growth. The problem is that the largest integer that you can store in four bytes is four billion, even storing it unsigned. And I had to assume that the North Pole operation might one day be expanded to cover the customer's parents as well as the customers themselves, which would require six billion records. This, I realised, was a major problem, as it meant that I couldn't use an off-the-shelf data handling package like dBase II, but would have to code from scratch. I had to invent a new type of integer, the six-byte integer, and all indexing would be done with those. It was also clear that one IBM PC would not be able to cope with the problem, so a network system would be needed, with its added complexity. There were many other problems that I'd never met before; one was the problem of magnetism. Being sited exactly at the North Pole, compasses went crazy, and I was worried about how the magnetic media would cope. So I specified Faraday cages for all equipment, and a second cage to go around the entire computer area. I decided to keep records for all potential customers, as well as current ones. This was partly with one eye on future expansion, but also to simplify the problem. So each record contained the customer's name, address and post code, codes that indicated past deliveries to that customer, the customer's birthday (again, with an eye on future expansion), and pointers to the records on the customer's parents. There were also fields for the current orders, and fields to record the progress of the order. The elves job would therefore be simplified - all they had to do was read the customer's order (sometimes the handwriting was atrocious), call up the customer's record and enter the order details. Invoicing would be automatically routed to the correct parents, using the pointers in the database, and the reindeer could deliver the invoices as part of the main drop, instead of having to make a separate journey. A separate data base would take care of production and stock control, but they would use common codes, and there would be links between the two data bases. The docket system would be abolished, and dwarves would simply make product to replenish stocks, whenever a computer-generated forecast of stocks got too low. This would free the dwarves from all paperwork, and let them get on with their real work, which would make them less grumpy. They wouldn't have to transport the goods around either. Part of the design called for pixies to do the gift-wrapping, and they could also transport the goods to the warehouse. The reindeer would have less paperwork also. The computer would be able to decide when a full sleigh-load for a particular area could be made up. It would them print out a list of goods and the dispatch address. The reindeer would draw product from inventory already gift-wrapped, stick the label on, and load it into the sleigh along with the invoices that needed sending to the same address. They could then concentrate on their main job of hauling the goods to market, and on arrival, insertion of the goods into the appropriate receptacle. The total size of the database was 2 billion records, with each record being 1K long, for a total of 2 trillion bytes (2 million megabytes, or 2 terabytes. Since the largest hard disk available is only about 400 megabytes, we would need 5,000 of them. I specified 5000 IBM ATs as file servers, linked together in groups of 64. Each group of 64 had an AT as a supervisor, and the supervisors were again grouped in a network into clusters of 64. With 4096 file servers per cluster, I needed only two clusters, but there was the potential to expand to three or more, as required. Backup looked as if it would be a bit of a problem. But I solved that in the usual way, by not having any. I did specify, however, that all the file servers be blessed, just as a precaution, which is more than is done at most sites. Printing was a problem. You can't put a typewritten label on a present, so I wanted to use laser printers to create the appearance of hand writing. But lasers can't handle colours, so eventually, I settled on flat-bed plotters as the primary output devices. The problem is that they are slow, so I had to have quite a lot of them. A label takes about three minutes to write; add another five for removing it and putting in a blank, and each plotter will turn out 200 labels per day. We would have about a month to make 2 billion labels. In the plotter room, therefore, there would have to be 300,000 plotters, with each group of four driven by its own plotter-server. The order data entry would mean keying in a peak of 200 million orders per day. Each elf could input 500 orders per day, so 400,000 data entry machines would be needed, networked using the same clustering system as the file servers. Finally, I had to think about the machine that would be used to sort and process the information. Clearly, something powerful would be required, and so the choice was easy - since the most powerful machine around was the 386, it had to be one of those. The overall design, therefore, called for half a million IBMs knitted together into the largest LAN ever built, connected to 2 terabytes of disk storage, and using 300,000 plotters for output. I was very pleased with the final design, and had it typed up by one of the fairies, and took a copy to Sir Sangrethta. "How much will all this cost?", he asked. "About four billion pounds", I said, "which works out at two pounds per customer." "Cheap at the price", said Santa, handing me a cheque with more noughts on it than I could comfortably carry. I went down to the leprachauns to have it converted into bags of gold, and then got the goblins in Purchasing to raise an order. I'm not sure exactly which box-shifter got the order but I'm sure that celebrations must have gone on far into the night, probably until dawn. By the time the sun rose, and the gold had changed back into straw, we'd got the machines safely under the Pole. It's a tough life, being a box-shifter. Then my problems really began, as we had to set up the hardware, and at the same time write the software and begin to enter the data. Have you ever tried to teach Turbo Pascal to a million fairies? The trouble is, they haven't got a very long attention span, and no notion of working systematically. All they seemed to want to do was wave their wretched wands about. But I got them working together, and soon they were churning out code at a great rate. Somehow, we got the elves all trained in keyboarding, in spite of the problem with their hands being too small. The reindeer were the worst. I don't know how I managed to overlook the fact that you can't operate a keyboard with hooves, but we managed to set up a system using a touch sensitive screen, which they regularly broke (it was the iron horseshoes that was the problem) until someone (I think it was Rudolf the Red) suggested that they use their noses. By the time I finished training the maintenance dwarves, whose idea of fixing things seemed to be to hit them with a hammer, and handed the system over to the team of pixies who would be running it and doing the software maintenance, it was the middle of November, and the system had to go live without any proper testing. I collected my fee from Santa (I insisted on five-pound notes, having seen what happened to fairy gold) and went back to London, expecting to get a call at any time. The design is sound, but it is a worryingly large system, and I'm not sure how the blessings on the disks will stand up to actual use. I'm also worried about some of the magic that the fairies used in the software engineering, and if those dwarves start bashing around with their hammers ... Well, you see the problem. The job was rushed, the documentation is inadequate, the testing was non-existent. I'll be very surprised if it all hangs together for this year's peak, and it really wants re-doing properly after the rush is over. If I were you, I wouldn't rely on getting your usual annual visit this year. If you want your little customers to be sure of getting their presents, you should go out and get them yourself, and pretend that they've come from the usual source.
Sunday, 22 December 2013
Another computer for Christmas
This was also written about 30 years ago.