Tuesday, 31 January 2017

Circling Banbury

There's a route that goes all around Banbury; 50 caches plus some bonuses. Yesterday, I went out to do that after several weeks of non-caching (I had a heavy cold, possibly flu). After so long without exercise, I was totally wasted by the time I got back home, barely able to walk.

The highlight of the day was the Banbury Cross webcam cache. I stood in the right place, with my bicycle, and took these pictures; these are screenshoots from my iPhone. I'm a bit blurry, but the bike came out well.

One important thing I discovered; my iPhone doesn't hold charge, barely at all. So I'm using an external battery now. But ladysolly's started talking about upgrading her iPhone, so I'll inherit her previous one.

Saturday, 28 January 2017

Simple backups aren't enough

I've said this before - it's not good enough to have a simple backup system that backs up your computer each day to the same place. Here's why.

"Data could not be recovered from backups, as the backup procedure kicked in shortly after the ransomware took root, and backed up copies of the encrypted files."

I'm glad to say that a lot of their data is on DVD or CD, and so is safe. But data for ongoing investigations is lost.

They aren't going to pay the ransom, and I think they're right. There's no guarantee that paying the ransom will get their data back, and they don't want to pay criminals.

So how should you do backups? Here's what I do. I do rolling backups. Yesterday, I backed up my main server to a server called Nigel. Tomorrow I'll back up to a server called Jezza. And there's two others like that. So if I suddenly discovered that my main server has been borked, and the borkedness has been copied to the backup, then I only need to go back to the backup before that.

So ask yourself this. If you suddenly found yourself in the position of the Cockrell Hill Police Department, would you be able to recover? If not, change your backup system.

And if you don't have a backup system, then you haven't been hearing what I've been saying for the last 30 years.

Tuesday, 24 January 2017

Maths isn't easy

Maths textbooks have been withdrawn after a ton of mistakes were found.

I have two comments to make on this.

First, you won't find anything like this in history or english literature textbooks. That's because in maths, an answer is usually right or wrong, and it's easy to tell the difference. In history, if you were asked what is 2+2, you could launch into a major essay about the post-modern interpretation of whatever, or the precise meaning of "+" in the age of diversity. In maths, any answer but "4" is wrong.

That's one of the reasons why maths isn't easy, but at the same time, we need to encourage good mathematicians. Why? Because a good mathematician can turn their hand to pretty much anything, but a poor mathematician is forever barred to anything involving computation. One of my pet hates is people saying "I'm not good at maths", (meaning, no good at simple arithmetic, because such people are ignorant of the joy of calculus) said with a light laugh, as if this isn't something to be deeply ashamed of.
When I was at school, maths was rated at about three times the importance of any other subject. You could see this in the number of hours devoted to it in the syllabus, and in the number of O and A levels you could get (I got three A levels in maths alone, plus one in physics which is mostly maths). But now?

My second comment is on proof reading.

Proof reading is *expensive*. You have to hire an actual person who actually comprehends what they are reading. So many books I read today are riddled with errors, and you can see that a spell checker has been run over the text (albeit imperfectly, there's still a few spelling errors) but riddled with homonym mistakes. And my guess is that these textbooks weren't proofread.

Thursday, 19 January 2017

How to fool 50 antiviruses

Subject: Fwd(98):the payment confirmation from In Hendrerit Inc.


Please see the payment confirmation attached.

You also need Word doc Password: 22lZ34

Winter I. Vazquez
In Hendrerit Inc.

Enclosed was an encrypted word doc file. I showed it to Virustotal, and 50 out of 50 products passed it as clean.

Because it's encrypted.

Inside the encryption could be anything. It could even be non-malware - fat chance. And because the password is given in the email, a foolish user might click on the attachement, give the password, and then what happens is whatever the criminal wanted. Ransomware is currently fashionable.

So how to deal with this?

I don't see how any scanner could handle this technique. The only answer is to A) use a word document reader that cannot run macros, or B) block all attachments.

My preferred method would be C), which is to block all attachments unless they can be "sanitised". In the case of a word doc, it would have all macros removed before giving access to the user. And in this particular case of an encrypted file, just strip off the attachment.

Thursday, 12 January 2017

2000 posts

My previous post was number 2000 on this blog.


Then I was at universtity, my favourite text books were the "Schaum outline series". They took you through a subject with worked examples, so you could see how it was done, and tons more examples so you could learn-by-doing.

I've been reading popular books on quantum mechanics, and I suddenly realised that I should get the Schaum book. So I went on Amazon and ordered it, about £5. Just so you know how cheap that is, 50 years ago, the typical textbook cost £25.

So while I was browsing, I saw the Schaum books on econometrics and on mathematical finance, and I thought about when I made the jump from maths to econometrics.

The big thing I discovered, which I don't think is widely known, is how inaccurate economic statistics are.  They are so inaccurate, that they really cannot support the weight of analysis that is put on them.

GDP figures, for example, are collected via a survey of a sample of companies. Consumer expenditure is measured via the "Family income and expenditure survey", with, as I recall, a sample of 7000.

And my best example of the problems with data, comes from population, which you'd think would be *very* accurate, since it's collected by a 100% census.

Now look at the population of Germany. If you want to use that in a time series analysis, there's a huge "gotcha". In 1964 (I might have misremembered the exact date) there's a jump of two million in the time series. But what isn't obvious, is that there weren't suddenly two million more Germans. The two million is because they started to include West Berlin in the numbers.

Before you do any analysis on data, you ought to understand how it was collected, and in what ways it's inaccurate. But I don't think economists ever do that.

Windows 7 reboots

Windows 7 reboots itself.

Probably not every day. It's 3:18 am, and it just did it. It put up a message first, to tell me it was going to. Why am I on the computer at 3:18 am? I'm a programmer.

Maybe there's a way to configure it not to do that.

Maybe I won't bother, I'll stick with linux.

Here's how to disable this misfeature.

Tuesday, 10 January 2017

I almost killed a person

I was driving along my lane. It was dark, and there's no street lights. A couple of cars came towards me, so I hugged the left side of the road. The headlights of the oncoming cars meant that I couldn't really see anything except their lights.

A moment after they had passed, I saw a pedestrian, on my side of the road, walking in the road (there's no pavement or verge at that point).

I saw him (or her) because he was carrying a torch, and I saw the torch flash, so I swerved away and avoided the walker.

If they hadn't been carrying a torch, I wouldn't have seen them at all. Would I have hit them? I actually don't know, but there's absolutely no verge at that point, and I was very close to the road side (because that's where I had to be owing to the oncoming cars).

Anyway, the lesson is this. If you're going to walk down an unlit road, in the dark, and there's no pavement or verge, be sure to carry a torch so that you can be seen.

Monday, 9 January 2017

Efficient programming

After you've written a program, you might spend some time optimising it. You might unroll loops, or write parts of it in assembler. And so on.

But the algorithm that you use is massively more important than any optimisations you do afterwards.

Two examples.

The first is from the antivirus I wrote. Someone once said to me "I couldn't write a program that just reads all the files, that runs as fast as your scanner." Yes, well. Certainly back then when we were talking about several thousand viruses, you didn't actually need to read the whole file to be able to be certain that none of the existing viruses were present. To get that, I had to analyse each virus to determine where in the file the virus had to be if it was there. And so I only needed to read part of each file.

I also did a three-stage process; the first stage scanned for two bytes only. That generated some "false alarms", but 99% of files needed no further examination, which made things faster.

And so on.

The second example is from a major database I run. Every six months or so, I have to run an indexing process, which usually takes several hours. I ran it recently, and it was nine hours. The problem is, the database has grown massively, and when I wrote the indexing program, I didn't think too hard about it.

Nine hours got me thinking, and I came up with a completely different algorithm to do the same job. I timed it - 45 seconds!

Friday, 6 January 2017


There are so many things that I should have been taught in school. Obviously teaching time is limited, but maybe instead of the entirely useless period of "Religious instruction", they could have had one period per week on "Things you need to know."

Maybe many of these should be taught by parents, but not all parents are as good as they could be. So what would I have in the syllabus of this course?

In Maths: how to estimate (how many bricks do you need to build a house?) and approximate.

In English: Replace "how to parse a sentence" with "How to write in a way that can most easily be read".

 And in the general "Life skills" course that would replace "Religious instruction":

How and why to clean your teeth (see previous blog post).
How to ride a bicycle.
Important things to know when driving a car.
Critical thinking and scepticism.
How to choose your bank, and how to open a bank account.
How credit cards work.
Alcohol and its effects.

And similar stuff.

Sunday, 1 January 2017

Cleaning teeth

When I was a lad, I was told to clean my teeth, but no-one explained why. Maybe dentists think that it's sufficient to tell me what to do, but if I'd known why, it would have made a big difference. When you know why you're doing soemthing, you do it more assiduously than if it's just one of the 613 commandments.

When I was a lad, if you had a cavity, the dentist would drill it out very throughly, removing quite a lot of tooth, then fill it up with a mercury amalgam filling. It turns out that this isn't the best approach; it meant that I had a number of almost hollow teeth, and in later life sometimes bits broke off. But that's the hand I was dealt. All in all, I'm OK with my teeth, but that's partly because I've been lucky to get a really good dentist.

Here's why you should claen your teeth. And how.

There's two reasons. One is acid. You get acid in your mouth from fruit. Eating fruit is a good idea, in moderation, but you don't want to leave the acid in your mouth to dissolve the teeth. Brushing helps with that.

The other reason is plaque. That's bacteria that grows on your teeth, which hardens into tartar. And plaque harms your teeth in three ways. A) it helps your teeth to get caries,  B) you can get Gingivitis (inflammation of the gums) and C) your gums don't like it, so they shrink away. This means that your teeth get longer (hence the expression "long in the tooth") but it means there's less holding them in place, which is obviously bad.

You need to brush your teeth each day, because when the plaque first gets into place, it's soft and easy to remove. If it's allowed to harden, then it can only be removed by a professional dentist (or dental hygenist). Your dentist will be able to recommend one.

I use an electric toothbrush. A manual is probably just as good if you use it right, but an electric is quicker. I use a Braun Oral-B, with AA batteries. I use AAs because if you buy the kind of toothbrust that is rechargable, you'll find that after a couple of years, the battery inside won't take a charge; if you use rechargable AAs then if the batteries won't take charge, you replace them.

I use two brushes (and because I'm lazy, I have two toothbrushes so I don't have to swap the brush heads). The first is the ordinary kind. The other is called "interspace". Instead of having a head that's half an inch across, the brushes converge to a point. This gets down to the gum, and to some extent between the teeth. The Oral-B part number is IP17. I usually buy these on Ebay; you can also get them from Amazon, Boots and probably lots of other places.

Brushes don't clean between your teeth, the "interdental" spaces. Some people use dental floss for this; my dentist told me that interdental brushes are just as good, and I find them a lot easier to use. I use the Tepe brand of brush; they come in a good range of sizes. In practice, I find that just two sizes will give me the range I need. Again, Ebay.

Listen to  Pam Ayres.