A couple of disclaimers first. Firstly - I do kind-of have a dog in this race; the McAfee antivirus still uses the engine that I designed and implemented. But I'm sure that it's had major additions since I let go of it about 20 years ago! Secondly - what do I use? Well, I use Linux, and I don't run an antivirus.
And now a definition. I do actually know the definition of "virus", and it's become irrelevant. I well remember at a conference once, a user speaking from the floor said "I don't care whether you call it virus, trojan, malware or whatever, I don't want it on my systems."
Today, the main threat in this area comes from trojans, delievered by email or by someone accessing a web site. My guess is that viruses are now well below 1% of all malicious software that's in the wild.
So the problem has changed. The key fact about a virus is that it self-replicates, and once you have a specimen of it, and you've analysed it, you know exactly what it does and what byte-sequences can identify it. Even the most difficult polymorphic (they change their byte-stream in each virus instance) could be reliably detected, using an emulator.
But that's not the case with a trojan. If you send out ten million spam emails bearing a trojan, you can make each instance of that trojan different, so that it isn't possible to detect them all with a single byte sequence. And because the virus researcher doesn't have access to the generator of these files, he can't predict the limits of variability. This is called "server side polymorphism". And it's made the field a lot more difficult.
Imagine such an email generator, spewing out ten million emails containing the trojan, each one very different from all the others. Suddenly, the AV industry has 10 million things to detect. How long will it take to catch up? And how many false alarms will there be when they send out the detector to their customers.
The other game changer has been the internet. 20 years ago, the internet was not an issue. Viruses spread via floppy-sharing and file-sharing. I used to reckon that between the time that I saw a virus, and the time that an ordinary customer would see it, might be a year. Now? It's the other way round. The trojan creator sends out his ten million emails,, the AV company sees it shortly after that.
So there's been an increasing dependence on heuristics. That means, not using a byte-sequence to detect something nasty, but instead, analysing the things that the file does (read files, write files, for example) and deciding whether that is likely to be malicious.
So - a program reads a file, and replaces it with that same file, only encrypted with a strong encryption system. Is that malicious? It is if it's Cryptolocker, it isn't if its one of a zillion good encryption systems. To decide whether it's malicious, you have to understand what it does (which is difficult enough) and compare it with what the user expected it to do (and computers can't read minds).
I've had a look around at various studies on the effectiveness of antivirus. I'm not going to name names, because none of them come up smelling of roses. I avoided looking at reports by companies that have a product to sell, or who are under common ownership with companies selling an AV. And yes, I know very well the difficulty of testing antivirus products, having been on the wrong end of some very mis-designed tests.
cyveillance.com reckons that AV products take from 2 to 27 days to detect new threats. Krebs says that the average detection rate for email-based malware is 24%
Imperva say that it takes from 1 1/2 to 4 weeks for an AV to catch up with a new virus.
And then there's false alarms. AV Comparatives shows all the vendors giving false alarms, some less than others. A false alarm can lead to all sorts of problems.
It's a sad situation. The defences against modern-day malware have large holes. That's not to say you shouldn't use some defence; every problem blocked is one less problem to sort out.
But one study I read thought that there was a big overspend on AV, at the expense of other security defences, and they thought that this might be down to "compliance". Or, in other words, some companies spend too much on AV because there's a box they have to tick.
Some people opine that user education should help. I think it would if that were possible. I don't think it's possible. I don't think users are interested in this stuff, and think that it's the responsibility of the IT people. If you drive on any motorway and see all the people driving far too close to the car in front, you see how seriously people take security, even when it's their personal well-being that's at stake.
I don't have an answer. Fortunately, it's not really my problem. I'll continue to use Linux, and hope that the bad guys don't take aim at it. My guess is that the main problems will continue to be on various levels of Windows, and that in the future, mobile phones will become an issue.