More questionable statistics

I hate bad statistics. I posted before about the statistics of the Hitchens vs Blair debate. Statistics are important to make good decisions. Statistics are vital in things like medicine. Statistics help us see whether a drug is or isn’t safe. Some of the worst examples in medicines are because mistakes are made in the statistics. They are also used to decide great areas of public life – whether we change policy and even legislation. It is therefore important to have statistics that are accurate and looked at properly.

There are a number of questionable statements made yesterday and today on the radio that really should have a health warning against them.

Firstly, there was a discussion on the PM programme last night on Radio 4 about how often we should have an MOT test. The government is thinking of going to the


continental system  of no MOT before 4 years and then an MOT every 2 years and no longer annually. Now I don’t have much to say about this per se. What I do find difficult is when the person on the programme defending the current system says of vans:

At the moment about 50% fail [about van MOT’s], which basically means 1 in 2 vans you see on the road is probably an MOT failure.

Now. Lets look at this. The implication is firstly that all MOT failures are major events. Anyone who has an older car will know that this is not true. For example a car does not require ABS brakes to be road worthy. But if you had ABS installed and it doesn’t work then it can fail (even if you know it doesn’t work and you are not depending on it).

But more importantly, this is statistical nonsense. Firstly, there are all the newer vans, under 3 years old, that are far less likely to fail. Then each year every van over 3 years has to have an MOT that means that it cannot go back onto the road unless it passes. It means that therefore there is a failure rate over the course of the year where vans get up to a 50% failure rate. Now this may not be a straight line graph – it could be a slow start and then a speeding up of a rate of failure (more likely) or a faster rate and then a slowing down of failure.

For the sake of the argument lets assume a straight line failure rate as a middle road. That would mean that to reach 50% of the van population you would need about a 4% failure rate per month of vans.

According to the DVLA there appear to be about 2 million light vans registered in this country. In addition there have been 220,000 vans bought new in this country this year. Assume that this is the case each year and we find 1/3 of all vans are less than 3 years old.

So the maths would be % of new vans failed plus the no. of months that a van is away from MOT x monthly failure rate x %of vans

So if failure rate of new vans is near zero (as they are likely to be under warranty) then the sums are 6 x 4% x 2/3 = 16%. A third of the man on the radio’s estimate. So the answer when he says “probably 1 in 2 is an MOT failure” the answer is “probably not”.

Oxford University

Then there has been an article in the Guardian today by David Lammy an ex education minister in the last government. He makes some valid points about the fact that Oxbridge does not reflect the country – BUT – the statistics that he uses really are questionable. There are two articles worth reading here and here. What David Lammy is right to say is that it is a disgrace that Oxbridge candidates are drawn from such a small pool when there are so many others who are every bit as bright.

What the statistics that he quoted say far louder than anything else is that the great pupils around the country who are capable of going for Oxbridge aren’t being encouraged by their schools. One of my daughter’s friends in Merseyside was every bit as bright but has gone to a local college rather than applying to Oxbridge. David Lammy says that no-one in the borough of Knowsley has applied to Cambridge since 2003. But  why isn’t the education authority addressing this? Why aren’t they demanding that schools encourage and support candidate?

Why are we satisfied with a school system that means that young black people do badly at A level or indeed working class white people? The conclusion that I draw from David Lammy’s piece is that the statistics say that the real problem lies earlier in our education system. The BBC are reporting how far we are falling in the international rankings. I find this totally unsurprising when many state schools take only 9 GCSE’s (and most top universities require 9 A*’s to even consider you) and too often they are not academically rigorous enough (combined science or BTEC’s). Unless we deal with this then we will have fewer and fewer students capable of coping with Oxbridge.

So lets be driven by good statistics. Lets look at why the school system is not working in parts of the country. As a governor of a local school my constant refrain is to ask how we can help those groups who are performing least well. In terms of those on free school meals its results are “outstanding”. What “outstanding” mean is in the top few percentage as gauged against other schools and how their students on free school meals achieve. It does not mean students being in the top few percent in the country as a whole. We haven’t cracked this problem. We are nowhere near solving the problem. But to put this purely onto Oxbridge (which has its fair share of the blame) is ludicrous and isn’t supported by the statistics.

Update 9th December

There is a great article by the statistical magazine significance regarding David Lammy’s claim about Oxbridge entrance (which he repeated today).


One comment on “More questionable statistics

  1. chris
    December 7, 2010 at 7:40 pm #

    I am beginning to think that we have to much time on our hands, you for writing it and me for reading it. However I loved the van calculation, also there is the additional point about the validity of the MOT, like all functional tests its invalid the moment the test is finished, as something may break (such as a headlights bulb) which is a MOT failure….

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s