False positive tests

Mon, 28 Sep 2020 14:00:43 GMT

28th September 2020

There has been a lot of noise about false positive COVID19 tests in the news. So, I thought I would try to explain what it all means. Or do my best anyway.

There are two measures in most medical screening tests which are usually defined as the ‘sensitivity’ and the ‘specificity’ of a test. In my opinion, these two words are far too close together in sound, so they are very easy to mix up in your brain.

I find it easier to think of the accuracy of test results in this way.

False negatives
False positives

A false negative is a result which informs someone that they do not have a disease, when in fact they do.

A false positive is a result which informs someone they do have a disease, when they don’t.

Ideally a test should never give a false negative (100% sensitivity) nor give a false positive (100% specificity). There is no known test that does this. In general, there is a trade-off going on between these two measures.

By which I mean, if you aim for 100% sensitivity, the specificity often falls away – and vice-versa

For example, in cancer screening the primary objective is you must never miss a case. So, the sensitivity rate is set very high. By definition the rate of false negatives is very low.

A shadow on the breast, a few strange cells here, a few strange cells there – ‘that might be cancer, better to be safe than sorry. Don’t take the risk’. Positive cancer test.

To put this another way. The fear of missing any cases of cancer results in the number of false positives being high. This raises the question with COVID19. Is it better to underdiagnose – many false negatives. Or over diagnose – many false positives?

Note I am talking here primarily about the naso-pharyngeal swab tests (i.e., antigen tests) which are used to see if you have the virus NOW and not the blood (antibody) test done which may be done later to see if you have ever had the virus.

This issue does not seem to have been discussed. If you want to prevent spread of COVID19, you would presumably want very few false negatives in these swab tests. Otherwise people will be told they don’t have the disease – when they do – and happily go off spreading it around. Equally, you would be relaxed about false positives. People would isolate when they don’t need to, but not a great health issue.

Weirdly, however, this does not seem to be the case.

COVID19 false negatives

With COVID19, there are a lot of false negatives, with some studies quoting figures as high as 50%. That is, half of those told they are not infected with COVID19, are probably infected¹. A systematic review got figures between 2% and 29%. So, we could call that an average of 16%?

As you can see, these figures are clearly all over the place. This is in major part because there is no ‘gold-standard’ COVID19 test. By which I mean that we do not have a ‘test of tests.’ Namely, the expensive and time-consuming test by which we absolutely can know if someone truly is infected. The test against which your ‘field tests’ can be calibrated/verified.

Indeed, currently, there is no current agreement as to what ‘infected’ means with COVID19. Does it mean finding viral particles in the nose, sputum, or throat – or all three? Does it mean finding viral particles in these places, and also isolating it in the bloodstream, or lungs? Does it mean finding evidence of antibodies specific to COVID19 two to three weeks following ‘infection?’ Or what? It would be nice to know.

COVID19 false positives

More troubling, right now, than the very poor sensitivity of COVID19 testing (high number of false negatives) is the knotty question of how many false positive tests there are? This is important, because we are told that cases are rising and rising as we suffer a ‘second wave’ of COVID19.

However, if we have a high rate of false positives, then the rise in ‘cases’ could be driven by a rise in testing – and nothing else. And you don’t need a high percentage of false positive tests to do this. If the false positive rate is as little as just one per cent (1%) this means the majority of people told they are positive for COVID19, do not have COVID19!

I know that most people find this a difficult one. It goes like this.

First, you have to know the estimated prevalence of the disease in the community. That is, the total number currently infected. Last time I looked it was one in nine hundred. For the sake of this calculation I shall call it one in a thousand. [Or, to put it another way, sixty-seven thousand people in the UK (population 67 million) are currently infected with COVID19].

Using this one in a thousand figure. This means, if you randomly tested ten thousand people, you would expect to find ten COVID19 cases [forgetting the false negatives for now].

On the other side of the coin. If the false positive rate is one per cent, you would have an additional one hundred false positives cases.

10,000 x.01(1%) = 100

Putting this another way. With a prevalence of one in a thousand, and a false positive rate of one per-cent you would have ten true COVID19 positive cases, and ninety false positives. Ergo, the vast majority of people told that they have COVID19, do not. Is this actually happening?

There is heated debate. As in much heat and little light.

In order to shed a little light, I have been communicating with a senior scientist in a COVID19 facility who feels things have gone very wrong. Below is his take on the false positive situation, from a couple of weeks ago. It is highly technical, but for those who can follow it, I think the author makes some critical points. I have not named him for, were I to do so, he would almost certainly land in very hot water. However, the references are verifiable.

1: https://www.bmj.com/content/bmj/369/bmj.m1808.full.pdf
What do positive SARS-CoV-2 RT-PCR tests mean? (Absolutely Nothing!)
The Cepheid Xpert Xpress SARS-CoV-2 RT-PCR test is the “Gold Standard” COVID-19 antigen test used in our laboratory. The specificity of this test from the manufacturer’s package insert¹. [Here referred to as negative percentage agreement or NPA) is 95.6% or 0.956 when expressed as a fraction].

I don’t know about other RT-PCR tests, but I imagine the specificity will be similar for all widely used commercially available kits.

The specificity of a test is defined by the equation:

SP = TN / (TN + FP)

Where SP = specificity, TN = number of true negatives, FP = number of false positives. TN + FP = the total number of tests carried out.

Now the latest Government figures from Monday 7^th September state that 350,100 tests were carried out and 2,948 people tested positive ². So, if we apply the above equation to our PCR test and the Government’s figures, we get:

0.956 = TN / 350,100

Therefore, the number of true negatives is:

TN = 350,100 * 0.956 = 334696

Therefore, the number of false positives, FP we would expect from 350,100 tests is:

FP = 350,100 – 334,696 = 15,404

This is more than five times the number of positive tests reported, which means we cannot have any confidence that any one of those positive tests represents a genuine case.

What these figures show is that it is totally inappropriate to use RT-PCR as a screening test for a virus in an asymptomatic population when the prevalence of the infection is very low.

Even if there were a test with 99% specificity, you would still expect to get 3500 false positives from performing 350,000 tests – which is still greater than the number of “cases” reported. When the number of “cases” is lower than your rate of false positives, then a positive result on its own is virtually meaningless.

The PCR test is best utilized as a diagnostic test to confirm the diagnosis of an infection based on clinical signs and symptoms. It certainly should not be used as a screening test when there is low prevalence of disease and should NEVER be used as the sole determinant in the diagnosis of a case.

One source of false positives is the persistence of fragments of viral RNA long after a patient may have recovered and is no longer infective. These fragments will be amplified by PCR and will give a positive result that is indistinguishable from a genuine case. We’ve had a patient whose swabs have been testing positive in our lab every week for over 3 months!

Non-specific amplification is another possible source of false positives. The nasopharyngeal swab samples are “dirty” samples: they are full of bacterial, fungal, other viral, and host DNA and RNA. Some of these will have high percentage sequence homology [NB homology basically means a similar sequence of base pairs- my words] to the gene sequences targeted by the PCR assay and these can also be amplified. The risk that this may have occurred is higher if the positive test has a very high Cycle Threshold (Ct) value – say 35 or above.

Recently, it has come to my attention that one of the primers – an 18-base primer for a region of the RdRP gene – has exact sequence homology with a region on human chromosome 8 ^3,4.

So, if any laboratory uses a PCR assay with that particular primer, they’re likely to get a lot of false positives!

Politicians and Health Officials are basing their numbers of cases entirely on the results of these tests, which are not fit for this purpose.

They are then using these figures to terrorise the population, and to justify decisions to impose local lockdowns, and increase nonsensical general restrictions which are having a massive impact on people’s lives and their health, and also on the economy, particularly hitting small businesses hard.

Dr. Malcolm Kendrick

False positive tests