Okay, back to some edumacation.
Apologies for the length, but I wanted to make a primer for antibody tests so everyone can actually understand how they work and how useful or LESS useful they may be to you depending on your situation, especially due to the number of tests currently available on the market. The goal is to keep you safe so understanding how these tests are applied in the real world is essential.
I chose three companies who have received Emergency Use Authorization by the FDA: Abbott, Roche, and Diasorin.
At first glance, all 3 seem pretty darn good:
Abbott: Sensitivity 100%, Specificity 99.5% Roche: Sensitivity 100%, Specificity 99.8% Diasorin: Sensitivity 97.4%, Specificity 98.5%
Remember: Sensitivity and specificity are characteristics of the screening test itself. These values demonstrate how well the screening test does when they are compared to a gold standard, when you know a person has or does NOT have the disease. Also keep in mind these stated values have not been independently verified as far as I know, so trust of the company is essential.
There are two main types of antibodies these tests may look for: IgM and IgG. IgM production may form before resolution of the disease and are the first antibody our body produces to fight infection. Over time though, IgG production increases and eventually fully replaces IgM as the predominant antibody and is long-lasting to prevent future disease. At this time, we do not know how long these antibodies last, how effective they are in preventing future re-infection, nor what are considered sufficient levels of IgG to fight re-infection.
As a result of this delay in production, these tests could be negative one day but positive a week later. And both could be correct because your body may just not have produced the antibodies by the time you took the test. This is why these tests are most effective when used in patients where you have knowledge of being infected, and why it gets trickier in the asymptomatic population.
Roche tests for both IgM and IgG antibodies, while Abbott and Diasorin test for only IgG. I have not found anything to confirm this, but it is plausible the reason Roche also tests for IgM is an assumption if this is positive, eventually IgG antibodies will be formed at a later time.
Another important thing to note: these tests do NOT distinguish between non-neutralizing and neutralizing types of antibodies found in a person's blood. Antibodies can form to numerous protein sites on COVID-19, so not all antibodies are created equal - hence why vaccines are being targeted towards certain antigen sites.
Okay! So let us put this all into perspective. And this little exercise is something you can mimic anytime you see a new test come out, post-COVID-19 as well. Facebook does not allow me to insert tables, so please look at the image of the tables and follow along - I hope it is not too cumbersome.
Example 1: Say you have 2000 patients, 1000 of which definitely have COVID-19 antibodies and 1000 of which you know definitely do NOT have COVID-19 antibodies - this indicates a population with 50% prevalence of antibodies. This is represented in Column 1.
Remember:
For sensitivity and specificity, look at columns.
For positive predictive value (PPV) and negative predictive value (NPV), look at rows.
For both Abbott and Roche, all 1000 of the patients WITH antibodies tested positive, so no false negatives - thus sensitivity is 100%. But with Diasorin, only 974 patients WITH antibodies tested positive, resulting in 26 false negatives as a result and hence a 97.4% sensitivity.
For Abbott, 995 of the patients WITHOUT antibodies tested negative, with only 5 false positives - thus specificity is 99.5%. For Roche, there were 2 false positives - thus specificity is 99.8%. But with Diasorin, there were 15 false positives - thus specificity is 98.5%.
On first glance, the number of false results may seem acceptable given each cohort consisted of 1000 people. And if the prevalence of antibodies is high enough, it does seem acceptable when looking at the tests PPV and NPV.
And PPV and NPV are what we ACTUALLY care about when determining the usefulness of a screening test. It tells us how likely a positive test means we actually are positive, and how likely a negative test means we actually are negative WHEN WE DO NOT KNOW OUR ANTIBODY STATUS.
Both PPV and NPV are extremely high for all 3 tests due to the prevalence being so high. In fact, both PPV and NPV are essentially identical to the tests sensitivity and specificity:
Abbott: PPV = number of true positives/(true positives + false positives) = 1000/1005 = 99.5%, NPV = number of true negatives/(true negatives + false negatives) = 995/995 = 100%.
Roche: PPV = 1000/1002 = 99.8% NPV = 998/998 = 100%
Diasorin: PPV = 974/989 = 98.5% NPV = 985/1011 = 97.4%
But COVID-19 antibodies are almost assuredly NOT that prevalent in the broader population. And their prevalence will vary depending on region and patient population tested. So let's take another example.
Example 2: This time, let us take the same population size, but set the prevalence of antibodies to only 5% instead - much more realistic than a prevalence of 50%. This is represented by Column 2.
This means out of 2000 people, only 100 individuals would have antibodies, while the remaining 1900 would NOT have antibodies. Notice the sensitivity and specificity did NOT change because again, those values are characteristics of the test itself.
But can you immediately notice the difference when calculating PPV and NPV?
Because the disease prevalence is so low, NPV in this case would be close to or equal to 100% for all 3 tests, meaning a negative result has an almost 100% chance of being a true negative. So you can believe a negative result in all 3 cases. That makes sense.
But the PPV has DRAMATICALLY changed for all 3 tests by decreasing the prevalence of disease to only 5%, and thus drastically altered the usefulness of these tests as screening tests:
Abbott: PPV = 100/110 = 90.9% Roche: PPV = 100/104 = 96.2% Diasorin: PPV = 97/125 = 77.6%
Ouch. Especially with Diasorin. It catfished you HARD.
Remember: PPV for these tests increases with increased prevalence of antibodies, and decreases with decreased prevalence of antibodies. This is why you cannot just look at sensitivity and specificity and say, "well, that looks pretty darn great to me!", and why you need to be knowledgeable regarding the test you take.
Roche, even with a much lower prevalence, still has a pretty great PPV. This means a positive result has a 96.2% chance of being a true positive, so if you test positive, you have a pretty great chance of actually having antibodies.
With Abbott, you can trust a positive result 10 out of 11 times, which is still pretty good, but that 0.3% difference in specificity between Roche and Abbott made a 5.3% difference in their PPV in the same population. That is significant. And could be consideration in using only Roche tests in cohorts of the lowest prevalence, and both Roche and Abbott as the prevalence increases - for example in NYC.
But Diasorin, well, its PPV, and its usefulness as a screening test, went to shit with this reduced prevalence. A positive result was wrong almost 1 out of every 4 times, and this is a problem if presence of antibodies is used at some point to determine immune status. False positives at this high of a rate are not acceptable, as almost 23% of those who tested positive would actually NOT have antibodies and would be at risk of contracting COVID-19. But here is the kicker: if you had a positive COVID-19 PCR swab and/or symptoms highly consistent with COVID-19, guess what? The prevalence of disease would be MUCH higher in this cohort.
And since an antibody test is MOST accurate when you belong to a population that has a high chance of having antibodies - eg those who actually had or likely had COVID-19, all 3 antibody tests would have an incredibly high PPV that closely approximate each tests' specificity. At least Diasorin can hang their hat on that.
So those who have never had symptoms of COVID-19 or tested positive for it, like me, the PPV declines as a result. But I got the Abbott test done anyway since my hospital made it available, and it seemed impossible I could not have been exposed to COVID-19 since I was taking care of the HIGHEST risk patients in the ICU.
But it came back negative. Which is what I expected, but still. Poop.
However, I can take solace in knowing if these tests were screening for sexiness, a true positive result for me would be a foregone conclusion.