Dr Rodney Richards on HIV Testing

This is a letter to the BMJ by Dr Rodney Richards:

http://bmj.com/cgi/eletters/326/7381/126/e#30151

*Practicing in different worlds, the two groups of scientists see different things when they look from the same point in the same direction. ... Each group uses its own paradigm to argue in that paradigm's defense...the status of the circular argument is only that of persuasion. It cannot be made logically or even probabilistically compelling for those who refuse to step into the circle.* (Thomas Kuhn. The Structure of Scientific Revolutions, 1962.)

Peter J Flegg suggests that David Rasnick's critique of problems with the HIV tests (HIV antibody test is the Achilles heel of AIDS, Inc.) is lacking credibility because he relies on "quotes [from] several sources, most of which are either not peer-reviewed or which are so old as to have been superceded by new data (of which he is no doubt aware but prepared to ignore)."

I would like to suggest that information taken from package inserts for diagnostic testing products represents perfectly valid reference material for scientific inquiry and discourse. Even though this information is not formally "peer-reviewed," it is thoroughly reviewed by panels of experts in the field of HIV testing at the United States Food and Drug Administration (FDA) prior to product release, and it is certainly not "old."

Flegg attempts to discount the information Dr. Rasnick quotes from these package inserts by asserting: "Virtually every diagnostic assay will have a legal disclaimer of some sort specifying that the tests should not be the sole basis on which a disease is diagnosed." While it is true that manufactures tend to disclaim their products with regard to the "intended use," they have no liability associated with physicians who choose to use their products "off label" for other purposes. Relevant to this is the little known fact that none of the 30 so-called HIV tests currently approved by the FDA (1) claim to be intended for use in diagnosing infection with HIV in the first place. As such, the notion that statements pertaining to the use of these tests for diagnosing infection with HIV might be cautiously vague or understated due to liability considerations is without merit.

The inappropriate use of antibody tests for the purpose of diagnosing infection with HIV can be traced back to 1987, when the US Centers for Disease Control (CDC) declared: "The presence of antibody indicates current infection." (2) Interestingly, the CDC has never offered any reference to any scientific study to substantiate this claim. Indeed, a substantial proportion (and in some studies the majority) of antibody positive individuals have no evidence of infection whatsoever when their tissues are subjected to the accepted gold standard of HIV culture analysis, which in principle can detect a single infectious viral particle. Nevertheless, the above uncorroborated proclamation from the CDC was quickly accepted as fact and has formed the foundation of all subsequent diagnoses of HIV infection; including the most recent UNAIDS/WHO estimates of 40 million current global infections and 25 million cumulative AIDS deaths. (3)

This is particularly disturbing when one considers that the manufacturers and the FDA explicitly warn against the use of these tests for diagnosing infection. In fact, in the very same year that the CDC endorsed the use of antibody tests for diagnosing infection (1987), the FDA announced, "The significance of antibodies [to HIV] in an asymptomatic individual is not known." (4) Furthermore, according to manufacturers of ELISA tests currently on the market, "The risk of an asymptomatic person with a repeatedly reactive serum sample developing AIDS or an AIDS related condition is not known." (5,6) And even the manufacturers of the so-called confirmatory Western Blot [WB] assays currently on the market, caution, "The clinical implications of antibodies to HIV-1 in an asymptomatic person are not known." (7)

In fact, according to the manufacturers, one should be cautious using these tests even for the purpose of diagnosing antibodies to HIV, let alone infection. According to the manufacturer of the most widely used ELISA, "At present there is no recognized standard for establishing the presence and absence of HIV-1 antibody in human blood." (5) And even the manufactures of WB assays, which supposedly confirm the presence of antibodies (not HIV), inform us that, "A sample that is reactive in both the EIA [ELISA] screening test and the Western blot is *presumed* to be positive for *antibody* to HIV-1." (8) (*emphasis* added)

Well aware of the fact that these tests will ultimately be used to tell people they are infected, the manufactures do warn, "A person who has antibodies to HIV-1 is *presumed* to be infected with the virus." (5,6,8) (*emphasis* added) However, the manufactures themselves will only go so far as to declare, "A positive result *may* indicate infection with [HIV]." (7,8) (*emphasis* added) And this statement pertains to samples that have been "confirmed" positive by WB testing! One could equally well declare, "a positive result for antibodies to herpes simplex virus *may* indicate infection with HIV."

All of this is further complicated by the fact that various institutions and countries use different definitions of what combination of antibody test results should be considered sufficient to hand out declarations of infection. Some countries consider a single ELISA sufficient, while others might require two or three different ELISA tests. Yet other countries or institutions insist their unsubstantiated declarations of infection be based on follow-up WB testing; and even in this case, currently used rules (interpretive criteria, or evaluation criteria) for deciding what combination of bands constitutes a positive result on WB vary dramatically. Consider a person with the following common WB banding pattern: (p24 + gp160). Should this person be told they are infected? According to the criteria of Australia, France, the UK, and the WHO, the answer is, "no." In contrast, according to the criteria of the Germany, the CDC, the Ugandan Viral Research Institute, and the National Institute of Virology (NIV) in South Africa, the answer is unequivocally, "yes!"

Flegg asserts such discrepancies in WB interpretation represent "historical inconsistencies." It is true that various institutions have changed their interpretive criteria over the years, however, as illustrated above, it is not true that the discrepancies have disappeared. Institutions within the USA appear to be closing in on a consensus as to what constitutes a positive WB, however, the USA represents only about 2.5% of the currently estimated 40 million global infections. Discrepancies in WB interpretations from institution to institution and country to country are just as dramatic today as they were ten or fifteen years ago. Unfortunately, until such time that we have an international standard for scoring WB results, any comparison of data from study to study, country to country, or year to year, will remain compromised. However, it cannot be overemphasized that even if the global research community were to eventually agree on a common set of rules for scoring WB, that still doesn't mean the results can be used to diagnose infection with HIV. According to the manufacturer quoted above, the best we can do with a positive WB declaration is to *presume* it is positive for antibodies, which *may* indicate infection with HIV.

Some experts will argue that their declarations of infection are valid because the majority of individuals who have been told that they are infected also test positive on Viral Load (VL) tests; which detect fragments of RNA or DNA supposedly unique to the virus. However, according to the manufacturer of the only FDA approved PCR test on the market, their test, "is not intended to be used...as a diagnostic test to confirm the presence of HIV infection." (9) Other experts have warned, "the trend toward earlier and more aggressive treatment of HIV infection has lead to the inappropriate use of these [VL] assays," and that, "viral load tests for HIV-1 were neither developed nor evaluated for the diagnosis of HIV infection." (10) According to others, this is because, "there is no 'gold-standard' laboratory test that defines the true infection status, and a true positive PCR test cannot be distinguished from a false positive." (11)

Unfortunately, there is a systemic illusion in the research community that the magnificent Sensitivities and Specificities reported in the contemporary literature (for various HIV test kits) pertain to the tests ability to diagnose the presence or absence of HIV in a sample. In fact, these reported accuracies represent nothing more than measures of concordance between various antibody test kits; none of which have been validated or approved for diagnosing infection with HIV. Since all of these tests are based on the same molecular principle of antibodies in a particular sample binding to prepared antigens in a test kit (be they on plastic plates, beads, microspheres, or nitrocellulose strips), it is not surprising that concordance between various test kits is quite high. However, even a 100% concordance between two different tests does not justify the use of either for purposes other than what they have been validated and approved for.

In summary, there simply is no laboratory test that has ever been approved for medical use by the FDA that claims to be able to diagnose the presence or absence of HIV in any sample, with any degree of accuracy. Until we have such a laboratory test, debates over issues such as whether or not HIV is sexually transmitted are likely to go unresolved. A validated and approved test for diagnosing infection with HIV would undoubtedly serve as an invaluable contribution to our understanding of AIDS, and to our ability to effectively manage the millions who suffer its effects.

References:

1. Licensed / Approved HIV, HTLV and Hepatitis Tests. http://www.fda.gov/cber/products/testkits.htm

2. CDC. MMWR 1987; 36:509-15.

3. UNAIDS/WHO. AIDS epidemic update December 2001; UNAIDS/01.74E - WHO/CDS/CSR/NCS/2001.2 (www.unaids.org).

4. Suzan Cruzan. FDA News Release April 30, 1987; P87-11.

5. Abbott Laboratories, Abbott Park Il. Package Insert for Human Immunodeficiency Virus Type 1 [Abbott] HIVAB[TM] HIV-1 EIA. US License No. 43, January, 1997.

6. Genetic Systems Corp., Redmond WA. Package Insert for Genetic Systems HIV-1/HIV-2 Peptide EIA. US License No. 978, Revised February 2000.

7. Cambridge Biotech Coirp., Rockville MD. Package Insert for HIV-1 Western Blot. US License No. 1063. June 2, 1998.

8. Epitope, Inc., Beaverton, OR. Package Insert for OraSure[R] HIV-1 Western Blot Kit. US License No. 1133, January 10, 1996

9. Roche Diagnostic Systems, Inc., Branchburg, NJ. Package Insert for Amplicor[R] HIV-1 Monitor[TM] Test. US License No. 83088, 1996.

10. Rich JD, et al. Ann Intern Med 1999, 130:37-9.

11. Sheppard, HW, et al. JAIDS 1991; 4:819-23.


Response to Dr Flegg in BMJ:

*The superior physician listens to the voice, the common physician observes the color, the inferior physician feels the pulse.* (T'ien-t'ai, Chinese scholar, 594 a.d.)

In response to my clarification that there currently exists no approved laboratory test for diagnosing infection with HIV, Peter J Flegg emphasizes: "[The clinician] should use all available information such as history of the illness, examination findings and results of laboratory tests in order to reach a diagnosis." In my opinion, this represents the spirit of the "superior physician" quoted above; and I have no issue with any physician who chooses to diagnose or treat any illness based on their clinical experience, with or without additional information from peripheral laboratory tests. I personally regard a physician's clinical experience to be far more valuable than any diagnostic test could ever hope to be.

However, Flegg goes on to suggest that when it comes to HIV, clinical information is not necessary to reach a diagnosis of infection: "When one is faced with the clinical scenario of a patient with repeated positive HIV antibody assays, positive confirmatory Western Blot, HIV viral load of 58000 copies/ml, Subtype B infection, no resistance mutations, CD4 lymphocyte count of 26/ml, it hardly requires the inputting of clinical information that he has thrush and Pneumocystis jiroveci pneumonia to reach a diagnosis of HIV."

The justification for his position on this issue is that: "The post- test odds of the first test become the pre-test odds for the second test, and so on, arriving at a final post-test probability that enables a diagnosis to be reached with a high degree of precision." While this is a true statement, it is important to note that none of the laboratory tests listed in the above scenario claim to be correlated with the presence or absence of HIV in a sample with any degree of probability whatsoever.

According to the manufactures of antibody tests, the best we can do with a sample that tests positive on ELISA and WB is to "presume" it is positive for antibodies to HIV. What precisely is the post-test probability that such a sample is infected with HIV? Is it 99.9%? Is it 5%? The answer is, we don't know because it has never been determined. It is unknown.

The manufacturers of FDA approved viral load (VL) and HIV resistance tests make no claim that their products can be used to predict the presence or absence of HIV in a sample with any degree of probability whatsoever; and the package inserts for both of these tests explicitly warn their products are "not intended to be used...to confirm the presence of HIV infection." (1) In other words, the post-test probability that a sample is infected with HIV cannot be determined using these tests. It is unknown.

The FDA has never even approved a sub-typing test for HIV, let alone one that claims to be correlated with infection. And finally, laboratory methods for determining CD4 counts are neither approved nor specific for diagnosing infection with HIV. As such, the post-test probability that a sample is infected with HIV cannot be determined using these tests. It is unknown.

Since the pre- and post-test probability that a sample is infected with HIV is unknown for every single test listed in the above clinical scenario, I assert that the combined probability is also necessarily unknown. Perhaps this is why the manufactures of these tests along with the FDA also claim the "significance of antibodies to HIV in an asymptomatic individual is not known."

Flegg asserts I am wrong in saying some countries use a single ELISA for the purpose of diagnosing infection with HIV. In fact, the use of a single ELISA for diagnosing infection in certain populations is a recommendation put forth by UNAIDS/WHO. According to their 1997 revisions, a single ELISA is sufficient "to confirm the clinical diagnosis of [HIV in] individuals meeting the WHO criteria of stage III or IV of HIV infection and when the HIV prevalence in the sample population (e.g., patients from a tuberculosis ward) is above 30%." (2) This report also recommends that in cases where two ELISA algorithms are used for diagnosing infection, persons with stage III or IV conditions should be considered infected even if they test negative on one of the two ELISAs.

In other words, a single positive ELISA is considered sufficient to tell persons in the less developed World that they are infected with HIV, provided they are experiencing conditions such as weight loss of more than 10%, chronic diarrhea, prolonged fever, oral candidiasis, oral hairy leukoplakia, pulmonary TB, or bacterial infections (Stage III conditions). (3)

Flegg goes on the assert I am, "wrong to state that someone in Australia, France or the UK showing bands to p24 and gp160 would be told they are not infected." Had I made such a statement I would be wrong. However, I actually said such a person would not be told that they "are" infected. Indeed they would be considered indeterminate and therefore receive the benefit of follow-up testing, which may well confirm them to be uninfected. In contrast, had such an individual lived elsewhere in the world, they would have been declared unequivocally infected and therefore understandably never subjected to retesting. Is this a likely scenario?

Flegg suggest the likely outcome of follow-up testing for individuals initially WB indeterminate is either an eventual positive result (representing early seroconversion), or an "unusual" persistently indeterminate result. This may well be in the case for individuals in risk groups living in industrialized nations, however, the situation is dramatically different in the developing world.

For example, of 63 pregnant women in the Congo who tested positive on ELISA and indeterminate on WB, and who were retested 3 months later, "[t]wenty-five (41%) of those retested became negative, and 36 (59%) remained indeterminate." (4) Only two women (3.2%) in this study developed positive Western blot patterns. According to the authors of a more recent study using blood collected from Ethiopians in various community cohort studies: "Of 31 WB assays initially indeterminate...and with follow-up samples, 29 (93.5%) became negative when retested subsequently while 2 (6.5%) remained indeterminate for more than a year and were thus considered negative." (5) In other words, 100% of individuals with WB indeterminate results in this study were found to be uninfected on follow-up. According to data in this study, had these uninfected individuals lived in the USA where the CDC criteria is used, 19.4% of them would have been declared infected. This being the case, none of these individuals would have ever come to find out that they are currently negative for all bands on WB.

These examples vividly illustrate how discrepancies between institutional criteria for scoring WB results can have a profound effect on whether or not someone is told they are infected or retested. Since various ELISA testing algorithms are selected and tuned to concord with these various institutional criteria for scoring WB, they are likewise predisposed to this same phenomenon.

Flegg contends that dissidents hope "by pointing out tiny inconsistencies they can persuade themselves and others that the whole concept [of HIV/AIDS] is inconsistent and therefore mistaken."

According to UNAIDS and the WHO, 25 million humans on this planet have died from infection with a deadly virus, and another 40 million are estimated to be currently infected. Yet, after 20 years of research supported by $100 billion in public funds, we still don't have a have single diagnostic test approved by any government in the developed world that claims to be able to determine whether or not any given individual is infected with this deadly virus. This is not a "tiny inconsistency." It is time for the mainstream research community to stop ignoring this serious deficiency in AIDS research; and to redirect funds as well as research efforts to the goal of developing a reliable laboratory test intended for use in diagnosing infection with this virus. The citizens of this planet, especially those suffering the devastating consequences of AIDS, deserve such a test.

References:

1. TRUGENE [TM] HIV-1 Genotyping Kit. Visible Genetics Inc., Toronto, Ontario.

2. WHO. Wkly Epidemiol Rec 1997; 72:81-88.

3. WHO. Wkly Epidemiol Rec 1990; 65:221-8.

4. Lallemant M, et al. JAIDS 1992; 5:279-85.

5. Meles H, et al. Clin Diagn Lab Immunol 2002; 9:160-3.

Rodney Richards


Response to Tony Floyd in BMJ:

Tony Floyd suggests the reason I cite results from a study by Meles H, et al. (1) is to "throw doubt upon the validity of current HIV diagnosis." Ignoring for the moment that the tests used to diagnose infection with HIV are not validated or approved for that purpose in the first place, I specifically cited data from Meles et al study only to demonstrate that different institutional criteria for scoring Western Blot (WB) lead to different conclusions of who should be told they are infected. The observation in this study that 19.4% of persons considered indeterminate by one criterion (American Red Cross) are considered positive by another criterion (CDC) leaves no doubt that this is the case. The coincidental observation that all of these individuals were deemed to be "uninfected" on follow-up testing, which they would have never received in the USA, is alarming and unacceptable.

Floyd questions the significance of the discrepancy revealed in this study because there were only "31 initially indeterminate results out of a total of 1 437 [antibody positive] study participants." He goes on to emphasize that this means, "97.84% of [these] results were NOT indeterminate." The implication being, why am I trying to generalize conclusions from these 31 samples when they represent only about 2% of the total HIV positive designations? Why not focus on the other 98% that were "NOT indeterminate?" There is a very good reason for this. The 31 samples in this study are the only samples for which both WB and follow-up data are reported (actually, there were a total of 91 indeterminate results, however, follow-up data was available for only 31 of these samples). The vast majority of the remaining samples considered to be HIV positive in this study were never even subjected to WB testing, let alone follow-up testing.

The reason for this is that according to currently accepted standards of practice put forth by the WHO and UNAIDS, (2) samples that test positive on each of two screening assays in Africa can be declared infected without confirmation of banding patterns by WB. Only in cases where the initial screening assays are discordant (one positive and one negative) are samples routinely subjected to WB testing (or in some cases, a third screening assay). The 91 indeterminate results reported in this study arose from such discordant screening assays. The remaining samples scored positive on two screening assays and were therefore considered unequivocally infected without the need for WB confirmation. As such, we will never know how many of these other samples might have tested negative, positive, or indeterminate on WB; let alone how their banding patterns might have changed over time. (3)

Perhaps a better way to summarize the data in the Meles et al. study is as follows: Of all persons found to have antibodies to HIV specific antigens on WB, and for whom follow-up testing was available, 93.5% became negative when retested while 6.5% remained indeterminate for more than a year and were thus considered negative. Had the CDC criteria been used, nearly 20% of these individuals would have been declared unequivocally infected and never retested.

Granted the sample size in this study is small, and the confidence intervals are likely to be wide. Nevertheless, this is one of the few studies that present any follow-up data for changes in WB patterns over time. The true significance of the observations in this study lies in how many individuals might be expected to test indeterminate (or positive by someone else's criteria) in the first place.

Researchers are well aware that indeterminate WB results in Africa are anything but anecdotal. The Meles et al. publication itself emphasizes, "indeterminate WB profiles in HIV-uninfected subjects are frequent and are as high as 23 to 53% in some African populations." (1) According to the authors of another recent study in Uganda, "Western blot (WB) criteria in epidemiological studies in Africa exhibit an unacceptably high proportion of indeterminate results." (4) The authors of this study go on to demonstrate that the WHO and CDC criteria correctly classified only 57.6% and 57.8% of their study population, respectively. Furthermore, using their own algorithm as the gold standard, they calculate the specificity of the WHO and CDC criteria to be only 53.1%. This is slightly better than a coin flip.

Depending on the study and which WB criteria is embraced to represent the truth, anywhere from about 10% to 40% of African samples that are initially positive for antibodies on one or more screening assay can be expected to test indeterminate on WB. In other words, about 10 to 40% of samples considered to be positive for HIV using screening assays in Africa cannot be confirmed positive using WB. The reason we rarely hear about this data is because Africans are never subjected to WB testing in routine practice; and even in cases where WB is performed, such as validation studies for new test kits, all indeterminate samples are conveniently excluded from calculations of test kit accuracies.

Even more remarkably, depending on the study and the health of individuals, approximately 20 to 90% of ELISA negative individuals in Africa can be expected to test indeterminate on WB. For example, in one study (5) using blood collected in Tanzania, of 89 samples found to be negative on all of four different recombinant ELISA kits, 27 (30.3%) tested positive on WB for one or more HIV specific band. Furthermore, among those seeking care (i.e., sick) the percentage went up to 50%. In another study conducted at the Uganda Virus Research Institute, (6) researchers evaluated WB patterns among ELISA negative rural Africans with various illnesses; 57% of healthy controls, 79% with hookworm, 81% with malaria, and 84% with bacterial infections, were found to be positive on WB for one or more viral specific band. Remarkably, 10% (2/21) of the healthy ELISA negative controls in this study actually tested WB positive according to the CDC criteria.

It is important to remember the only thing that distinguishes individuals as having non-specific antibody reactions, or cross-reacting antibodies due to manageable infections, rather than HIV infection, is an arbitrarily selected WB criterion that designates them indeterminate rather than positive. As soon as an individual exhibits the magic combination of bands held out to represent infection in their locality, their antibodies are perceived to be specific and they are told they are infected with a deadly virus. Instead of considering the possibility that those with illnesses such as bacterial infections may have false positive designations due to cross-reacting antibodies (as illustrated above), the bacterial infection is instead blamed on HIV, and the individual is told they have Stage III HIV disease.

The reality that persons are told they are infected with HIV based on results from tests that have not been validated or approved for this purpose is unacceptable on its own accord. The fact that various institutional criteria lead to different conclusions as to who is inappropriately told they are infected is further objectionable. The observation in the Meles et al. study that 93.5% of persons initially positive for antibodies to HIV specific antigens on WB became negative on follow-up testing is irrelevant to the fact that these tests are being misused in the first place; however, in my opinion this does indeed "throw [further] doubt upon the validity of current HIV diagnosis."

References:

1. Meles H, et al. Clin Diagn Lab Immunol 2002; 9:160-3.

2. WHO. Wkly Epidemiol Rec 1997; 72: 81-8.

3. Actually, one of the "several" cohorts included in this study did require WB confirmation of samples testing concordantly positive on two ELISA screening assays; however, based on available published data (Sahlu T, et al. J Acquir Immune Defic Syndr Hum Retrovirol 1998; 17:149-55; Sahlu T, et al. AIDS 1999; 13: 1263-72.) for this study, fewer than 150 of the 1,437 positive screening result reported in the Meles et al study would have fallen into this category. Furthermore, there is no follow-up data for these individuals to indicate how their WB profiles may have changed over time.

4. Mahe C, et al. International Journal of Epidemiology 2002; 31:985- 990.

5. Christianseen CB, et al. AIDS 1990; 4:1039-40.

6. Medical Research Council (UK), Dept for International Development (UK), Uganda Virus Research Institute, PO Box 49, Entebbe, Uganda. Programme on AIDS in Uganda 1999; Annual Report.

Rodney Richards