A false positive means someone would be told that they had previously had a coronavirus when they were not - a potential danger as people might then think they were immune to the virus while they were still vulnerable.
"It's terrible. It's really terrible," said Dr. Caryn Bern, one of the study authors who reviewed the 12 tests.
She said it is not realistic to think that all tests will be 100% accurate all the time, their false positive rates should be 5% or less, or ideally 2% or less.
"It was a real wake-up call for me. We're not at the point where one of these tests can be used reliably," added study co-author Dr. Alexander Marson. "There is great danger in relying on them, but we hope to get to a point soon where we can count on these tests."
The COVID-19 test project is a consortium of researchers and doctors from the University of California at San Francisco, the University of California at Berkeley, Bio Zub Chan Zuckerberg and the Innovative Genomics Institute.
A rocky road to antibody testing
Bern said one of the reasons for the high false positive rates was the loosening of the requirements of the U.S. Food and Drug Administration.
In mid-March, when it became clear that Covid-19 was starting to get out of control, the FDA relaxed its approval standards to get more antibody tests on the market quickly.
The agency began to allow companies to sell tests without first providing evidence of their effectiveness. Some 175 test developers have taken advantage of these new rules and can legally market their antibody tests without first evaluating their validation data by the FDA.
"I think we are paying for it now," said Bern, professor of epidemiology and biostatistics at the UCSF School of Medicine.
After a large lab association complained that "crappy" tests were flooding the market, FDA Commissioner Dr. Stephen Hahn released a statement on April 18 that the FDA would work with the Centers for Disease Control and Prevention of the United States and the National Institutes of Health to assess antibody tests.
"We have evolved quickly and thoughtfully, and we continue to learn and adapt based on experience and actual data that we see," Hahn wrote in the statement.
When CNN asked an FDA spokesperson if there had been progress in this assessment effort, she replied that "there is not much we can share publicly yet ".
Federal officials have said that good antibody tests are essential to return to normal, as they could help determine who in the United States has immunity.
That's why false positives are so dangerous, said Marson, professor of microbiology and immunology at the UCSF School of Medicine.
"False positives will greatly blur our image of who was infected and who was not," he said. "We need clarity on this basic information to start guiding our way out of the pandemic."
Relaxed FDA rules
If the test developers wish, they can go beyond the minimum requirements of the FDA and ask the federal agency to review their data showing that the tests work. If the FDA gives the green light, the company obtains an emergency use authorization.
Of the 12 California journal tests, only one company has this FDA approval seal. This is a test from the Mount Sinai Laboratory in New York, which the Californian group has modified to use in its own laboratory.
This test only got one false positive out of 108 blood samples.
Californian researchers used blood samples before July 2018 to make sure they were Covid-free. Each test processed between 99 and 108 samples.
Another test gave better results than the Mt. Sinai test on negative samples, with no false positive results.
Of the remaining 10 tests, six had accuracy rates of about 95% or more for negative samples and four had rates between 84% and 95%.
These results are based on the search for two types of antibodies. One appears soon after infection and the other appears later.
The researchers also checked to see if the tests accurately detected the positive samples, but were less confident about the results, because there are variations in the time it takes people to mount a detectable antibody response.
California researchers have gotten as many tests as possible, but so far they haven't been able to screen for antibody tests by big test manufacturers like Roche and Abbott because they need proprietary instruments to run the tests, which the California tests did not have.
They hope to continue screening more antibody tests to see if they work, but have warned that even if a reliable test shows that you have antibodies, doctors still don't know what it means. The presence of antibodies may indicate that you are immune and will not be infected a second time, or it may indicate that you have no immunity. It could also mean something in between - for example, you have limited immunity for a period of a few months.
"We have this huge knowledge gap," said Bern. "This is the most important message."
Dr. Minali Nigam and Arman Azad contributed to this story.
