Estimation of clinical performance of antigen tests on Czech population data

FIS VŠE employees participated in research evaluating antigen tests.

The analysis is based on the export of aggregated Aantigen (AG) test results and PCR tests registered in the Infectious Diseases Information System (ISIN) for the period August-November 2021. In total, there were 6 million AG test results with an average positivity of 1.27%. For 217,000 AG tests, information was available on the follow-up PCR test performed within three days and on the region in which the AG test was performed. These paired AG and PCR assays were used for analysis. The average positivity of PCR assays was 17.5% and the average positivity of AG assays was 14%. The higher positivity also reflects the fact that some of the AG tests were so-called diagnostic (medical suspicion of Sars-Cov-2) or epidemiological (testing due to risk contact), but most of the tests were preventive.

Article [1] summarizing the results was sent to the review procedure, i.e. it has not yet passed the peer-review process (control and evaluation by other experts). The author team includes scientists from the Faculty of Informatics and Statistics of the University of Economics, the Faculty of Science, Charles University (BIOCEV), the National Institute for Pandemic Management, the National Reference Laboratory for Influenza and Non-Influenza Respiratory Viral Diseases and Faculty of Information Technology of the Czech Technical University.

Results

The average sensitivity of all AG tests determined from 217,000 cases is 69% and the average specificity 97%.

We focused on the analysis of selected subgroups.

Children

The lowest sensitivity was recorded in children under 12 years of age, where it averaged 65%, and in children aged 13-18 years, a sensitivity of 67% was found. The observed decrease in sensitivity may be related to the lower PCR positivity of the test sample of children.

Group Number of tests in analysis SARS-Cov-2 positive Positivity of PCR tests Sensitivity Specificity
0-12 30 635 3 438 11.2% 65.3% 97.1%
13-18 24 112 2 991 12.4% 67.1% 97.2%
19-25 22 882 3 429 15.0% 69.9% 97.3%
26+ 139 810 2 156 20.1% 69.4% 97.2%

Reduced incidence

The analysis was performed in subgroups corresponding to the 7-day incidence level of Sars-Cov-2 virus in the region (“kraj”). The sensitivity of the tests ranged from 66% for incidence to 100 new cases per 100,000 population to 71% for more than 1,000 cases per 100,000 population. In the group with the lowest incidence, the highest specificity was recorded at 98.2%, for the increasing incidence, the specificity gradually decreased to the level of 95%.

Group Number of tests in analysis SARS-Cov-2 positive Positivity of PCR tests Sensitivity Specificity
0 to 100 104 219 4 631 4.4% 66% 98.2%
100 to 500 51 144 11 361 22.2% 66.2% 96.9%
500 to 1,000 36 368 12 509 34.4% 70.5% 95.4%
1,000 to 1,727 25 718 9 514 37.0% 71.2% 95%

Saliva tests

Overall, salivary tests had a sensitivity of 39.9%. The analysis was based only on those tests, the name of which indicated the use of saliva as the only method of collection. The analyzed sample included 2332 tests performed. The obtained sensitivity is in accordance with the conclusions of the meta-analysis [1], where salivary tests reached a sensitivity of only 37.9%. These significantly worse saliva test results are in line with the European Union’s stance on saliva tests. The EU Health Security Committee states that only nasal, pharyngeal or oropharyngeal tests should be recognized for the issuance of the EU COVID Digital Certificate (see source).

Vaccination status

The result shows that in unvaccinated persons, AG tests have a higher sensitivity by less than three percent compared to vaccinated persons. According to research article [2], the viral load is similar in vaccinated and non-vaccinated.

Vaccination status Number of tests in analysis SARS-Cov-2 positive Positivity of PCR tests Sensitivity Specificity
unvaccinated 70 659 14 139 20.3% 70.2% 97.3%
vaccination* 145 224 23 337 16.1% 67.5% 97.2%

* completed vaccination – 14 days after the 2nd dose or 14 days after the J&J dose

Indication

The result shows that the highest sensitivity is expected for the diagnostic indication. The surprising result is a lower sensitivity for the epidemiological indication (risk contacts) than for preventive tests, where the sensitivity is only one percentage point below the sensitivity level of the diagnostic tests.

Indication Number of tests in analysis SARS-Cov-2 positive Positivity of PCR tests Sensitivity Specificity
diagnostic 23 399 7 949 34.0 69.9% 97.8%
epidemiological 37 100 6 680 18.0 62.6% 96.8%
preventive 154 344 22 178 14.4 68.8% 97.3%

A detailed explanation of the meaning of individual indications can be found on the COVID-19 | website Diseases currently from MZČR (mzcr.cz) .

Note the results for vaccination status and indication were obtained on an updated version of the data (difference in size of about 0.5%).

Results of specific tests

The following are the test results for which:

  1. at least 100 PCR positive and 300 PCR negative cases paired with AG tests are available for the evaluated test type, in accordance with the EU Health Committee’s AG test clinical trial methodology,
  2. The PCR positivity of the evaluated sample for the given test was in the range of 9% to 27%,
  3. above-average sensitivity (according to data) was found.

The condition of a sufficient number of paired AG and PCR results is met by 27 PCR tests, which represent approximately 75% of the 6 million AG test records in the database. Of these, 14 were above average. Of these, for two tests from one manufacturer (Wellion SARS-CoV-2 PLUS and VivaDiag Wellion SARS-CoV-2 Antigen), the PCR positivity was higher than 27%, which could skew the results and these are therefore not listed. VivaChek is represented by two other tests on the list.

As the list is based on data from official sampling points, tests intended for professional use only are included. Among the 12 selected tests, it is mainly the SD BIOSENSOR STANDARD F-Covid test, which is intended for use with analyzers.

List of tests with above-average sensitivity, sorted by manufacturer’s name
id Test name – manufacturer Detected sensitivity Identified specificity Foreign lists
1 Panbio Covid-19 Ag Rapid Test – Abbott Rapid Diagnostics 75.4% 98.4% WHO, EU, DE, UK
2 Flowflex SARS-CoV-2 Antigen rapid test – ACON Biotech (Hangzhou) Co., Ltd, 83.9% 96.8% EU, DE, UK
3 AMP Rapid Test SARS-CoV-2 Ag – AMEDA Labordiagnostic GmbH 74.5% 94.1% EU, DE, UK
4 WANTAI SARS-COV-2 Ag Rapid Test (Colloidal gold) – BEIJING WANTAI BIOLOGICAL PHARMACY ENTERPRISE CO., LTD 84.1% 96.0% EU, DE
5 DIAQUICK COVID-19 Ag Cassette – DIALAB Ges, m, b, H, Wr, Neudorf / AT 87.6% 96.4%
6 Humasis COVID-19 Ag Test – HUMASIS Co., Ltd 71.4% 98.3% DE, EU, UK
7 NADAL® COVID-19 Ag Test – nal von minden GmbH 84.5% 96.6% DE, EU, UK *
8 COVID-19 Antigen Detection Kit – New Gene (Hangzhou) Bioengineering Co. ,, Ltd, 76.3% 98.4% DE, EU
9 ViVaDiag SARS-CoV-2 Ag Rapid Test – VivaChek Biotech (Hangzhou) Co., Ltd, 74.4% 96.4%
10 SARS-CoV-2 Rapid Antigen Test – Roche (SD BIOSENSOR) 73.6% 98.9% WHO, EU, DE, UK **
11 STANDARD F COVID-19 Ag FIA – SD BIOSENSOR Inc * 78.7% 98.9% EU, DE, (UK) **
12 VivaDiagTM Pro SARS-CoV-2 Ag Rapid Test – VivaChek Biotech (Hangzhou) Co., Ltd, 74.1% 97.1%

Note within the analysis, duplicate variants of some tests were unified.
* Considered a variant of the “NADAL COVID-19 Ag plus test” from the UK list
** Considered a variant of the “Roche SD Biosensor Standard Q COVID-19 Ag Test” from the UK list.

Other evaluation of antigen tests – compliance with the performed analysis

The observed average sensitivity of 69% is consistent with a large foreign meta-analysis involving 133 studies [3], which reported an average sensitivity of AG tests of 71.2%. The average specificity found was 97.2%, with a study [3] reporting an average specificity of 98.9%.

Of the 12 tests, all but three (two different manufacturers) are represented in the list of tests that met the conditions for sufficient sensitivity published by the German Paul Ehrlich Institute (marked DE in the table; source in the version of 12.1.2022). The same nine tests are also included in the European Common List (EU; source in the version of 21.12.2021). For seven tests, a similar test from the same manufacturer is listed as having passed the evaluation in the United Kingdom (UK; source as of 24.12.2021). Evaluation in the UK is very demanding, only 30% of the best tests are selected for detailed evaluation. Variants of two of these tests are on the WHO Emergency Use Listing (source).

Where to find more information?

Lists published by other countries and institutions can serve as one of the guides for assessing the quality of other antigenic tests. The primary list that the public can use is the extensive list of EU Commonly Recognized Tests. This is largely based on a list published by the Paul Ehrlich Institute, which lists almost 200 detailed test evaluations. The list published by the Department of Health and Social Care of the United Kingdom currently contains less than 50 tests.

Research limitations

The results do not reflect a possible change in the sensitivity of the tests due to the Omikron mutation.

In a small number of people, the infection may have occurred after the AG test and before the PCR test. We assume that this phenomenon occurred evenly in all groups and types of tests, and therefore it does not affect the relative comparison. At the same time, the overall average sensitivity of the tests corresponds to the average sensitivity found in foreign studies, which supports the correctness of the methodology used.

Data on indication and vaccination status were calculated on the basis of additional exports, which had a minimal effect on the results.

Thanks

We thank the Institute of Health Information and Statistics of the Czech Republic for providing data.

Literature

[1] Kliegr, T, Tachezy R, Jarkovský, J, Jiřincová, H, Kuchař, J, Karel, T, Evaluation of rapid diagnostic tests for SARS-CoV-2 antigen with break downs per age group, incidence, indication, vaccination status and sample collection type based on large-scale clinical data, Czech Republic, August 2021 to November 2021. Preprint.
[2] Kissler, Stephen M et al. “Viral Dynamics of SARS-CoV-2 Variants in Vaccinated and Unvaccinated Persons.” The New England journal of medicine vol. 385,26 (2021): 2489-2491. doi: 10.1056 / NEJMc2102507
[3]Brümmer, L, E ,, Katzenschlager, S ,, Gaeddert, M ,, Erdmann, C ,, Schmitz, S ,, Bota, M ,, Grilli, M ,, Larmann, J ,, Weigand, M, A ,, Pollock , N, R ,, Macé, A ,, Carmona, S ,, Ongarello, S ,, Sacks, J, A ,, & Denkinger, C, M, (2021), Accuracy of novel antigen rapid diagnostics for SARS-CoV- 2: A living systematic review and meta-analysis, In PLoS Medicine (Vol, 18, Issue 8), doi: 10.1371 / journal.pmed.1003735

 

Weby Vysoké školy ekonomické v Praze využívají pro optimalizaci svého obsahu a poskytovaných služeb cookies. Prosíme o udělení souhlasu s jejich uložením.

Vyberte služby, pro které chcete povolit využívání cookies: