An empirical assessment of bivariate methods for meta-analysis of test accuracy

BACKGROUND: Meta-analyses of sensitivity and specificity pairs reported from diagnostic test accuracy studies employ a variety of statistical models for estimating mean performance and performance across different test thresholds. The impact of these alternative models on conclusions in applied sett...

Full description

Bibliographic Details
Main Author: Dahabreh, Issa J.
Corporate Authors: United States Agency for Healthcare Research and Quality, Tufts Evidence-based Practice Center
Format: eBook
Language:English
Published: Rockville, Md. Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services [2012], 2012
Series:Methods research report
Subjects:
Online Access:
Collection: National Center for Biotechnology Information - Collection details see MPG.ReNa
LEADER 05184nam a2200349 u 4500
001 EB000943550
003 EBX01000000000000000737140
005 00000000000000.0
007 tu|||||||||||||||||||||
008 150223 r ||| eng
100 1 |a Dahabreh, Issa J. 
245 0 0 |a An empirical assessment of bivariate methods for meta-analysis of test accuracy  |h Elektronische Ressource  |c prepared for Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services ; prepared by Tufts Evidence-based Practice Center, Tufts Medical Center ; investigators, Issa J. Dahabreh ... [et al.] 
260 |a Rockville, Md.  |b Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services  |c [2012], 2012 
300 |a 1 PDF file (1 v. (various pagings)  |b ill.) 
505 0 |a Includes bibliographical references 
653 |a Sensitivity and Specificity 
653 |a Multivariate Analysis 
653 |a Empirical Research 
653 |a Meta-Analysis as Topic 
653 |a Diagnostic Techniques and Procedures 
710 2 |a United States  |b Agency for Healthcare Research and Quality 
710 2 |a Tufts Evidence-based Practice Center 
041 0 7 |a eng  |2 ISO 639-2 
989 |b NCBI  |a National Center for Biotechnology Information 
490 0 |a Methods research report 
500 |a "Contract no. 290-2007-10055-I.". - "November 2012." 
856 4 0 |u https://www.ncbi.nlm.nih.gov/books/NBK115736  |3 Volltext  |n NLM Bookshelf Books  |3 Volltext 
082 0 |a 610 
520 |a BACKGROUND: Meta-analyses of sensitivity and specificity pairs reported from diagnostic test accuracy studies employ a variety of statistical models for estimating mean performance and performance across different test thresholds. The impact of these alternative models on conclusions in applied settings has not been studied systematically. METHODS: We constructed a database of PubMed-indexed meta-analyses (1987-2003) from which 2x2 tables for each included primary study could be readily extracted. We evaluated the following methods for meta-analysis of sensitivity and specificity: fixed and random effects univariate meta-analyses using inverse variance methods; univariate random effects meta-analyses with maximum likelihood (ML; both using a normal approximation and the exact binomial likelihood to describe between-study variability); bivariate random effects meta-analyses (both using a normal approximation and the exact binomial likelihood to describe between-study variability).  
520 |a The bivariate model using the exact binomial likelihood was also fit using a fully Bayesian approach. We constructed summary receiver operating characteristic (SROC) curves using the Moses-Littenberg fixed effects method (weighted and unweighted) and the Rutter-Gatsonis hierarchical SROC (HSROC) method. We also obtained alternative SROC curves corresponding to different underlying regression models [logit-true positive rate (TPR) over logit-false positive rate (FPR); logit-FPR over logit-TPR; difference of the logit-TPR and logit-TPR over their sum; and major axis regression of logit-TPR over logit-FPR]. RESULTS: We reanalyzed 308 meta-analyses of test performance. Fixed effects univariate analyses produced estimates with narrower confidence intervals compared to random effects methods.  
520 |a The SROC curves produced by the Moses-Littenberg and Rutter-Gatsonis models were similar in most examples. Alternative parameterizations of the HSROC regression resulted in markedly different summary lines in a third of the meta-analyses; this depends to a large extent on the estimated covariance between sensitivity and specificity in the bivariate model. Our results are generally in agreement with published simulation studies and the theoretically expected behavior of meta-analytic estimators. CONCLUSION: Bivariate models are more theoretically motivated compared to univariate analyses and allow estimation of the correlation between sensitivity and specificity. Bayesian methods fully quantify uncertainty and their ability to incorporate external evidence may be particularly useful for parameters that are poorly estimated in the bivariate model. Alternative SROC curves provide useful global summaries of test performance 
520 |a Methods using the normal approximation (both univariate and bivariate, inverse variance and ML) produced estimates of summary sensitivity and specificity closer to 0.5 and smaller standard errors compared to methods using the exact binomial likelihood. Point estimates from univariate and bivariate random effects meta-analyses were similar when performing pairwise (univariate vs. bivariate) comparisons, regardless of the estimation method (inverse variance, ML with normal approximation, or ML with the exact binomial likelihood for estimation). Fitting the bivariate model using ML and fully Bayesian methods produced almost identical point estimates of summary sensitivity and specificity; however, Bayesian results indicated additional uncertainty around summary estimates. The correlation of sensitivity and specificity across studies was imprecisely estimated by all bivariate methods.