{"id":91,"date":"2015-05-12T16:23:21","date_gmt":"2015-05-12T07:23:21","guid":{"rendered":"http:\/\/imgcom.jsrt.or.jp\/rocGroup\/?page_id=91"},"modified":"2017-10-31T11:33:02","modified_gmt":"2017-10-31T02:33:02","slug":"readings-in-roc-analysis-with-emphasis-on-medical-applications","status":"publish","type":"page","link":"http:\/\/imgcom.jsrt.or.jp\/rocGroup\/publications\/readings-in-roc-analysis-with-emphasis-on-medical-applications\/","title":{"rendered":"Readings in ROC Analysis, with Emphasis on Medical Applications"},"content":{"rendered":"<p style=\"text-align: justify;\">Some Papers appear more than once because they belong to multiple classifications<\/p>\n<p style=\"text-align: justify;\">Background<\/p>\n<ul style=\"text-align: justify;\">\n<li>Egan JP. Signal detection theory and ROC analysis. New York: Academic Press, 1975.<\/li>\n<li>Fryback DG, Thornbury JR. The efficacy of diagnostic imaging. Med Decis Making 1991; 11: 88.<\/li>\n<li>Green DM, Swets JA. Signal detection theory and psychophysics. New York, NY: Wiley, 1966.<\/li>\n<li>Griner PF, Mayewski RJ, Mushlin AI, Greenland P. Selection and interpretation of diagnostic tests and procedures: principles and applications. Annals Int Med 1981; 94: 553.<\/li>\n<li>International Commission on Radiation Units and Measurements. Medical imaging: the assessment of image quality (ICRU Report 54). Bethesda,MD: ICRU, 1996.<\/li>\n<li>Lusted LB. Signal detectability and medical decision-making. Science 1971; 171: 1217.<\/li>\n<li>McNeil BJ, Adelstein SJ. Determining the value of diagnostic and screening tests. J Nucl Med 1976; 17: 439.<\/li>\n<li>McNeil BJ, Keeler E, Adelstein SJ. Primer on certain elements of medical decision making. New Engl J Med 1975; 293: 211.<\/li>\n<li>Metz CE, Wagner RF, Doi K, Brown DG, Nishikawa RN, Myers KJ. Toward consensus on quantitative assessment of medical imaging systems. Med Phys 22: 1057-1061, 1995.<\/li>\n<li>National Council on Radiation Protection and Measurements. An introduction to efficacy in diagnostic radiology and nuclear medicine (NCRP Commentary 13). Bethesda, MD: NCRP, 1995.<\/li>\n<li>Robertson EA, Zweig MH, Van Steirtghem AC. Evaluating the clinical efficacy of laboratory tests. Am J Clin Path 1983; 79: 78.<\/li>\n<li>Swets JA, Pickett RM, Whitehead SF, et al. Assessment of diagnostic technologies. Science 1979; 205:753\u2013759.<\/li>\n<li>Swets JA, Pickett RM. Evaluation of diagnostic systems: methods from signal detection theory. New York, NY: Academic Press, 1982.<\/li>\n<li>Wagner RF, Metz CE, Campbell G. Assessment of medical imaging systems and computer aids: a tutorial review. Acad Radiol 2007; 14: 723\u2013748.<\/li>\n<li>Zweig MH, Campbell G. Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clinical Chemistry 1993; 39: 561. [Erratum published in Clinical Chemistry 1993; 39: 1589.]<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">General<br \/>\nBooks<\/p>\n<ul style=\"text-align: justify;\">\n<li>Pepe MS. The statistical evaluation of medical tests for classification and prediction. Oxford ; New York: Oxford University Press, 2004.<\/li>\n<li>Zhou X-H, Obuchowski NA, McClish DK. Statistical methods in diagnostic medicine. New York, NY: Wiley-Interscience, 2002<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">Articles<\/p>\n<ul style=\"text-align: justify;\">\n<li>Hanley JA. Receiver operating characteristic (ROC) methodology: the state of the art. Critical Reviews in Diagnostic Imaging 1989; 29: 307.<\/li>\n<li>International Commission on Radiation Units and Measurements. Receiver Operating Characteristic Analysis in Medical Imaging (ICRU Report 79). J ICRU 2008; 8:1\u201362.<\/li>\n<li>King JL, Britton CA, Gur D, Rockette HE, Davis PL. On the validity of the continuous and discrete confidence rating scales in receiver operating characteristic studies. Invest Radiol 1993; 28: 962.<\/li>\n<li>Metz CE. Basic principles of ROC analysis. Seminars in Nucl Med 1978; 8: 283.<\/li>\n<li>Metz CE. ROC methodology in radiologic imaging. Invest Radiol 1986; 21: 720.<\/li>\n<li>Metz CE. Some practical issues of experimental design and data analysis in radiological ROC studies. Invest Radiol 1989; 24: 234.<\/li>\n<li>Metz CE. Evaluation of CAD methods. In Computer-Aided Diagnosis in Medical Imaging (K Doi, H MacMahon, ML Giger and KR Hoffmann, eds.). Amsterdam: Elsevier Science (Excerpta Medica International Congress Series, Vol. 1182), pp. 543-554, 1999.<\/li>\n<li>Metz CE. Fundamental ROC analysis. In: Handbook of Medical Imaging, Vol. 1: Physics and Psychophysics (J Beutel, H Kundel and R Van Metter, eds.). Bellingham, WA; SPIE Press, 2000, pp. 751-769.<\/li>\n<li>Metz CE. Receiver operating characteristic (ROC) analysis: a tool for quantitative evaluation of observer performance and imaging systems. JACR 3: 413-422, 2006<\/li>\n<li>Metz CE, Shen J-H. Gains in accuracy from replicated readings of diagnostic images: prediction and assessment in terms of ROC analysis. Med Decis Making 1992; 12: 60.<\/li>\n<li>Rockette HE, Gur D, Metz CE. The use of continuous and discrete confidence judgments in receiver operating characteristic studies of diagnostic imaging techniques. Invest Radiol 1992; 27: 169.<\/li>\n<li>Swets JA. ROC analysis applied to the evaluation of medical imaging techniques. Invest Radiol 1979; 14: 109.<\/li>\n<li>Swets JA. Indices of discrimination or diagnostic accuracy: their ROCs and implied models. Psychol Bull 1986; 99: 100.<\/li>\n<li>Swets JA. Measuring the accuracy of diagnostic systems. Science 1988; 240: 1285.<\/li>\n<li>Swets JA. Signal detection theory and ROC analysis in psychology and diagnostics: collected papers. Mahwah, NJ; Lawrence Erlbaum Associates, 1996.<\/li>\n<li>Swets JA, Pickett RM. Evaluation of diagnostic systems: methods from signal detection theory. New York: Academic Press, 1982.<\/li>\n<li>Wagner RF, Beiden SV, Metz CE. Continuous vs. categorical data for ROC analysis: Some quantitative considerations. Academic Radiol 2001, 8: 328, 2001.<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">Bias<\/p>\n<ul style=\"text-align: justify;\">\n<li>Begg CB, Greenes RA. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics 1983; 39: 207.<\/li>\n<li>Begg CB, McNeil BJ. Assessment of radiologic tests: control of bias and other design considerations. Radiology 1988; 167: 565.<\/li>\n<li>Gray R, Begg CB, Greenes RA. Construction of receiver operating characteristic curves when disease verification is subject to selection bias. Med Decis Making 1984; 4: 151.<\/li>\n<li>Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. New Engl J Med 1978; 299: 926.<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">Curve Fitting<\/p>\n<ul style=\"text-align: justify;\">\n<li>Dorfman DD, Alf E. Maximum likelihood estimation of parameters of signal detection theory and determination of confidence intervals \u2014 rating method data. J Math Psych 1969; 6: 487.<\/li>\n<li>Dorfman DD, Berbaum KS, Metz CE, Lenth RV, Hanley JA, Dagga HA. Proper ROC analysis: the bigamma model. Academic Radiol 1997; 4: 138.<\/li>\n<li>Dorfman DD, Berbaum KS. A contaminated binormal model for ROC data: Part II. A formal model. Acad Radiol 2000; 7:427-437.<\/li>\n<li>Grey DR, Morgan BJT. Some aspects of ROC curve-fitting: normal and logistic models. J Math Psych 1972; 9: 128.<\/li>\n<li>Hanley JA. The robustness of the &#8220;binormal&#8221; assumptions used in fitting ROC curves. Med Decis Making 1988; 8: 197.<\/li>\n<li>Lloyd CJ. Estimation of a convex ROC curve. Stat Prob Lett 2002; 59: 99\u2013111.<\/li>\n<li>Metz CE, Herman BA, Shen J-H. Maximum-likelihood estimation of ROC curves from continuously-distributed data. Stat Med 1998; 17: 1033.<\/li>\n<li>Metz CE, Pan X. &#8220;Proper&#8221; binormal ROC curves: theory and maximum-likelihood estimation. J Math Psych 1999; 43: 1.<\/li>\n<li>Ogilvie J, Creelman CD. Maximum likelihood estimaton of receiver operating characteristic curve parameters. Journal of Mathematical Psychology. 1968;5:377-391<\/li>\n<li>Pan X, Metz CE. The &#8220;proper&#8221; binormal model: parametric ROC curve estimation with degenerate data. Academic Radiol 1997; 4: 380.<\/li>\n<li>Pesce LL, Metz CE. Reliable and computationally efficient maximum-likelihood estimation of &#8220;proper&#8221; binormal ROC curves. Acad Radiol. 2007;14(7):814-29<\/li>\n<li>Swensson RG. Unified measurement of observer performance in detecting and localizing target objects on images. Med Phys 1996; 23: 1709.<\/li>\n<li>Swets JA. Form of empirical ROCs in discrimination and diagnostic tasks: implications for theory and measurement of performance. Psychol Bull 1986; 99: 181.<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">Statistics<\/p>\n<ul style=\"text-align: justify;\">\n<li>Multi-Case statistical analysis: only case variation considered<br \/>\nAgresti A. A survey of models for repeated ordered categorical response data. Statistics in Medicine 1989; 8; 1209.<\/li>\n<li>Bamber D. The area above the ordinal dominance graph and the area below the receiver operating graph. J Math Psych 1975; 12: 387.<\/li>\n<li>Bandos AI, Rockette HE, Gur D. A permutation test sensitive to differences in areas for comparing ROC curves from a paired design. STATISTICS IN MEDICINE 24 (18): 2873-2893 SEP 30 2005<\/li>\n<li>DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988; 44: 837.<\/li>\n<li>Hajian-Tilaki KO, Hanley JA. Comparison of three methods for estimating the standard error of the area under the curve in ROC analysis of quantitative data. ACADEMIC RADIOLOGY 9 (11): 1278-1285 NOV 2002<\/li>\n<li>Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143: 29.<\/li>\n<li>Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983; 148: 839.<\/li>\n<li>Jiang Y, Metz CE, Nishikawa RM. A receiver operating characterisitc partial area index for highly sensitive diagnostic tests. Radiology 1996; 201: 745.<\/li>\n<li>Ma G, Hall WJ. Confidence bands for receiver operating characteristic curves. Med Decis Making 1993; 13: 191.<\/li>\n<li>McClish DK. Analyzing a portion of the ROC curve. Med Decis Making 1989; 9: 190.<\/li>\n<li>McClish DK. Determining a range of false-positive rates for which ROC curves differ. Med Decis Making 1990; 10: 283.<\/li>\n<li>McNeil BJ, Hanley JA. Statistical approaches to the analysis of receiver operating characteristic (ROC) curves. Med Decis Making 1984; 4: 137.<\/li>\n<li>Metz CE. Statistical analysis of ROC data in evaluating diagnostic performance. In: Multiple regression analysis: applications in the health sciences (D Herbert and R Myers, eds.). New York: American Institute of Physics, 1986, pp. 365.<\/li>\n<li>Metz CE. Quantification of failure to demonstrate statistical significance: the usefulness of confidence intervals. Invest Radiol 1993; 28: 59.<\/li>\n<li>Metz CE, Herman BA, Roe CA. Statistical comparison of two ROC curve estimates obtained from partially-paired datasets. Med Decis Making 1998; 18: 110.<\/li>\n<li>Metz CE, Kronman HB. Statistical significance tests for binormal ROC curves. J Math Psych 1980; 22: 218.<\/li>\n<li>Metz CE, Wang P-L, Kronman HB. A new approach for testing the significance of differences between ROC curves measured from correlated data. In: Information processing in medical imaging (F Deconinck, ed.). The Hague: Nijhoff, 1984, p. 432.<\/li>\n<li>Thompson ML, Zucchini W. On the statistical analysis of ROC curves. Statistics in Medicine 1989; 8: 1277.<\/li>\n<li>Wieand S, Gail MH, James BR, James KL. A family of nonparametric statistics for comparing diagnostic markers with paired or unpaired data. Biometrika 1989; 76: 585.<\/li>\n<li>Zhou XH, Gatsonis CA. A simple method for comparing correlated ROC curves using incomplete data. Statistics in Medicine 1996; 15: 1687-1693.<\/li>\n<li>Multi-Reader Multi-Case statistical analysis<br \/>\nBandos AI, Rockette HE, Gur D. A permutation test for comparing ROC curves in multireader studies ACADEMIC RADIOLOGY 13 (4): 414-420 APR 2006<\/li>\n<li>Beiden SV, Wagner RF, Campbell G. Components-of-variance models and multiple-bootstrap experiments: and alternative method for random-effects, receiver operating characteristic analysis. Academic Radiol. 2000; 7: 341.<\/li>\n<li>Beiden SV, Wagner RF, Campbell G, Metz CE, Jiang Y. Components-of-variance models for random-effects ROC analysis: The case of unequal variance structures across modalities. Academic Radiol. 2001; 8: 605.<\/li>\n<li>Beiden SV, Wagner RF, Campbell G, Chan H-P. Analysis of uncertainties in estimates of components of variance in multivariate ROC analysis. Academic Radiol. 2001; 8: 616.<\/li>\n<li>Dorfman DD, Berbaum KS, Metz CE. ROC rating analysis: generalization to the population of readers and cases with the jackknife method. Invest Radiol 1992; 27: 723.<\/li>\n<li>Dorfman DD, Berbaum KS, Lenth RV, Chen Y-F, Donaghy BA. Monte Carlo validation of a multireader method for receiver operating characteristic discrtet rating data: factorial experimental design. Academic Radiol 1998; 5: 591.<\/li>\n<li>Dorfman DD, Metz CE. Multi-reader multi-case ROC analysis: comments on Begg\u2019s commentary. Academic Radiol 1995; 2 (Supplement 1): S76.<\/li>\n<li>Gallas BD One-shot estimate of MRMC variance: AUC. ACADEMIC RADIOLOGY 13 (3): 353-362 MAR 2006<\/li>\n<li>Hillis SL, Obuchowski NA, Schartz KM, Berbaum KS. A comparison of the Dorfman-Berbaum-Metz and Obuchowski-Rockette methods for receiver operating characteristic (ROC) data. Stat Med 2005; 24:1579-1607.<\/li>\n<li>Hillis SL, Berbaum KS. Monte Carlo validation of the Dorfman-Berbaum-Metz method using normalized pseudovalues and less data-based model simplification. Academic Radiology 2005; 12:1534-1541.<\/li>\n<li>Hillis SL, Berbaum KS Power estimation for the Dorfman-Berbaum-Metz method ACADEMIC RADIOLOGY 11 (11): 1260-1273 NOV 2004<\/li>\n<li>Obuchowski NA. Multireader, multimodality receiver operating characteristic curve studies: hypothesis testing and sample size estimation using an analysis of variance approach with dependent observations. Academic Radiol 1995; 2 [Supplement 1]: S22.<\/li>\n<li>Obuchowski, NA. Sample size calculations in studies of test accuracy. Stat Methods Med Res 1998; 7: 371.<\/li>\n<li>Obuchowski NA, Beiden SV, Berbaum KS, et al. Multireader, multicase receiver operating characteristic analysis: An empirical comparsion of five methods ACADEMIC RADIOLOGY 11 (9): 980-995 SEP 2004<\/li>\n<li>Rockette HE, Obuchowski N, Metz CE, Gur D. Statistical issues in ROC curve analysis. Proc SPIE 1990; 1234: 111.<\/li>\n<li>Roe CA, Metz CE. The Dorfman-Berbaum-Metz method for statistical analysis of multi-reader, multi-modality ROC data: validation by computer simulation. Academic Radiol 1997; 4: 298.<\/li>\n<li>Roe CA, Metz CE. Variance-component modeling in the analysis of receiver operating characteristic index estimates. Academic Radiol 1997; 4: 587.<\/li>\n<li>Regression analysis of ROC curves Pepe MS. The statistical evaluation of medical tests for classification and prediction. Oxford ; New York: Oxford University Press, 2004.<\/li>\n<li>Pepe MS, Cai TX. The analysis of placement values for evaluating discriminatory measures. BIOMETRICS 60 (2): 528-535 JUN 2004<\/li>\n<li>Toledano A, Gatsonis CA. Regression analysis of correlated receiver operating characteristic data. Academic Radiol 1995; 2 [Supplement 1]: S30.<\/li>\n<li>Toledano AY, Gatsonis C. Ordinal regression methodology for ROC curves derived from correlated data. Statistics in Medicine 1996, 15: 1807.<\/li>\n<li>Toledano AY, Gatsonis C. GEEs for ordinal categorical data: arbitrary patterns of missing responses and missingness in a key covariate. Biometrics 1999; 22, 488.<\/li>\n<li>Tosteson A, Begg C. A general regression methodology for ROC curve estimation. Med Decis Making 1988; 8: 204.<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">Relationships with Cost\/Benefit Analysis<\/p>\n<ul style=\"text-align: justify;\">\n<li>Halpern EJ, Alpert M, Krieger AM, Metz CE, Maidment AD. Comparisons of ROC curves on the basis of optimal operating points. Academic Radiology 1996; 3: 245-253.<\/li>\n<li>Metz CE. Basic principles of ROC analysis. Seminars in Nucl Med 1978; 8: 283-298.<\/li>\n<li>Metz CE, Starr SJ, Lusted LB, Rossmann K. Progress in evaluation of human observer visual detection performance using the ROC curve approach. In: Information Processing in Scintigraphy (C Raynaud and AE Todd-Pokropek, eds.). Orsay, France: Commissariat \u00e0 l&#8217;Energie Atomique, D\u00e9partement de Biologie, Service Hospitalier Fr\u00e9d\u00e9ric Joliot, 1975, p. 420.<\/li>\n<li>Phelps CE, Mushlin AI. Focusing technology assessment. Med Decis Making 1988; 8: 279.<\/li>\n<li>Sainfort F. Evaluation of medical technologies: a generalized ROC analysis. Med Decis Making 1991; 11: 208.<\/li>\n<li>Wagner RE, Beam CA, Beiden SV. Reader variability in mammography and its implications for expected utility over the population of readers and cases. MEDICAL DECISION MAKING 24 (6): 561-572 NOV-DEC 2004<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">Generalizations<\/p>\n<ul style=\"text-align: justify;\">\n<li>Anastasio MA, Kupinski MA, Nishikawa RN. Optimization and FROC analysis of rule-based detection schemes using a multiobjective approach. IEEE Trans Med Imaging 1998; 17: 1089<\/li>\n<li>Bunch PC, Hamilton JF, Sanderson GK, Simmons AH. A free response approach to the measurement and characterization of radiographic observer performance. Proc SPIE 1997; 127: 124.<\/li>\n<li>Chakraborty DP. Maximum likelihood analysis of free-response receiver operating characteristic (FROC) data. Med Phys 1989; 16: 561.<\/li>\n<li>Chakraborty DP, Winter LHL. Free-response methodology: alternate analysis and a new observer-performance experiment. Radiology 1990; 174: 873.<\/li>\n<li>Chakraborty DP, Berbaum KS. Observer studies involving detection and localization: Modeling, analysis and validation. Medical Physics 2004; 31:2313-2330.<\/li>\n<li>Chakraborty DP. A search model and figure of merit for observer data acquired according to the free-response paradigm. Phys. Med. Biol. 2006; 51:3449-3462.<\/li>\n<li>Chakraborty DP. ROC Curves predicted by a model of visual search. Phys. Med. Biol. 2006; 51:3463-3482.<\/li>\n<li>Edwards DC, Metz CE. Evaluating Bayesian ANN estimates of ideal observer decision variables by comparison with identity functions. Proc. SPIE 5749: 174-182, 2005.<\/li>\n<li>Edwards DC, Metz CE. Optimization of an ROC hypersurface constructed only from an observer&#8217;s within-class sensitivities. Proc. SPIE 6146: 61460A1-61460A7, 2006.<\/li>\n<li>Edwards DC, Metz CE. Analysis of proposed three-class classification decision rules in terms of the ideal observer decision rule. J. Math. Psych. (in press), 2006.<\/li>\n<li>Egan JP, Greenberg GZ, Schulman AI. Operating characteristics, signal detection, and the method of free response. J Acoust Soc Am 1961; 33: 993.<\/li>\n<li>HajianTilaki KO, Hanley JA, Joseph L, et al. Extension of receiver operating characteristic analysis to data concerning multiple signal detection tasks. ACADEMIC RADIOLOGY 4 (3): 222-229 MAR 1997<\/li>\n<li>Metz CE, Starr SJ, Lusted LB. Observer performance in detecting multiple radiographic signals: prediction and analysis using a generalized ROC approach. Radiology 1976; 121: 337.<\/li>\n<li>Obuchowski NA, Lieber ML, Powell KA.Data analysis for detection and localization of multiple abnormalities with application to mammography. ACADEMIC RADIOLOGY 7 (7): 516-525 JUL 2000<\/li>\n<li>Starr SJ, Metz CE, Lusted LB, Goodenough DJ. Visual detection and localization of radiographic images. Radiology 1975; 116: 533.<\/li>\n<li>Swensson RG. Unified measurement of observer performance in detecting and localizing target objects on images. Med Phys 1996; 23: 1709.<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">Papers related specifically to our Current Software<br \/>\nROCKIT<\/p>\n<ul style=\"text-align: justify;\">\n<li>Dorfman DD, Alf E. Maximum likelihood estimation of parameters of signal detection theory and determination of confidence intervals \u2014 rating method data. J Math Psych 1969; 6: 487.<\/li>\n<li>Metz CE, Herman BA, Shen J-H. Maximum-likelihood estimation of ROC curves from continuously-distributed data. Stat Med 1998; 17: 1033.<\/li>\n<li>Metz CE, Herman BA, Roe CA. Statistical comparison of two ROC curve estimates obtained from partially-paired datasets. Med Decis Making 1998; 18: 110.<\/li>\n<li>Metz CE. Statistical analysis of ROC data in evaluating diagnostic performance. In: Multiple regression analysis: applications in the health sciences (D Herbert and R Myers, eds.). New York: American Institute of Physics, 1986, pp. 365.<\/li>\n<li>Metz CE. Quantification of failure to demonstrate statistical significance: the usefulness of confidence intervals. Invest Radiol 1993; 28: 59.<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">LABMRMC &amp; MRMC<\/p>\n<ul style=\"text-align: justify;\">\n<li>Dorfman DD, Berbaum KS, Metz CE. ROC rating analysis: generalization to the population of readers and cases with the jackknife method. Invest Radiol 1992; 27: 723.<\/li>\n<li>Dorfman DD, Metz CE. Multi-reader multi-case ROC analysis: comments on Begg\u2019s commentary. Academic Radiol 1995; 2 (Supplement 1): S76.<\/li>\n<li>Roe CA, Metz CE. The Dorfman-Berbaum-Metz method for statistical analysis of multi-reader, multi-modality ROC data: validation by computer simulation. Academic Radiol 1997; 4: 298.<\/li>\n<li>Roe CA, Metz CE. Variance-component modeling in the analysis of receiver operating characteristic index estimates. Academic Radiol 1997; 4: 587.<\/li>\n<li>Hillis SL, Obuchowski NA, Schartz KM, Berbaum KS. A comparison of the Dorfman-Berbaum-Metz and Obuchowski-Rockette methods for receiver operating characteristic (ROC) data. Stat Med 2005; 24:1579-1607.<\/li>\n<li>Hillis SL, Berbaum KS. Monte Carlo validation of the Dorfman-Berbaum-Metz method using normalized pseudovalues and less data-based model simplification. Academic Radiology 2005; 12:1534-1541.<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">LABROC4<\/p>\n<ul style=\"text-align: justify;\">\n<li>Metz CE, Herman BA, Shen J-H. Maximum-likelihood estimation of ROC curves from continuously-distributed data. Stat Med 1998; 17: 1033.<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">ROCPWR<\/p>\n<ul style=\"text-align: justify;\">\n<li>Metz CE, Wang P-L, Kronman HB. A new approach for testing the significance of differences between ROC curves measured from correlated data. In: Information processing in medical imaging (F Deconinck, ed.). The Hague: Nijhoff, 1984, p. 432.<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">PROPROC<\/p>\n<ul>\n<li style=\"text-align: justify;\">Pan X, Metz CE. The &#8220;proper&#8221; binormal model: parametric ROC curve estimation with degenerate data. Academic Radiol 1997; 4: 380.<\/li>\n<li style=\"text-align: justify;\">Metz CE, Pan X. &#8220;Proper&#8221; binormal ROC curves: theory and maximum-likelihood estimation. J Math Psych 1999; 43: 1.<\/li>\n<li style=\"text-align: justify;\">Pesce LL, Metz CE. Reliable and computationally efficient maximum-likelihood estimation of &#8220;proper&#8221; binormal ROC curves. Acad Radiol 2007; 14:814\u2013829.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Some Papers appear more than once because they belong to multiple classifications Background Egan JP. Signal detection theory and ROC analysis. New York: Academic Press, 1975. Fryback DG, Thornbury JR. The efficacy of diagnostic imaging. Med Decis Making 1991; 11: 88. Green DM, Swets JA. Signal detection theory and psychophysics. New York, NY: Wiley, 1966. &hellip; <a href=\"http:\/\/imgcom.jsrt.or.jp\/rocGroup\/publications\/readings-in-roc-analysis-with-emphasis-on-medical-applications\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Readings in ROC Analysis, with Emphasis on Medical Applications<\/span> <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":89,"menu_order":1,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"_links":{"self":[{"href":"http:\/\/imgcom.jsrt.or.jp\/rocGroup\/wp-json\/wp\/v2\/pages\/91"}],"collection":[{"href":"http:\/\/imgcom.jsrt.or.jp\/rocGroup\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"http:\/\/imgcom.jsrt.or.jp\/rocGroup\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"http:\/\/imgcom.jsrt.or.jp\/rocGroup\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/imgcom.jsrt.or.jp\/rocGroup\/wp-json\/wp\/v2\/comments?post=91"}],"version-history":[{"count":2,"href":"http:\/\/imgcom.jsrt.or.jp\/rocGroup\/wp-json\/wp\/v2\/pages\/91\/revisions"}],"predecessor-version":[{"id":93,"href":"http:\/\/imgcom.jsrt.or.jp\/rocGroup\/wp-json\/wp\/v2\/pages\/91\/revisions\/93"}],"up":[{"embeddable":true,"href":"http:\/\/imgcom.jsrt.or.jp\/rocGroup\/wp-json\/wp\/v2\/pages\/89"}],"wp:attachment":[{"href":"http:\/\/imgcom.jsrt.or.jp\/rocGroup\/wp-json\/wp\/v2\/media?parent=91"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}