Optimal thresholding for binary classification applied in credit scoring

Agnieszka Rossa

doi:10.18778/2391-6478.2.50.03

Authors

Agnieszka Rossa Institute of Statistics and Demography, University of Lodz https://orcid.org/0000-0002-0444-4181

DOI:

https://doi.org/10.18778/2391-6478.2.50.03

Keywords:

binary classification, sequential random sampling, sensitivity and specificity, time-dependent ROC curves

Abstract

The paper concerns a new method of classifying individuals into two subpopulations and demonstrates the application of this method in credit scoring. Individuals are classified into two subpopulations depending on the duration of a certain phenomenon (e.g., default). The duration may be shorter or longer than a certain fixed value . It is assumed that the variable is not known at the time of classification, so the explanatory continuous predictive marker is used instead. The optimal acceptance threshold for a predictive marker is determined by a time-dependent receiver operating curve (ROC) estimated from a random sample. A typical complexity of time-to-event data is that observations in the sample can be right-censored. Therefore, the estimation is based on a sequential random sampling and the Kaplan-Meier estimator.

Downloads

Download data is not yet available.

References

Abellan, J. & Castellano, J.G. (2017). A comparative study on base classifiers in ensemble methods for credit scoring. Expert Systems with Applications 73(1): 1–10. https://doi.org/10.1016/j.eswa.2016.12.020.

Altman, D.G. & Bland, J.M. (1994). Diagnostic tests. 1: Sensitivity and specificity. British Medical Journal 308(6943): 1552. https://doi.org/10.1136/bmj.308.6943.1552.

Altman, E. (1968). Financial ratios, discriminant analysis and the prediction of corporate bankruptcy. Journal of Finance 23(4): 589–609. https://doi.org/10.1111/j.1540-6261.1968.tb00843.x.

Baker, S.G. (2003), The central role of receiver operating characteristic (roc) curves in evaluating tests for the early detection of cancer. Journal of the National Cancer Institute 95(7): 511–515. https://doi.org/10.1093/jnci/95.7.511.

Breiman, L., Friedman, J.H., Olshen, R.A., et al. (1984), Classification and Regression Trees. Chapman and Hall/CRC, New York.

Chambless, L. & Diao, G. (2006), Estimation of time-dependent area under the roc curve for long-term risk. Statistics in Medicine 25(20): 3474–3486. https://doi.org/10.1002/sim.2299.

Crook, J.N., Edelman, D.B. & Thomas, L.C. (2007), Recent developments in consumer credit risk assessment. European Journal of Operational Research 183(3): 1447–1465. https://doi.org/10.1016/j.ejor.2006.09.100.

Diaz-Quijano, F.A. (2012), A simple method for estimating relative risk using logistic regression. BMC Medical Research Methodology 12(14): 1471–2288. https://doi.org/10.1186/1471-2288-12-14.

Dickie, G.L. (1994), Statistical notes. defining sensitivity and specificity. British Medical Journal 309(6953): 539. https://doi.org/10.1136/bmj.309.6953.539a.

Fisher, R.A. (1936), The use of multiple measurement in taxonomic problems. Annals of Eugenics 7(2): 179–188. https://doi.org/10.1111/j.1469-1809.1936.tb02137.x.

Godyn, J.J., Tomaszewski, J.E. & Zmijewski, C.M. (1991), Specificity of a reproducible qualitative urine examination is not a constant test characteristic. American Journal of Clinical Pathology 95(2): 265–266. https://doi.org/10.1093/ajcp/95.2.265.

Green, D.M. & Swets, J.A. (1966), Signal detection theory and psychophysics. Wiley, New York.

Hand, D.J. & Henley, W.E. (1997), Statistical classification methods in consumer credit scoring: a review. Journal of the Royal Statistical Society A: Statistics in Society 160(3): 523–541. https://doi.org/10.1111/j.1467-985X.1997.00078.x.

Heagerty, P.J., Lumley, T. & Pepe, M.S. (2000), Time-dependent roc curves for censored survival data and a diagnostic marker. Biometrics 56(2): 337–344. https://doi.org/10.1111/j.0006-341x.2000.00337.x.

Hu, Y.C. & Ansell, J. (2007), Measuring retail company performance using credit scoring techniques. European Journal of Operational Research 183(3): 1595–1606. https://doi.org/10.1016/j.ejor.2006.09.101.

Hung, H. & Chiang, C. (2010a), Estimation methods for time-dependent AUC models with survival data. Canadian Journal of Statistics 38(1): 8–26. https://doi.org/10.1002/cjs.10046.

Hung, H. & Chiang, C. (2010b), Optimal composite markers for time-dependent receiver operating characteristic curves with censored survival data. Scandinavian Journal of Statistics 37(4): 664–679. https://doi.org/10.1111/j.1467-9469.2009.00683.x.

Jamain, A. & Hand, D.J. (2008), Mining supervised classification performance studies: A meta-analytic investigation. Journal of Classification 25(1): 87–112. https://doi.org/10.1007/s00357-008-9003-y.

Kaplan, E.L. & Meier, P. (1958), Nonparametric estimation from incomplete observations. Journal of the American Statistical Association 53(282): 457–481. https://doi.org/10.2307/2281868.

Kovacova, M. & Kliestik, T. (2017), Logit and probit application for the prediction of bankruptcy in Slovak companies. Equilibrium-Quarterly Journal of Economics and Economic Policy 12(4): 775–791. https://doi.org/10.24136/eq.v12i4.40.

Kovacova, M., Kliestik, T., Valaskova, K., et al. (2019), Systematic review of variables applied in bankruptcy prediction models of visegrad group countries. Oeconomia Copernicana 10(4): 743–772. https://doi.org/10.24136/oc.2019.034.

Lloyd, C.J. (1998), Using smoothed receiver operating characteristic curves to summarize and compare diagnostic systems. Journal of the American Statistical Association 93(444): 1356–1364. https://doi.org/10.1080/01621459.1998.10473797.

McNutt, L.A., Wu, C., Xue, X., et al. (2003), Estimating the relative risk in cohort studies and clinical trials of common outcomes. American Journal of Epidemiology 157(10): 940–943. https://doi.org/10.1093/aje/kwg074.

Ohlson, J.A. (1980), Financial ratios and the probabilistic prediction of bankruptcy. Journal of Accounting Research 18(1): 109–131. https://doi.org/10.2307/2490395.

Patil, M. & Durairaj, M. (2015), Risk factors of hypertension among adult men: Evidence from a real world outcomes investigation in a western Indian population. International Journal of Advanced Research 3(7): 274–282. https://doi.org/10.2307/2490395.

Pepe, M.S. (1997), A regression modelling framework for receiver operating characteristic curves in medical diagnostic testing. Biometrika 84(3): 595–608. https://doi.org/10.1093/biomet/84.3.595.

Pepe, M.S. (1998), Three approaches to regression analysis of receiver operating characteristic curves for continuous test results. Biometrics 54(1): 124–135. https://doi.org/10.2307/2534001.

Pepe, M.S. (2000), Receiver operating characteristic methodology. Journal of the American Statistical Association 95(449): 308–311. https://doi.org/10.1080/01621459.2000.10473930.

Rossa, A. (2008), Estimation of survival distributions under right-censoring when sample size is random. Sequential Analysis 27(2): 174–184. https://doi.org/10.1080/0747494080198915.

Rossa, A. (2009), The Nelson-Aalen and Kaplan-Meier estimators under a sequential sampling scheme. Communications in Statistics  Theory and Methods 38(16–17): 3077–3098. https://doi.org/10.1080/03610920902947568.

Tetrault, G. (1991), Sensitivity and specificity of clinical tests. American Journal of Clinical Pathology 96(4): 556. https://doi.org/doi:10.1093/ajcp/96.4.556.

Thomas, L.C. (2000), A survey of credit and behavioural scoring: forecasting financial risk of lending to consumers. International Journal of Forecasting 16(2): 149–172. https://doi.org/10.1016/S0169-2070(00)00034-0.

Thomas, L.C. (2009), Modelling the credit risk for portfolios of consumer loans: Analogies with corporate loan models. Mathematics and Computers in Simulation 79(8): 2525–2534. https://doi.org/10.1016/j.matcom.2008.12.006.

Uno, H., Cai, T., Tian, L., et al. (2007), Evaluating prediction rules for t-year survivors with censored regression models. Journal of the American Statistical Association 102(478): 527–537. https://doi.org/10.1198/016214507000000149.

Zhang, J. & Yu, K.F. (1998), What’s the relative risk? a method of correcting the odds ratio in cohort studies of common outcomes. Journal of the American Medical Association 280(19): 1690–1691. https://doi.org/10.1001/jama.280.19.1690.