Machine-Learning Classification of Archived Employability-Readiness Ratings: Performance, Calibration, and Internal Validation
Article Sidebar
Main Article Content
Abstract
This study examined the explanatory structure and predictive reproducibility of an end-of-program administrative score used to assess graduate preparedness in vocational education. A cross-sectional secondary analysis was conducted using 450 de-identified learner records collected from four anonymized polytechnic-style institutions during the 2023–2024 academic year. The outcome was a 0–100 composite score analyzed continuously and as three prespecified categories: low (<60), moderate (60–79), and high (≥80). Candidate predictors comprised practical skills, digital competence, soft skills, internship intensity, project performance, and an industry–education integration index treated as an institution-linked contextual proxy. Pearson correlations, heteroskedasticity-consistent ordinary least-squares regression, criterion-overlap sensitivity analysis, and leakage-controlled model development were applied. Six algorithms were compared against a majority-class baseline using a stratified 70:30 development-test split. Bootstrap intervals, ordinal-error measures, and probability-calibration indices were used to quantify uncertainty and performance. The full regression model explained 76% of outcome variance (adjusted R² = 0.75), whereas the reduced model retained an adjusted R² of 0.58 after removal of predictors susceptible to shared rubric content. In the untouched test partition (n = 135), the back-propagation neural network achieved an accuracy of 0.904 and a macro-F1 of 0.903; all 13 errors occurred between adjacent categories. These findings indicate a coherent within-system scoring structure and strong reproducibility across analytical approaches. However, they do not establish causal effects, transportability across institutions, fairness, operational utility, or prediction of subsequent labor-market outcomes. The model should therefore be restricted to low-stakes auditing, data-quality review, and identification of borderline records pending prospective multi-institutional evaluation.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Copyright of the published article belongs to the authors and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlike 4.0 (CC BY SA) International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See the Effect of Open Access).
References
[1] S. Jeon, How Can Innovative Technologies Transform Vocational Education and Training: Insights for Ukraine. Paris, France: OECD Publishing, 2025, https://doi.org/10.1787/fb40f416-en
[2] N. Kholifah et al., “Unlocking workforce readiness through digital employability skills in vocational education Graduates: A PLS-SEM analysis based on human capital Theory,” Soc. Sci. Humanit. Open, vol. 11, p. 101625, 2025, https://doi.org/10.1016/j.ssaho.2025.101625
[3] M. Hardini, S. A. Anjani, S. Triandari, F. Amelia, and M. Rodriguez, “Orchestrating Big Data and Artificial Intelligence for Adaptive Digital Business Strategy,” J. Comput. Sci. Technol. Appl., vol. 3, no. 2, pp. 185–198, 2026, https://doi.org/10.33050/qx8e0j55
[4] M. Clarke, “Rethinking graduate employability: the role of capital, individual attributes and context,” Stud. High. Educ., vol. 43, no. 11, pp. 1923–1937, 2018, https://doi.org/10.1080/03075079.2017.1294152
[5] A. Eimer and C. Bohndick, “Employability models for higher education: A systematic literature review and analysis,” Soc. Sci. Humanit. Open, vol. 8, no. 1, p. 100588, 2023, https://doi.org/10.1016/j.ssaho.2023.100588
[6] M. Tomlinson, “Forms of graduate capital and their relationship to graduate employability,” Educ. Train., vol. 59, no. 4, pp. 338–352, 2017, https://doi.org/10.1108/ET-05-2016-0090
[7] H. Tushar and N. Sooraksa, “Global employability skills in the 21st century workplace: A semi-systematic literature review,” Heliyon, vol. 9, no. 11, 2023, https://doi.org/10.1016/j.heliyon.2023.e21023
[8] K. Peersia, N. A. Rappa, and L. B. Perry, “Work readiness: definitions and conceptualisations,” High. Educ. Res. Dev., vol. 43, no. 8, pp. 1830–1845, 2024, https://doi.org/10.1080/07294360.2024.2366322
[9] J. Borg, C. M. Scott-Young, and T. Bartram, “A Review of Graduate Work Readiness Literature: A Conceptual Exploration of the Implications for HRM Research and Practice,” Asia Pacific J. Hum. Resour., vol. 63, no. 2, p. e70012, 2025, https://doi.org/10.1111/1744-7941.70012
[10] C. L. Caballero et al., “Untangling graduate employability and work readiness to inform evidence-based practice in higher education,” High. Educ. Res. Dev., vol. 45, no. 5, pp. 1247–1264, 2026, https://doi.org/10.1080/07294360.2026.2615303
[11] D. Jackson, “Re-conceptualising graduate employability: The importance of pre-professional identity,” High. Educ. Res. Dev., vol. 35, no. 5, pp. 925–939, 2016, https://doi.org/10.1080/07294360.2016.1139551
[12] S. Messick, “Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning,” Am. Psychol., vol. 50, no. 9, pp. 741–749, 1995, https://doi.org/10.1037/0003-066X.50.9.741
[13] P. M. Skorupiński, “American Educational Research Association, American Psychological Association, National Council on Measurement in Education, Standards for educational and psychological testing,” Kwart. Pedagog., vol. 60, no. 4, pp. 201–203, 2015, [Online]. Available: https://bibliotekanauki.pl/articles/892305
[14] D. Jackson and B. A. Dean, “The contribution of different types of work-integrated learning to graduate employability,” High. Educ. Res. Dev., vol. 42, no. 1, pp. 93–110, 2023, https://doi.org/10.1080/07294360.2022.2048638
[15] P. M. L. Ng, T. M. Wut, and J. K. Y. Chan, “Enhancing perceived employability through work-integrated learning,” Educ. Train., vol. 64, no. 4, pp. 559–576, 2022, https://doi.org/10.1108/ET-12-2021-0476
[16] R. Scandurra, D. Kelly, S. Fusaro, R. Cefalo, and K. Hermannsson, “Do employability programmes in higher education improve skills and labour market outcomes? A systematic review of academic literature,” Stud. High. Educ., vol. 49, no. 8, pp. 1381–1396, 2024, https://doi.org/10.1080/03075079.2023.2265425
[17] P. M. Podsakoff, S. B. MacKenzie, J. Y. Lee, and N. P. Podsakoff, “Common Method Biases in Behavioral Research: A Critical Review of the Literature and Recommended Remedies,” J. Appl. Psychol., vol. 88, no. 5, pp. 879–903, 2003, https://doi.org/10.1037/0021-9010.88.5.879
[18] S. Kapoor and A. Narayanan, “Leakage and the reproducibility crisis in machine-learning-based science,” Patterns, vol. 4, no. 9, 2023, https://doi.org/10.1016/j.patter.2023.100804
[19] D. R. Roberts et al., “Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure,” Ecography (Cop.)., vol. 40, no. 8, pp. 913–929, 2017, https://doi.org/10.1111/ecog.02881
[20] E. Alyahyan and D. Düştegör, “Predicting academic success in higher education: literature review and best practices,” Int. J. Educ. Technol. High. Educ., vol. 17, no. 1, p. 3, 2020, https://doi.org/10.1186/s41239-020-0177-7
[21] M. Kuhn and K. Johnson, Applied predictive modeling, vol. 26. Springer, 2013, https://doi.org/10.1007/978-1-4614-6849-3
[22] X. Wang et al., “Chinese interpretation of PROBAST+AI: An updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods,” Chinese J. Clin. Thorac. Cardiovasc. Surg., vol. 32, no. 12, pp. 1686–1695, 2025, https://doi.org/10.7507/1007-4848.202507088
[23] E. W. Steyerberg et al., “Assessing the performance of prediction models: A framework for traditional and novel measures,” Epidemiology, vol. 21, no. 1, pp. 128–138, 2010, https://doi.org/10.1097/EDE.0b013e3181c30fb2
[24] G. S. Collins et al., “TRIPOD+AI statement: Updated guidance for reporting clinical prediction models that use regression or machine learning methods,” Bmj, vol. 385, 2024, https://doi.org/10.1136/bmj-2023-078378
[25] B. Van Calster et al., “Calibration: The Achilles heel of predictive analytics,” BMC Med., vol. 17, no. 1, p. 230, 2019, https://doi.org/10.1186/s12916-019-1466-7
[26] H. Khosravi et al., “Explainable Artificial Intelligence in education,” Comput. Educ. Artif. Intell., vol. 3, p. 100074, 2022, https://doi.org/10.1016/j.caeai.2022.100074
[27] G. Ramaswami, T. Susnjak, A. Mathrani, and R. Umer, “Use of Predictive Analytics within Learning Analytics Dashboards: A Review of Case Studies,” Technol. Knowl. Learn., vol. 28, no. 3, pp. 959–980, 2023, https://doi.org/10.1007/s10758-022-09613-x
[28] C. Rudin, “Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead,” Nat. Mach. Intell., vol. 1, no. 5, pp. 206–215, 2019, https://doi.org/10.1038/s42256-019-0048-x
[29] R. S. Baker and A. Hawn, “Algorithmic Bias in Education,” Int. J. Artif. Intell. Educ., vol. 32, no. 4, pp. 1052–1092, 2022, https://doi.org/10.1007/s40593-021-00285-9
[30] J. A. Idowu, “Debiasing Education Algorithms,” Int. J. Artif. Intell. Educ., vol. 34, no. 4, pp. 1510–1540, 2024, https://doi.org/10.1007/s40593-023-00389-4
[31] M. Khalil, P. Prinsloo, and S. Slade, “Fairness, Trust, Transparency, Equity, and Responsibility in Learning Analytics,” J. Learn. Anal., vol. 10, no. 1, pp. 1–7, 2023, https://doi.org/10.18608/jla.2023.7983
[32] R. F. Kizilcec and H. Lee, “Algorithmic fairness in education,” in The Ethics of Artificial Intelligence in Education: Practices, Challenges, and Debates, Routledge, 2022, pp. 174–202. https://doi.org/10.4324/9780429329067-10
[33] S. Slade and P. Prinsloo, “Learning Analytics: Ethical Issues and Dilemmas,” Am. Behav. Sci., vol. 57, no. 10, pp. 1510–1529, 2013, https://doi.org/10.1177/0002764213479366
[34] B. Williamson and R. Eynon, “Historical threads, missing links, and future directions in AI in education,” Learning, Media and Technology, vol. 45, no. 3. Taylor & Francis, pp. 223–235, 2020. https://doi.org/10.1080/17439884.2020.1798995
[35] A. Mathrani, T. Susnjak, G. Ramaswami, and A. Barczak, “Perspectives on the challenges of generalizability, transparency and ethics in predictive learning analytics,” Comput. Educ. Open, vol. 2, p. 100060, 2021, https://doi.org/10.1016/j.caeo.2021.100060
[36] M. Fugate, A. J. Kinicki, and B. E. Ashforth, “Employability: A psycho-social construct, its dimensions, and applications,” J. Vocat. Behav., vol. 65, no. 1, pp. 14–38, 2004, https://doi.org/10.1016/j.jvb.2003.10.005
[37] R. L. Brennan, “Commentary on ‘Validating the Interpretations and Uses of Test Scores,’” J. Educ. Meas., vol. 50, no. 1, pp. 74–83, 2013, https://doi.org/10.1111/jedm.12001
[38] A. Rausch et al., “Designing an International Large-Scale Assessment of Professional Competencies and Employability Skills: Emerging Avenues and Challenges of OECD’s PISA-VET,” Vocat. Learn., vol. 17, no. 3, pp. 393–432, 2024, https://doi.org/10.1007/s12186-024-09347-0
[39] R. Vuorikari, S. Kluzer, and Y. Punie, DigComp 2.2: The Digital Competence Framework for Citizens—With New Examples of Knowledge, Skills and Attitudes. Luxembourg: Publications Office of the European Union, 2022, https://doi.org/10.2760/115376
[40] E. van Laar, A. J. A. M. van Deursen, J. A. G. M. van Dijk, and J. de Haan, “The relation between 21st-century skills and digital skills: A systematic literature review,” Comput. Human Behav., vol. 72, pp. 577–588, 2017, https://doi.org/10.1016/j.chb.2017.03.010
[41] C. Succi and M. Canovi, “Soft skills to enhance graduate employability: comparing students and employers’ perceptions,” Stud. High. Educ., vol. 45, no. 9, pp. 1834–1847, 2020, https://doi.org/10.1080/03075079.2019.1585420
[42] W. M. Adegbite, “Unpacking mediation and moderating effect of digital literacy and life-career knowledge in the relationship between work-integrated learning and graduate employability,” Soc. Sci. Humanit. Open, vol. 10, p. 101161, 2024, https://doi.org/10.1016/j.ssaho.2024.101161
[43] D. T. Campbell and D. W. Fiske, “Convergent and discriminant validation by the multitrait-multimethod matrix,” Psychol. Bull., vol. 56, no. 2, pp. 81–105, 1959, https://doi.org/10.1037/h0046016
[44] P. M. Podsakoff, S. B. MacKenzie, and N. P. Podsakoff, “Sources of method bias in social science research and recommendations on how to control it,” Annu. Rev. Psychol., vol. 63, no. 1, pp. 539–569, 2012, https://doi.org/10.1146/annurev-psych-120710-100452
[45] J. F. Binning and G. V. Barrett, “Validity of Personnel Decisions: A Conceptual Analysis of the Inferential and Evidential Bases,” J. Appl. Psychol., vol. 74, no. 3, pp. 478–494, 1989, https://doi.org/10.1037/0021-9010.74.3.478
[46] D. G. Altman and P. Royston, “The cost of dichotomising continuous variables,” Br. Med. J., vol. 332, no. 7549, p. 1080, 2006, https://doi.org/10.1136/bmj.332.7549.1080
[47] G. C. Cawley and N. L. C. Talbot, “On over-fitting in model selection and subsequent selection bias in performance evaluation,” J. Mach. Learn. Res., vol. 11, pp. 2079–2107, 2010, [Online]. Available: https://www.jmlr.org/papers/v11/cawley10a.html
[48] T. Saito and M. Rehmsmeier, “The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets,” PLoS One, vol. 10, no. 3, p. e0118432, 2015, https://doi.org/10.1371/journal.pone.0118432
[49] G. W. Brier, “Verification of forecasts expressed in terms of probability,” Mon. Weather Rev., vol. 78, no. 1, pp. 1–3, 1950, https://doi.org/10.1175/1520-0493(1950)078%3C0001:VOFEIT%3E2.0.CO;2
[50] B. Van Calster, D. Nieboer, Y. Vergouwe, B. De Cock, M. J. Pencina, and E. W. Steyerberg, “A calibration hierarchy for risk models was defined: From utopia to empirical data,” J. Clin. Epidemiol., vol. 74, pp. 167–176, 2016, https://doi.org/10.1016/j.jclinepi.2015.12.005
[51] E. Christodoulou, J. Ma, G. S. Collins, E. W. Steyerberg, J. Y. Verbakel, and B. Van Calster, “A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models,” J. Clin. Epidemiol., vol. 110, pp. 12–22, 2019, https://doi.org/10.1016/j.jclinepi.2019.02.004
[52] J. Cohen, “Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit,” Psychol. Bull., vol. 70, no. 4, pp. 213–220, 1968, https://doi.org/10.1037/h0026256
[53] A. Fisher, C. Rudin, and F. Dominici, “All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously,” J. Mach. Learn. Res., vol. 20, no. 177, pp. 1–81, 2019, [Online]. Available: https://jmlr.org/papers/v20/18-760.html
[54] S. M. Lundberg and S. I. Lee, “A unified approach to interpreting model predictions,” Adv. Neural Inf. Process. Syst., vol. 30, pp. 4766–4775, 2017, [Online]. Available: https://proceedings.neurips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html
[55] M. Ghassemi, L. Oakden-Rayner, and A. L. Beam, “The false hope of current approaches to explainable artificial intelligence in health care,” Lancet Digit. Heal., vol. 3, no. 11, pp. e745–e750, 2021, https://doi.org/10.1016/S2589-7500(21)00208-9
[56] W. Holmes et al., “Ethics of AI in Education: Towards a Community-Wide Framework,” Int. J. Artif. Intell. Educ., vol. 32, no. 3, pp. 504–526, 2022, https://doi.org/10.1007/s40593-021-00239-1
[57] A. D. Selbst, D. Boyd, S. A. Friedler, S. Venkatasubramanian, and J. Vertesi, “Fairness and abstraction in sociotechnical systems,” in FAT 2019 - Proceedings of the 2019 Conference on Fairness, Accountability, and Transparency, 2019, pp. 59–68. https://doi.org/10.1145/3287560.3287598
[58] M. Mitchell et al., “Model cards for model reporting,” in FAT* 2019 - Proceedings of the 2019 Conference on Fairness, Accountability, and Transparency, 2019, pp. 220–229. https://doi.org/10.1145/3287560.3287596
[59] R. F. Wolff et al., “PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies,” 2019, https://doi.org/10.7326/M18-1376
[60] R. D. Riley et al., “Calculating the sample size required for developing a clinical prediction model,” BMJ, vol. 368, 2020, https://doi.org/10.1136/bmj.m441
[61] R. D. Cook, “Detection of Influential Observation in Linear Regression,” Technometrics, vol. 19, no. 1, pp. 15–18, 1977, https://doi.org/10.1080/00401706.1977.10489493
[62] R. J. A. Little, “A test of missing completely at random for multivariate data with missing values,” J. Am. Stat. Assoc., vol. 83, no. 404, pp. 1198–1202, 1988, https://doi.org/10.1080/01621459.1988.10478722
[63] S. van Buuren and K. Groothuis-Oudshoorn, “mice: Multivariate imputation by chained equations in R,” J. Stat. Softw., vol. 45, no. 3, pp. 1–67, 2011, https://doi.org/10.18637/jss.v045.i03
[64] I. R. White, P. Royston, and A. M. Wood, “Multiple imputation using chained equations: Issues and guidance for practice,” Stat. Med., vol. 30, no. 4, pp. 377–399, 2011, https://doi.org/10.1002/sim.4067
[65] B. Efron, “Better bootstrap confidence intervals,” J. Am. Stat. Assoc., vol. 82, no. 397, pp. 171–185, 1987, https://doi.org/10.1080/01621459.1987.10478410
[66] Sture Holm, “A Simple Sequentially Rejective Multiple Test Procedure,” Scand. J. Stat., vol. 6, no. 2, pp. 65–70, 1979, https://doi.org/10.2307/4615733
[67] J. G. MacKinnon and H. White, “Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties,” J. Econom., vol. 29, no. 3, pp. 305–325, 1985, https://doi.org/10.1016/0304-4076(85)90158-7
[68] P. McCullagh, “Regression models for ordinal data,” J. R. Stat. Soc. Ser. B, vol. 42, no. 2, pp. 109–127, 1980, https://doi.org/10.1111/j.2517-6161.1980.tb01109.x
[69] C. Cortes and V. Vapnik, “Support-vector networks,” Mach. Learn., vol. 20, no. 3, pp. 273–297, 1995, https://doi.org/10.1007/BF00994018
[70] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015, https://doi.org/https://doi.org/10.1038/nature14539
[71] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,” J. Mach. Learn. Res., vol. 15, no. 1, pp. 1929–1958, 2014, [Online]. Available: https://www.jmlr.org/papers/v15/srivastava14a.html
[72] B. Efron, “Bootstrap confidence intervals for a class of parametric problems,” Biometrika, vol. 72, no. 1, pp. 45–58, 1985, https://doi.org/10.1093/biomet/72.1.45
[73] R. M. O’Brien, “A caution regarding rules of thumb for variance inflation factors,” Qual. Quant., vol. 41, no. 5, pp. 673–690, 2007, https://doi.org/10.1007/s11135-006-9018-6
[74] O. S. Kalange, R. S. Kahat, A. S. Kale, T. R. Kale, and P. S. Joglekar, “Implementation of Various Machine Learning Algorithms for Traffic Sign Detection and Recognition,” Int. Res. J. Eng. Technol., pp. 356–361, 2022, [Online]. Available: https://www.irjet.net/volume10-issue01
[75] A. N. Muttaqin, M. Lubis, T. Mulhartono, and A. R. Lubis, “Quantifying the Causal Impact of Employment Trends on Academic Performance Using Time-Series and Public Interest Data in Indonesia,” Adv. Sustain. Sci. Eng. Technol., vol. 7, no. 4, 2025, https://doi.org/10.26877/asset.v7i4.2358
[76] T. Erfando and R. Khariszma, “Sensitivity Study of the Effect Polymer Flooding Parameters To Improve Oil Recovery Using X-Gradient Boosting Algorithm,” J. Appl. Eng. Technol. Sci., vol. 4, no. 2, pp. 873–884, 2023, https://doi.org/10.37385/jaets.v4i2.1871