Skip to main content
SpringerLink
Log in

An Adaptive Simulated Annealing-Based Machine Learning Approach for Developing an E-Triage Tool for Hospital Emergency Operations

  • Published:
Information Systems Frontiers Aims and scope Submit manuscript

Abstract

Patient triage at emergency departments (EDs) is necessary to prioritize care for patients with critical and time-sensitive conditions. In this paper, the metaheuristic optimization algorithms simulated annealing (SA) and adaptive simulated annealing (ASA) are proposed to optimize the parameters of extreme gradient boosting (XGB) and categorical boosting (CaB). The proposed algorithms are SA-XGB, ASA-XGB, SA-CaB, and ASA-CaB. Grid search (GS), a traditional approach used for machine learning fine-tuning, is also used to fine-tune the parameters of XGB and CaB, which are named GS-XGB and GS-CaB. The optimized model is used to develop an e-triage tool that can be used at EDs to predict ED patients' emergency severity index (ESI). The results show ASA-CaB outperformed all the proposed algorithms with accuracy, precision, recall, and f1 of 83.3%, 83.2%, 83.3%, and 83.2%, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Algorithm 2
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data Availability

The IRB, as approved, does not allow for data sharing. Those interested in access to the data should contact the corresponding author for the proper process.

References

  • Ahmed, A., Ashour, O., Ali, H., & Firouz, M. (2022). An integrated optimization and machine learning models to predict the admission status of emergency patients. Expert Systems with Applications, 117314. https://doi.org/10.1016/j.eswa.2022.117314

  • Ali, H., Cole, A., Ahmed, A., Hamasha, S., & Panos, G. (2020). Major stressors and coping strategies of frontline nursing staff during the outbreak of Coronavirus Disease 2020 (COVID-19) in Alabama. Journal of Multidisciplinary Healthcare, 13, 2057–2068. https://doi.org/10.2147/JMDH.S285933

    Article  Google Scholar 

  • Al-Sarem, M., Saeed, F., Boulila, W., Emara, A. H., Al-Mohaimeed, M., & Errais, M. (2021). Feature selection and classification using CatBoost method for improving the performance of predicting Parkinson’s disease. In F. Saeed, T. Al-Hadhrami, F. Mohammed, & E. Mohammed (Eds.), Advances on smart and soft computing (pp. 189–199). Springer. https://doi.org/10.1007/978-981-15-6048-4_17

    Chapter  Google Scholar 

  • Andradóttir, S. (2015). A review of random search methods. In Handbook of Simulation Optimization, ser. International Series in Operations Research & Management Science, M. C. Fu, Ed. Springer New York, 216, 277–292.

  • Arora, M., Sharma, A., Katoch, S., Malviya, M., & Chopra, S. (2021). A State of the Art Regressor Model’s comparison for Effort Estimation of Agile software. In 2021 2nd International Conference on Intelligent Engineering and Management (ICIEM) (pp. 211–215). Presented at the 2021 2nd International Conference on Intelligent Engineering and Management (ICIEM). https://doi.org/10.1109/ICIEM51511.2021.9445345

  • Ashwood, Z., & Myers, M. (2018) Proteomic breast cancer classification. https://zashwood.github.io/pdfs/proteomic_classification.pdf

  • Azizi, N., & Zolfaghari, S. (2004). Adaptive temperature control for simulated annealing: A comparative study. Computers & Operations Research, 31(14), 2439–2451.

    Article  Google Scholar 

  • Badrouchi, S., Ahmed, A., Mongi Bacha, M., Abderrahim, E., & Ben Abdallah, T. (2021). A machine learning framework for predicting long-term graft survival after kidney transplantation. Expert Systems with Applications, 182, 115235. https://doi.org/10.1016/j.eswa.2021.115235

    Article  Google Scholar 

  • Bereta, M. (2019). Regularization of boosted decision stumps using tabu search. Applied Soft Computing, 79, 424–438.

    Article  Google Scholar 

  • Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of machine learning research, 13(2), 281–305.

  • Bergstra, J., Bardenet, R., Bengio, Y., & Kégl, B. (2011). Algorithms for hyper-parameter optimization. In 25th annual conference on neural information processing systems (NIPS 2011) (Vol. 24). Neural Information Processing Systems Foundation.

  • Bibaeva, V. (2018). Using Metaheuristics for Hyper-Parameter Optimization of Convolutional Neural Networks. In 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP) (pp. 1–6). Presented at the 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP). https://doi.org/10.1109/MLSP.2018.8516989

  • Cai, J., Luo, J., Wang, S., & Yang, S. (2018). Feature selection in machine learning: A new perspective. Neurocomputing, 300, 70–79. https://doi.org/10.1016/j.neucom.2017.11.077

    Article  Google Scholar 

  • Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785–794). Association for Computing Machinery. https://doi.org/10.1145/2939672.2939785

  • Chen, J., Zhao, F., Sun, Y., & Yin, Y. (2020). Improved XGBoost model based on genetic algorithm. International Journal of Computer Applications in Technology. https://www.inderscienceonline.com/doi/abs/10.1504/IJCAT.2020.106571. Accessed 5 June 2021.

  • Chonde, S. J., Ashour, O. M., Nembhard, D. A., & Kremer, G. E. O. (2013). Model comparison in emergency severity index level prediction. Expert Systems with Applications, 40(17), 6901–6909. https://doi.org/10.1016/j.eswa.2013.06.026

    Article  Google Scholar 

  • Chou, J.-S., Cheng, M.-Y., Wu, Y.-W., & Pham, A.-D. (2014). Optimizing parameters of support vector machine using fast messy genetic algorithm for dispute classification. Expert Systems with Applications, 41(8), 3955–3964.

    Article  Google Scholar 

  • Colkesen, I., Kavzoglu, T., Atesoglu, A., Tonbul, H., & Ozturk, M. Y. (2023). Multi-seasonal evaluation of hybrid poplar (P. Deltoides) plantations using Worldview-3 imagery and State-Of-The-Art ensemble learning algorithms. Advances in Space Research, 71(7), 3022–3044. https://doi.org/10.1016/j.asr.2022.10.044

    Article  Google Scholar 

  • Deif, M. A., Solyman, A. A. A., Alsharif, M. H., & Uthansakul, P. (2021). Automated triage system for intensive care admissions during the COVID-19 pandemic using hybrid XGBoost-AHP approach. Sensors, 21(19), 6379. https://doi.org/10.3390/s21196379

    Article  Google Scholar 

  • Dowsland, K. A. (1993). Some experiments with simulated annealing techniques for packing problems. European Journal of Operational Research, 68(3), 389–399.

    Article  Google Scholar 

  • Elhaj, H., Achour, N., Tania, M. H., & Aciksari, K. (2023). A comparative study of supervised machine learning approaches to predict patient triage outcomes in hospital emergency departments. Array, 17, 100281. https://doi.org/10.1016/j.array.2023.100281

    Article  Google Scholar 

  • Fakoor, R., Ladhak, F., Nazi, A., & Huber, M. (2013). Using deep learning to enhance cancer diagnosis and classification. In Proceedings of the international conference on machine learning (Vol. 28, pp. 3937–3949). ACM.

  • Fong, R. Y., Glen, W. S. S., Mohamed Jamil, A. K., Tam, W. W. S., & Kowitlawakul, Y. (2018). Comparison of the emergency severity index versus the patient acuity category scale in an emergency setting. International Emergency Nursing, 41, 13–18. https://doi.org/10.1016/j.ienj.2018.05.001

    Article  Google Scholar 

  • Gaspar, A., Oliva, D., Cuevas, E., Zaldívar, D., Pérez, M., & Pajares, G. (2021). Hyperparameter optimization in a convolutional neural network using metaheuristic algorithms. In D. Oliva, E. H. Houssein, & S. Hinojosa (Eds.), Metaheuristics in machine learning: Theory and applications (pp. 37–59). Springer International Publishing. https://doi.org/10.1007/978-3-030-70542-8_2

    Chapter  Google Scholar 

  • Goto, T., Camargo, C. A., Jr., Faridi, M. K., Freishtat, R. J., & Hasegawa, K. (2019). Machine learning-based prediction of clinical outcomes for children during emergency department triage. JAMA Network Open, 2(1), e186937. https://doi.org/10.1001/jamanetworkopen.2018.6937

    Article  Google Scholar 

  • Guo, B., Hu, J., Wu, W., Peng, Q., & Wu, F. (2019). The Tabu_Genetic algorithm: A novel method for hyper-parameter optimization of learning algorithms. Electronics, 8(5), 579. https://doi.org/10.3390/electronics8050579

    Article  Google Scholar 

  • Guo, J., Yang, L., Bie, R., Yu, J., Gao, Y., Shen, Y., & Kos, A. (2019). An XGBoost-based physical fitness evaluation model using advanced feature selection and Bayesian hyper-parameter optimization for wearable running monitoring. Computer Networks, 151, 166–180. https://doi.org/10.1016/j.comnet.2019.01.026

    Article  Google Scholar 

  • Hinson, J. S., Martinez, D. A., Schmitz, P. S. K., Toerper, M., Radu, D., Scheulen, J., et al. (2018). Accuracy of emergency department triage using the Emergency Severity Index and independent predictors of under-triage and over-triage in Brazil: A retrospective cohort analysis. International Journal of Emergency Medicine, 11(1), 3. https://doi.org/10.1186/s12245-017-0161-8

    Article  Google Scholar 

  • Hoang, N.-D., & Tran, V.-D. (2019). Image processing-based detection of pipe corrosion using texture analysis and metaheuristic-optimized machine learning approach. Computational Intelligence and Neuroscience, 2019, e8097213. https://doi.org/10.1155/2019/8097213

    Article  Google Scholar 

  • Huljanah, M., Rustam, Z., Utama, S., & Siswantining, T. (2019). Feature selection using random forest classifier for predicting prostate cancer. IOP Conference Series: Materials Science and Engineering, 546(5), 052031. https://doi.org/10.1088/1757-899X/546/5/052031

    Article  Google Scholar 

  • Ivanov, O., Wolf, L., Brecher, D., Masek, K., Lewis, E., Liu, S., et al. (2020). Improving emergency department esi acuity assignment using machine learning and clinical natural language processing. https://arxiv.org/abs/2004.05184v2. Accessed 11 Jan 2022.

  • Joseph, J. W., Leventhal, E. L., Grossestreuer, A. V., Wong, M. L., Joseph, L. J., Nathanson, L. A., et al. (2020). Deep-learning approaches to identify critically Ill patients at emergency department triage using limited information. Journal of the American College of Emergency Physicians Open, 1(5), 773–781. https://doi.org/10.1002/emp2.12218

    Article  Google Scholar 

  • Kamrani, F., Ghaemipour, F., Nikravan, M., & Alavi, M. H. (2013). Prevalence of miss triage and outcomes under triage of patients in emergency department. 2(3), 17–23.

  • Kauten, C., Gupta, A., Qin, X., & Richey, G. (2022). Predicting blood donors using machine learning techniques. Information Systems Frontiers, 24(5), 1547–1562. https://doi.org/10.1007/s10796-021-10149-1

    Article  Google Scholar 

  • Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220(4598), 671–680.

    Article  Google Scholar 

  • Klug, M., Barash, Y., Bechler, S., Resheff, Y. S., Tron, T., Ironi, A., et al. (2020). A gradient boosting machine learning model for predicting early mortality in the emergency department triage: Devising a nine-point triage score. Journal of General Internal Medicine, 35(1), 220–227. https://doi.org/10.1007/s11606-019-05512-7

    Article  Google Scholar 

  • Levin, S., Toerper, M., Hamrock, E., Hinson, J. S., Barnes, S., Gardner, H., et al. (2018). Machine-learning-based electronic triage more accurately differentiates patients with respect to clinical outcomes compared with the emergency severity index. Annals of Emergency Medicine, 71(5), 565-574.e2. https://doi.org/10.1016/j.annemergmed.2017.08.005

    Article  Google Scholar 

  • Lin, X., Ke, S., Li, Z., Weng, H., & Han, X. (2010). A Fault Diagnosis Method of Power Systems Based on Improved Objective Function and Genetic Algorithm-Tabu Search. IEEE Transactions on Power Delivery, 25(3), 1268–1274. Presented at the IEEE Transactions on Power Delivery. https://doi.org/10.1109/TPWRD.2010.2044590

  • Liu, S., Hobgood, C., & Brice, J. H. (2003). Impact of critical bed status on emergency department patient flow and overcrowding. Academic Emergency Medicine, 10(4), 382–385. https://doi.org/10.1111/j.1553-2712.2003.tb01353.x

    Article  Google Scholar 

  • Liu, P., Fu, B., Yang, S. X., Deng, L., Zhong, X., & Zheng, H. (2021). Optimizing Survival Analysis of XGBoost for Ties to Predict Disease Progression of Breast Cancer. IEEE Transactions on Biomedical Engineering, 68(1), 148–160. Presented at the IEEE Transactions on Biomedical Engineering. https://doi.org/10.1109/TBME.2020.2993278

  • Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., et al. (2020). From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence, 2(1), 56–67. https://doi.org/10.1038/s42256-019-0138-9

    Article  Google Scholar 

  • Ly, N., & McCaig, L. F. (2000). National hospital ambulatory medical care survey: 2000 outpatient department summary. Advance data from vital and health statistics, 327, 1–27.

  • Minggawati, Z. A., Faried, A., & Priambodo, A. P. (2020). Comparison of four-level modification triage with five level Emergency Severity Index (ESI) triage based on level of accuracy and time Triase. Jurnal Keperawatan Padjadjaran, 8(1), 49–55.

    Article  Google Scholar 

  • Morley, C., Unwin, M., Peterson, G. M., Stankovich, J., & Kinsman, L. (2018). Emergency department crowding: A systematic review of causes, consequences and solutions. PLoS One, 13(8), e0203316. https://doi.org/10.1371/journal.pone.0203316

    Article  Google Scholar 

  • Mu, L., Sugumaran, V., & Wang, F. (2020). A hybrid genetic algorithm for software architecture re-modularization. Information Systems Frontiers, 22(5), 1133–1161. https://doi.org/10.1007/s10796-019-09906-0

    Article  Google Scholar 

  • Mun, S.-K., & Chang, M. (2021). Development of prediction models for the incidence of pediatric acute otitis media using Poisson regression analysis and XGBoost. Environmental Science and Pollution Research. https://doi.org/10.1007/s11356-021-17135-9

    Article  Google Scholar 

  • Nkiama, H., Said, S. Z. M., & Saidu, M. (2016). A subset feature elimination mechanism for intrusion detection system. International Journal of Advanced Computer Science and Applications, 7(4), 148–157.

    Article  Google Scholar 

  • Otchere, D. A., Ganat, T. O. A., Ojero, J. O., Tackie-Otoo, B. N., & Taki, M. Y. (2022). Application of gradient boosting regression model for the evaluation of feature selection techniques in improving reservoir characterisation predictions. Journal of Petroleum Science and Engineering, 208, 109244. https://doi.org/10.1016/j.petrol.2021.109244

    Article  Google Scholar 

  • Pham, H. N. A., & Triantaphyllou, E. (2011). A meta-heuristic approach for improving the accuracy in some classification algorithms. Computers & Operations Research, 38(1), 174–189. https://doi.org/10.1016/j.cor.2010.04.011

    Article  Google Scholar 

  • Platts-Mills, T. F., Travers, D., Biese, K., McCall, B., Kizer, S., LaMantia, M., et al. (2010). Accuracy of the Emergency Severity Index Triage instrument for identifying elder emergency department patients receiving an immediate life-saving intervention. Academic Emergency Medicine, 17(3), 238–243. https://doi.org/10.1111/j.1553-2712.2010.00670.x

    Article  Google Scholar 

  • Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A. V., & Gulin, A. (2017). CatBoost: unbiased boosting with categorical features. arXiv preprint arXiv:1706.09516.

  • Putatunda, S., & Rama, K. (2019). A modified Bayesian optimization based hyper-parameter tuning approach for extreme gradient boosting. Fifteenth International Conference on Information Processing (ICINPRO), 2019, 1–6. https://doi.org/10.1109/ICInPro47689.2019.9092025

    Article  Google Scholar 

  • Raita, Y., Goto, T., Faridi, M. K., Brown, D. F. M., Camargo, C. A., & Hasegawa, K. (2019). Emergency department triage prediction of clinical outcomes using machine learning models. Critical Care (London, England), 23(1), 64. https://doi.org/10.1186/s13054-019-2351-7

    Article  Google Scholar 

  • Safarov, R. Z., Shomanova, Z. K., Nossenko, Y. G., Berdenov, Z. G., Bexeitova, Z. B., Shomanov, A. S., & Mansurova, M. (2020). Solving of classification problem in spatial analysis applying the technology of gradient boosting Catboost. Folia Geographica, 62(1), 112–126.

    Google Scholar 

  • Sarkar, S., Vinay, S., Raj, R., Maiti, J., & Mitra, P. (2019). Application of optimized machine learning techniques for prediction of occupational accidents. Computers & Operations Research, 106, 210–224.

    Article  Google Scholar 

  • Sekandari, Z., & Saleh, S. (2019). Emergency department triage prediction of emergency severity index using machine learning models. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-259402. Accessed 11 Jan 2022.

  • Singer, R. F., Infante, A. A., Oppenheimer, C. C., West, C. A., & Siegel, B. (2012). The use of and satisfaction with the emergency severity index. Journal of Emergency Nursing, 38(2), 120–126. https://doi.org/10.1016/j.jen.2010.07.004

    Article  Google Scholar 

  • Snoek, J., Rippel, O., Swersky, K., Kiros, R., Satish, N., Sundaram, N., et al. (2015). Scalable bayesian optimization using deep neural networks. In International conference on machine learning (pp. 2171–2180). PMLR.

  • SoRelle, R. (2006). Homicide charges against Illinois ED stun EM. Emergency Medicine News, 28(12), 1–35.

    Google Scholar 

  • Tsai, C.-C., & Li, S. H. (2009). A two-stage modeling with genetic algorithms for the nurse scheduling problem. Expert Systems with Applications, 36(5), 9506–9512.

    Article  Google Scholar 

  • Tsai, C.-W., Hsia, C.-H., Yang, S.-J., Liu, S.-J., & Fang, Z.-Y. (2020). Optimizing hyperparameters of deep learning in predicting bus passengers based on simulated annealing. Applied Soft Computing, 88, 106068. https://doi.org/10.1016/j.asoc.2020.106068

    Article  Google Scholar 

  • Tutun, S., Johnson, M. E., Ahmed, A., Albizri, A., Irgil, S., Yesilkaya, I., et al. (2022). An AI-based decision support system for predicting mental health disorders. Information Systems Frontiers. https://doi.org/10.1007/s10796-022-10282-5

    Article  Google Scholar 

  • Wuerz, R. C., Milne, L. W., Eitel, D. R., Travers, D., & Gilboy, N. (2000). Reliability and validity of a new five-level triage instrument. Academic Emergency Medicine, 7(3), 236–242. https://doi.org/10.1111/j.1553-2712.2000.tb01066.x

    Article  Google Scholar 

  • XGBoost Parameters — xgboost 1.4.0-SNAPSHOT documentation (2021). https://xgboost.readthedocs.io/en/latest/parameter.html#parameters-for-tree-booster. Accessed 28 February 2022

  • Yahia, N. B., Hlel, J., & Colomo-Palacios, R. (2021). From big data to deep data to support people analytics for employee attrition prediction. IEEE Access, 9, 60447–60458. Presented at the IEEE Access. https://doi.org/10.1109/ACCESS.2021.3074559

  • Yang, L., & Shami, A. (2020). On hyperparameter optimization of machine learning algorithms: Theory and practice. Neurocomputing, 415, 295–316. https://doi.org/10.1016/j.neucom.2020.07.061

    Article  Google Scholar 

  • Yu, C.-S., Lin, Y.-J., Lin, C.-H., Lin, S.-Y., Wu, J. L., & Chang, S.-S. (2020). Development of an online health care assessment for preventive medicine: A machine learning approach. Journal of Medical Internet Research, 22(6), e18585. https://doi.org/10.2196/18585

    Article  Google Scholar 

  • Zhou, J., Qiu, Y., Zhu, S., Armaghani, D. J., Li, C., Nguyen, H., & Yagiz, S. (2021). Optimization of support vector machine through the use of metaheuristic algorithms in forecasting TBM advance rate. Engineering Applications of Artificial Intelligence, 97, 104015. https://doi.org/10.1016/j.engappai.2020.104015

    Article  Google Scholar 

Download references

Acknowledgements

We would like to extend our sincere appreciation to the editors and reviewers of the journal for their valuable time, expertise, and thoughtful feedback during the review process. Their insightful comments and constructive suggestions have significantly contributed to improving the quality and clarity of this paper.

Author information

Authors and Affiliations

  1. Department of Health Services Administration, School of Health Professions, The University of Alabama at Birmingham, Birmingham, AL, USA

    Abdulaziz Ahmed

  2. University of Passau, Passau, Germany

    Mohammed Al-Maamari

  3. Department of Management, Information Systems & Quantitative Methods, Collat School of Business, The University of Alabama at Birmingham, Birmingham, AL, USA

    Mohammad Firouz

  4. Center for Health Systems Innovation, Spears School of Business, Oklahoma State University, Stillwater, OK, USA

    Dursun Delen

  5. Department of Industrial Engineering, Faculty of Engineering and Natural Sciences, Istinye University, Istanbul, Turkey

    Dursun Delen

Authors
  1. Abdulaziz Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammed Al-Maamari
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Firouz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dursun Delen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Abdulaziz Ahmed: Conceptualization, Methodology, Software, Formal analysis, Investigation, Data Curation and Preprocessing, Writing—Original Draft, Writing—Review & Editing, Visualization, Supervision. Mohammed Al-Maamari: Writing—Original Draft. Mohammad Firouz: Writing—Review & Editing, Formal Analysis. Dursun Delen: Writing—Review & Editing.

Corresponding author

Correspondence to Abdulaziz Ahmed.

Ethics declarations

Ethics Approval and Consent to Participate

The data presented in this study were obtained following the approval of the research protocol by the University of Minnesota IRB. The IRB number is STUDY00008314. The study was conducted in accordance with the ethical principles and guidelines set forth by the committee.

Consent for Publication

All the authors declare their consent for publication.

Competing Interests

None declared.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmed, A., Al-Maamari, M., Firouz, M. et al. An Adaptive Simulated Annealing-Based Machine Learning Approach for Developing an E-Triage Tool for Hospital Emergency Operations. Inf Syst Front (2023). https://doi.org/10.1007/s10796-023-10431-4

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10796-023-10431-4

Keywords

Search

Navigation