This study introduces a novel simulation-based framework that integrates Agent-Based Modelling (ABM) with Reinforcement Learning (RL) to evaluate and optimize policies for mental health crisis response. As mental health crises become increasingly complex and context-specific, traditional fixed-resource strategies may fail to adapt to evolving population needs. To address this, we simulate a diverse synthetic population characterized by varying demographic attributes such as age and gender, as well as factors like baseline mental health, stress exposure, social support, and access to care. Agents evolve over time based on stochastic stressors and receive interventions through three modelled resource types: hotlines, counselling services, and emergency care. We compare four policy strategies: hotline-only, counselling-only, a mixed-resource approach, and a PPO-trained RL policy designed to dynamically allocate resources based on real-time population states. Each strategy is simulated over a 100-day period. Key evaluation metrics include crisis rate, intervention coverage, unmet need rate, average stress level, total interventions, and a Policy Efficiency Score (PES). Spatial resource usage and demographic subgroup outcomes are also tracked and analysed. Our results reveal that counselling-focused strategies offer the most sustainable balance of low crisis rates and stress levels with moderate intervention coverage. While the RL-optimized policy achieves 100% intervention coverage and zero unmet needs, it also maintains the highest average stress, suggesting an over-saturation of interventions without long-term mental health relief. The findings underscore the importance of not only maximizing access but also prioritizing effective and sustainable care. This framework serves as a decision-support tool to guide public health resource allocation and policy design in crisis settings.
Cite this paper
Filippis, R. D. and Foysal, A. A. (2025). AI-Driven Policy Testing for Mental Health Crisis Response: An Agent-Based Modelling and Reinforcement Learning Approach. Open Access Library Journal, 12, e3507. doi: http://dx.doi.org/10.4236/oalib.1113507.
Chua, B., Al-Ansi, A., Lee, M.J. and Han, H. (2020) Impact of Health Risk Perception on Avoidance of International Travel in the Wake of a Pandemic. Current Issues in Tourism, 24, 985-1002. https://doi.org/10.1080/13683500.2020.1829570
Rey, S.J. and Franklin, R.S. (2022) Introduction: Spatial Analysis and the Social Sciences in a Rapidly Changing Landscape. In: Handbook of Spatial Analysis in the Social Sciences, Edward Elgar Publishing, 11-20.
LeNoble, C., Naranjo, A., Shoss, M. and Horan, K. (2023) Navigating a Context of Severe Un-certainty: The Effect of Industry Unsafety Signals on Employee Well-Being during the COVID-19 Crisis. Occupational Health Science, 7, 707-743. https://doi.org/10.1007/s41542-023-00155-x
Mollica, R., Cardozo, B.L., Osofsky, H., Raphael, B., Ager, A. and Salama, P. (2004) Mental Health in Complex Emergencies. The Lancet, 364, 2058-2067. https://doi.org/10.1016/s0140-6736(04)17519-3
Li, G., Shi, W., Gao, X., Shi, X., Feng, X., Liang, D., et al. (2024) Mental Health and Psychosocial Interventions to Limit the Adverse Psychological Effects of Disasters and Emergencies in China: A Scoping Review. The Lancet Regional Health—Western Pacific, 45, Article ID: 100580. https://doi.org/10.1016/j.lanwpc.2022.100580
Bogdan, G.M., Seroka, A.M., Watson, J. and Johnson, M. (2007) Adapting Community Call Centers for Crisis Support: A Model for Home-Based Care and Monitoring. Agency for Healthcare Research and Quality.
Afroogh, S., Mostafavi, A., Akbari, A., Pouresmaeil, Y., Goudarzi, S., Hajhosseini, F., et al. (2023) Embedded Ethics for Responsible Artificial Intelligence Systems (EE-RAIS) in Disaster Management: A Conceptual Model and Its Deployment. AI and Ethics, 4, 1117-1141. https://doi.org/10.1007/s43681-023-00309-1
Kirchner, T.R. and Shiffman, S. (2016) Spatio-Temporal Determinants of Mental Health and Well-Being: Advances in Geographically-Explicit Ecological Momentary Assessment (GEMA). Social Psychiatry and Psychiatric Epidemiology, 51, 1211-1223. https://doi.org/10.1007/s00127-016-1277-5
Wheaton, B. and Clarke, P. (2003) Space Meets Time: Integrating Temporal and Contextual Influences on Mental Health in Early Adulthood. American Sociological Review, 68, 680-706. https://doi.org/10.1177/000312240306800502
Nelson, B., McGorry, P.D., Wichers, M., Wigman, J.T.W. and Hart-mann, J.A. (2017) Moving from Static to Dynamic Models of the Onset of Mental Disorder: A Review. JAMA Psychiatry, 74, Article No. 528. https://doi.org/10.1001/jamapsychiatry.2017.0001
Cummins, S., Curtis, S., Diez-Roux, A.V. and Macintyre, S. (2007) Understanding and Representing “Place” in Health Research: A Relational Approach. Social Science & Medicine, 65, 1825-1838. https://doi.org/10.1016/j.socscimed.2007.05.036
Patil, Y.M., Abraham, A.R., Chaubey, N.K., K., B. and Chidambaranathan, S. (2024) A Comparative Analysis of Machine Learning Techniques in Creating Virtual Replicas for Healthcare Simulations. In: Ponnusamy, S., et al., Eds., Harnessing AI and Digital Twin Technologies in Busi-nesses, IGI Global, 14-25. https://doi.org/10.4018/979-8-3693-3234-4.ch002
Alhuwaydi, A. (2024) Exploring the Role of Artificial Intelligence in Mental Healthcare: Current Trends and Future Directions—A Narrative Review for a Com-prehensive Insight. Risk Management and Healthcare Policy, 17, 1339-1348. https://doi.org/10.2147/rmhp.s461562
Koutsouleris, N., Hauser, T.U., Skvortsova, V. and De Choudhury, M. (2022) From Promise to Practice: Towards the Realisation of AI-Informed Mental Health Care. The Lancet Digital Health, 4, e829-e840. https://doi.org/10.1016/s2589-7500(22)00153-4
Karunakaran, M., Venkatachalam, C., Mahesh, T.R., Krishnan, B. and Nagaraj, S. (2024) Chapter 5. Machine Learning for Twinning the Human Body. In: Malviya, R., et al., Eds., Digital Transformation in Healthcare 5.0, De Gruyter, 105-130. https://doi.org/10.1515/9783111398549-005
Tracy, M., Cerdá, M. and Keyes, K.M. (2018) Agent-Based Modeling in Public Health: Current Applications and Future Directions. Annual Review of Public Health, 39, 77-94. https://doi.org/10.1146/annurev-publhealth-040617-014317
Silverman, B.G., Hanrahan, N., Bharathy, G., Gordon, K. and Johnson, D. (2015) A Systems Approach to Healthcare: Agent-Based Modeling, Community Mental Health, and Pop-ulation Well-Being. Artificial Intelligence in Medicine, 63, 61-71. https://doi.org/10.1016/j.artmed.2014.08.006
Ali, H. (2022) Reinforcement Learning in Healthcare: Optimizing Treatment Strategies, Dynamic Resource Allocation, and Adap-tive Clinical Decision-Making. International Journal of Computer Applications Technology and Research, 11, 88-104.
Yan, Y., Zhang, B. and Guo, J. (2016) An Adaptive Decision Making Approach Based on Reinforcement Learning for Self-Managed Cloud Applications. 2016 IEEE International Conference on Web Services (ICWS), San Francisco, 27 June-2 July 2016, 720-723. https://doi.org/10.1109/icws.2016.102
Lewis, F.L., Vrabie, D. and Vamvoudakis, K.G. (2012) Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Con-trollers. IEEE Control Systems Magazine, 32, 76-105.
Hussin, M., Asilah Wati Abdul Hamid, N. and Kasmiran, K.A. (2015) Improving Reliability in Resource Management through Adaptive Reinforcement Learning for Distributed Systems. Journal of Parallel and Distributed Computing, 75, 93-100. https://doi.org/10.1016/j.jpdc.2014.10.001
Marcus, N. and Stergiopoulos, V. (2022) Re‐Examining Mental Health Crisis Intervention: A Rapid Review Comparing Outcomes across Police, Co‐Responder and Non‐Police Models. Health & Social Care in the Community, 30, 1665-1679. https://doi.org/10.1111/hsc.13731
Steadman, H.J., Deane, M.W., Borum, R. and Morrissey, J.P. (2000) Comparing Outcomes of Major Models of Police Responses to Mental Health Emergencies. Psychiatric Services, 51, 645-649. https://doi.org/10.1176/appi.ps.51.5.645
Paton, F., Wright, K., Ayre, N., Dare, C., Johnson, S., Lloyd-Evans, B., et al. (2016) Improving Outcomes for People in Mental Health Crisis: A Rapid Synthesis of the Evidence for Available Models of Care. Health Technology Assessment, 20, 1-162. https://doi.org/10.3310/hta20030
Boonekamp, P. (2006) Actual Interaction Effects between Policy Measures for Energy Efficiency—A Qualitative Matrix Method and Quantitative Simulation Results for Households. Energy, 31, 2848-2873. https://doi.org/10.1016/j.energy.2006.01.004
Pan, X.H., et al. (2022) Mate: Benchmarking Multi-Agent Reinforce-ment Learning in Distributed Target Coverage Control. Advances in Neural Information Processing Systems (NeurIPS 2021), 6-14 December 2021, 27862-27879.
Aydemir, F. and Cetin, A. (2023) Multi-Agent Dynamic Area Coverage Based on Reinforcement Learning with Connected Agents. Computer Systems Science and Engineering, 45, 215-230. https://doi.org/10.32604/csse.2023.031116
Din, A., Ismail, M.Y., Shah, B., Babar, M., Ali, F. and Baig, S.U. (2022) A Deep Reinforcement Learning-Based Multi-Agent Area Coverage Control for Smart Agriculture. Computers and Electrical Engineering, 101, Article ID: 108089. https://doi.org/10.1016/j.compeleceng.2022.108089
Slade, M., Amering, M., Farkas, M., Hamilton, B., O’Hagan, M., Panther, G., et al. (2014) Uses and Abuses of Recovery: Implementing Recov-ery-Oriented Practices in Mental Health Systems. World Psychiatry, 13, 12-20. https://doi.org/10.1002/wps.20084
Duffy, P. and Baldwin, H. (2013) Recovery Post Treatment: Plans, Barriers and Motivators. Substance Abuse Treatment, Prevention, and Policy, 8, 1-12. https://doi.org/10.1186/1747-597x-8-6
Santangelo, P., Procter, N. and Fassett, D. (2017) Mental Health Nursing: Daring to Be Different, Special and Leading Recovery‐Focused Care? International Journal of Mental Health Nursing, 27, 258-266. https://doi.org/10.1111/inm.12316
Glegg, S.M.N. and Levac, D.E. (2018) Barriers, Facilitators and Inter-ventions to Support Virtual Reality Implementation in Rehabilitation: A Scoping Review. PM&R, 10, 1237-1251. https://doi.org/10.1016/j.pmrj.2018.07.004
Kunvardia, N. (2017) A Service Evaluation Study Exploring the Thera-peutic Effectiveness of a Reiki Intervention in the Local Community of Cancer Patients. PhD Diss., Queen Margaret Universi-ty.
Javeth, A., Salina, S. and Joshi, P. (2024) Dr. MT Bhatia Award Winner-Athar Javeth Aromatherapy for the Management of Cancer-Related. Indian Journal of Palliative Care, 30, 121.
Miller, N. (2015) Creating Opportunity after Crisis: Examining the Development of the Post-Earthquake Haitian Mental Health Care System. California Southern University.
Berger, T. (2001) Agent-Based Spatial Models Applied to Agriculture: A Simulation Tool for Technology Diffusion, Resource Use Changes and Policy Analysis. Agricultural Economics, 25, 245-260. https://doi.org/10.1016/s0169-5150(01)00082-2
Bao, W., Gong, A., Zhang, T., Zhao, Y., Li, B. and Chen, S. (2023) Mapping Population Distribution with High Spatiotemporal Resolution in Beijing Using Baidu Heat Map Data. Remote Sens-ing, 15, Article No. 458. https://doi.org/10.3390/rs15020458
Ouda, E., Sleptchenko, A. and Simsekler, M.C.E. (2023) Comprehensive Review and Future Research Agenda on Discrete-Event Simulation and Agent-Based Simulation of Emergency Departments. Simulation Modelling Practice and Theory, 129, Article ID: 102823. https://doi.org/10.1016/j.simpat.2023.102823
Fagan, A.A., Bumbarger, B.K., Barth, R.P., Bradshaw, C.P., Cooper, B.R., Supplee, L.H., et al. (2019) Scaling up Evidence-Based Interventions in US Public Systems to Prevent Behavioral Health Problems: Challenges and Opportunities. Prevention Science, 20, 1147-1168. https://doi.org/10.1007/s11121-019-01048-8
Thomson, K., Hillier-Brown, F., Todd, A., McNamara, C., Huijts, T. and Bambra, C. (2018) The Effects of Public Health Policies on Health Inequalities in High-Income Countries: An Umbrella Re-view. BMC Public Health, 18, Article No. 869. https://doi.org/10.1186/s12889-018-5677-1
Sangavi, C., Kollarmalil, R. and Abraham, S. (2025) Post-Mastectomy Wound Care—Need for an Empathetic Approach. Psychology, Health & Medi-cine. https://doi.org/10.1080/13548506.2025.2490229
Okonkwo, O. (2019) Program Guide for Division 40 (Socie-ty Forclinical Neuropsychology) at the Annual Convention of the American Psychological Association, August 8-11, 2019; Chicago, IL. The Clinical Neuropsychologist, 33, 1216-1348. https://doi.org/10.1080/13854046.2019.1628306
Rowe, J.B., Datta, D., Fiebach, C.J., Jaeggi, S.M., Liston, C., Luna, B., et al. (2024) Translating Prefrontal Cortex Insights to the Clinic and Society. In: Banich, M.T., et al., Eds., The Frontal Cortex, The MIT Press, 319-360. https://doi.org/10.7551/mitpress/15679.003.0019
Kurete, F. (2020) Enhancing Student Resilience through Access to Psychological Counselling Services in Selected Zimbabwean Polytechnics. PhD Diss., University of Pretoria (South Afri-ca).
Crowe, M., Eggleston, K., Douglas, K. and Porter, R.J. (2020) Effects of Psychotherapy on Comorbid Bipolar Dis-order and Substance Use Disorder: A Systematic Review. Bipolar Disorders, 23, 141-151. https://doi.org/10.1111/bdi.12971
Kalusivalingam, A.K., Sharma, A., Patel, N. and Singh, V. (2020) Optimizing In-dustrial Systems through Deep Q-Networks and Proximal Policy Optimization in Reinforcement Learning. International Journal of AI and ML, 1, 1-25.
Memarzadeh, M. and Pozzi, M. (2019) Model-Free Reinforcement Learning with Model-Based Safe Exploration: Optimizing Adaptive Recovery Process of Infrastructure Systems. Structural Safety, 80, 46-55. https://doi.org/10.1016/j.strusafe.2019.04.003
Mark, M., Chehrazi, N., Liu, H. and Weber, T.A. (2022) Optimal Recovery of Unsecured Debt via Interpretable Reinforcement Learning. Machine Learning with Applica-tions, 8, Article ID: 100280. https://doi.org/10.1016/j.mlwa.2022.100280
Zon, M., Ganesh, G., Deen, M.J. and Fang, Q. (2023) Context-Aware Medical Systems within Healthcare Environments: A System-atic Scoping Review to Identify Subdomains and Significant Medical Contexts. International Journal of Environmental Re-search and Public Health, 20, Article No. 6399. https://doi.org/10.3390/ijerph20146399
Lynch, C.J., Diallo, S.Y., Kavak, H. and Padilla, J.J. (2020) A Content Analysis-Based Approach to Explore Simulation Verification and Identify Its Current Challenges. PLOS ONE, 15, e0232929. https://doi.org/10.1371/journal.pone.0232929
Heath, B., Hill, R. and Ciarallo, F. (2009) A Survey of Agent-Based Modeling Practices (January 1998 to July 2008). Journal of Artificial Societies and Social Simulation, 12, 9.
Grant, R.W., Adams, A.S., Bayliss, E.A. and Heisler, M. (2013) Establishing Visit Priorities for Complex Patients: A Summary of the Literature and Conceptual Model to Guide Innovative Interventions. Healthcare, 1, 117-122. https://doi.org/10.1016/j.hjdsi.2013.07.008
Kazdin, A.E. (2017) Addressing the Treatment Gap: A Key Challenge for Extending Evidence-Based Psychosocial Interventions. Behaviour Research and Therapy, 88, 7-18. https://doi.org/10.1016/j.brat.2016.06.004
Ogundeko-Olugbami, O., Ogundeko, O., Lawan, M. and Foster, E. (2025) Harnessing Data for Impact: Transforming Public Health Interventions through Evidence-Based Deci-sion-Making.
Todd, J. and Stern, S. (2023) Towards More Nuanced Patient Management: Decomposing Readmission Risk with Survival Models. The Operation-al Research Society’s Annual Conference, Bath, 12-14 September 2023, 157-158.