The rapid advances in machine learning (ML) and information fusion have made it possible to use machines/computers with the ability of understanding, recognition, and analysis of human emotion, mood and stress, and related mental diseases. The recognition methods based on physiological modalities are the most performant. Wearable technologies enable non-invasive long-term data gathering and analysis. The number of mental health issues are correlated with emotional states and can be possibly detected by similar methods to general emotion recognition. The scientific interest in the recognition of mental disorders is growing, and most of the available studies are uni-modal based on either ECG or EEG sensor data, while some recent studies also utilize multiple modalities and sensor fusion.
[1] E. M. Sledjeski, B. Speisman, and L. C. Dierker, "Does a number of lifetime traumas explain the relationship between PTSD and chronic medical conditions? Answers from the National Comorbidity Survey-Replication (NCSR)", Journal of Behavioral Medicine, vol. 31, no. 4. Springer Science and Business Media LLC, pp. 341-349, Jun. 14, 2008. DOI: 10.1007/s10865-008-9158-3.
https://doi.org/10.1007/s10865-008-9158-3
[2] R. H. Belmaker and G. Agam, "Major Depressive Disorder", New England Journal of Medicine, vol. 358, no. 1. Massachusetts Medical Society, pp. 55-68, Jan. 03, 2008. doi: 10.1056/nejmra073096.
https://doi.org/10.1056/NEJMra073096
[3] L. Sher, "Recognizing post-traumatic stress disorder", QJM: An International Journal of Medicine, vol. 97, no. 1. Oxford University Press (OUP), pp. 1-5, Jan. 2004. DOI: 10.1093/qjmed/hch003.
https://doi.org/10.1093/qjmed/hch003
[4] J. D. Elhai, A. A. Contractor, P. A. Palmieri, D. Forbes, and J. D. Richardson, "Exploring the relationship between underlying dimensions of posttraumatic stress disorder and depression in a national, trauma-exposed military sample", Journal of Affective Disorders, vol. 133, no. 3. Elsevier BV, pp. 477-480, Oct. 2011. doi: 10.1016/j.jad.2011.04.035.
https://doi.org/10.1016/j.jad.2011.04.035
[5] J. R. T. Davidson, D. Hughes, D. G. Blazer, and L. K. George, "Post-traumatic stress disorder in the community: an epidemiological study", Psychological Medicine, vol. 21, no. 3. Cambridge University Press (CUP), pp. 713- 721, Aug. 1991. DOI: 10.1017/s0033291700022352.
https://doi.org/10.1017/S0033291700022352
[6] J. R. T. Davidson, "Recognition and Treatment of Posttraumatic Stress Disorder", JAMA, vol. 286, no. 5. American Medical Association (AMA), p. 584, Aug. 01, 2001. doi: 10.1001/jama.286.5.584.
https://doi.org/10.1001/jama.286.5.584
[7] L. Amaya-Jackson et al., "Functional impairment and utilization of services associated with posttraumatic stress in the community", Journal of Traumatic Stress, vol. 12, no. 4. Wiley, pp. 709-724, Oct. 1999. DOI: 10.1023/a:1024781504756.
https://doi.org/10.1023/A:1024781504756
[8] G. E. Switzer, M. A. Dew, K. Thompson, J. M. Goycoolea, T. Derricott, and S. D. Mullins, "Posttraumatic stress disorder and service utilization among urban mental health center clients", Journal of Traumatic Stress, vol. 12, no. 1. Wiley, pp. 25-39, Jan. 1999. DOI: 10.1023/a:1024738114428.
https://doi.org/10.1023/A:1024738114428
[9] R.W. Picard, Affective computing. Cambridge, MA: MIT Press., 1997.https://mitpress.mit.edu/9780262161701/
[10] J. Zhang, Z. Yin, P. Chen, and S. Nichele, "Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review", Information Fusion, vol. 59. Elsevier BV, pp. 103-126, Jul. 2020. DOIi: 10.1016/j.inffus.2020.01.011.
https://doi.org/10.1016/j.inffus.2020.01.011
[11] A. Minassian, M. A. Geyer, D. G. Baker, C. M. Nievergelt, D. T. O'Connor, and V. B. Risbrough, "Heart Rate Variability Characteristics in a Large Group of Active-Duty Marines and Relationship to Posttraumatic Stress", Psychosomatic Medicine, vol. 76, no. 4. Ovid Technologies (Wolters Kluwer Health), pp. 292-301, May 2014. doi: 10.1097/psy.0000000000000056.
https://doi.org/10.1097/PSY.0000000000000056
[12] V.A. Petrushin, "Emotion in speech: recognition and application to call centers", Proceeding of the 1999 Conference on Artificial Neural Networksin Engineering, 1999, pp. 7-10. https://www.researchgate.net/publication/2611186_Emotion_in_S peech_Recognition_and_Application_to_Call_Centers
[13] K. Anderson and P. W. McOwan, "A real-time automated system for the recognition of human facial expressions", IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics), vol. 36, no. 1. Institute of Electrical and Electronics Engineers (IEEE), pp. 96-105, Feb. 2006. DOI: 10.1109/tsmcb.2005.854502.
https://doi.org/10.1109/TSMCB.2005.854502
[14] M. Pantie and L. J. M. Rothkrantz, "Automatic analysis of facial expressions: the state of the art", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 12. Institute of Electrical and Electronics Engineers (IEEE), pp. 1424-1445, 2000. DOI: 10.1109/34.895976.
https://doi.org/10.1109/34.895976
[15] Z. Yin, M. Zhao, Y. Wang, J. Yang, and J. Zhang, "Recognition of emotions using multimodal physiological signals and an ensemble deep learning model", Computer Methods and Programs in Biomedicine, vol. 140. Elsevier BV, pp. 93- 110, Mar. 2017. doi: 10.1016/j.cmpb.2016.12.005.
https://doi.org/10.1016/j.cmpb.2016.12.005
[16] X.W. Wang, D. Nie, and B.L. Lu, "Emotional state classification from EEG data using machine learning approach", Neurocomputing, vol. 129. Elsevier BV, pp. 94- 106, Apr. 2014. doi: 10.1016/j.neucom.2013.06.046.
https://doi.org/10.1016/j.neucom.2013.06.046
[17] R. W. Picard, E. Vyzas, and J. Healey, "Toward machine emotional intelligence: analysis of affective physiological state", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 10. Institute of Electrical and Electronics Engineers (IEEE), pp. 1175-1191, 2001. DOI: 10.1109/34.954607.
https://doi.org/10.1109/34.954607
[18] R. W. Levenson, P. Ekman, K. Heider, and W. V. Friesen, "Emotion and autonomic nervous system activity in the Minangkabau of West Sumatra", Journal of Personality and Social Psychology, vol. 62, no. 6. American Psychological Association (APA), pp. 972-988, 1992. DOI: 10.1037/0022- 3514.62.6.972.
https://doi.org/10.1037/0022-3514.62.6.972
[19] A. Vujic, S. Tong, R. Picard, and P. Maes, "Going with our Guts", Proceedings of the 2020 International Conference on Multimodal Interaction. ACM, Oct. 21, 2020. DOI: 10.1145/3382507.3418882.
https://doi.org/10.1145/3382507.3418882
[20]J. Hernandez et al., "Stress measurement from tongue color imaging", 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, Oct. 2017. doi: 10.1109/acii.2017.8273593.
https://doi.org/10.1109/ACII.2017.8273593
[21] A. Ghandeharioun et al., "Objective assessment of depressive symptoms with machine learning and wearable sensors data", 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, Oct. 2017. doi: 10.1109/acii.2017.8273620.
https://doi.org/10.1109/ACII.2017.8273620
[22] E. A. D. Lee and S. A. Theus, "Lower Heart Rate Variability Associated With Military Sexual Trauma Rape and Posttraumatic Stress Disorder", Biological Research For Nursing, vol. 14, no. 4. SAGE Publications, pp. 412-418, Aug. 15, 2012. DOI: 10.1177/1099800412454453.
https://doi.org/10.1177/1099800412454453
[23] A. Agorastos et al., "Diminished vagal activity and blunted diurnal variation of heart rate dynamics in posttraumatic stress disorder", Stress, vol. 16, no. 3. Informa UK Limited, pp. 300-310, Jan. 03, 2013. DOI: 10.3109/10253890. 2012.751369.
https://doi.org/10.3109/10253890.2012.751369
[24] A. J. Shah, R. Lampert, J. Goldberg, E. Veledar, J. D. Bremner, and V. Vaccarino, "Posttraumatic Stress Disorder and Impaired Autonomic Modulation in Male Twins", Biological Psychiatry, vol. 73, no. 11. Elsevier BV, pp. 1103- 1110, Jun. 2013. DOIi: 10.1016/j.biopsych.2013.01.019.
https://doi.org/10.1016/j.biopsych.2013.01.019
[25] M. Schneider and A. Schwerdtfeger, "Autonomic dysfunction in posttraumatic stress disorder indexed by heart rate variability: a meta-analysis", Psychological Medicine, vol. 50, no. 12. Cambridge University Press (CUP), pp. 1937- 1948, Aug. 28, 2020. doi: 10.1017/s003329172000207x.
https://doi.org/10.1017/S003329172000207X
[26]J. Zhu et al., "Content-based multiple evidence fusion on EEG and eye movements for mild depression recognition", Computer Methods and Programs in Biomedicine, vol. 226. Elsevier BV, p. 107100, Nov. 2022. DOI: 10.1016/j.cmpb. 2022.107100.
https://doi.org/10.1016/j.cmpb.2022.107100
[27] Y. Liu, C. Pu, S. Xia, D. Deng, X. Wang, and M. Li, "Machine learning approaches for diagnosing depression using EEG: A review", Translational Neuroscience, vol. 13, no. 1. Walter de Gruyter GmbH, pp. 224-235, Jan. 01, 2022. doi: 10.1515/tnsci-2022-0234.
https://doi.org/10.1515/tnsci-2022-0234
[28] O. Faust, P. Ang, S.D. Puthankattil, and P. K. Joseph, "Depression diagnosis support system based on EEG signal entropies". Journal of Mechanics in Medicine and Biology, vol. 14, no. 03. World Scientific Pub Co Pte Lt, p. 1450035, Mar. 13, 2014. DOI: 10.1142/s0219519414500353.
https://doi.org/10.1142/S0219519414500353
[29] M. Čukić, V. López, and J. Pavón, "Classification of Depression Through Resting-State Electroencephalogram as a Novel Practice in Psychiatry: Review", Journal of Medical Internet Research, vol. 22, no. 11. JMIR Publications Inc., p. e19548, Nov. 03, 2020. DOI: 10.2196/19548.
https://doi.org/10.2196/19548
[30] M. Elgendi and C. Menon, "Assessing Anxiety Disorders Using Wearable Devices: Challenges and Future Directions", Brain Sciences, vol. 9, no. 3. MDPI AG, p. 50, Mar. 01, 2019. doi: 10.3390/brainsci9030050.
https://doi.org/10.3390/brainsci9030050
[31] E. H. Jang, K. W. Choi, A. Y. Kim, H. Y. Yu, H. J. Jeon, and S. Byun, "Automated detection of panic disorder based on multimodal physiological signals using machine learning", ETRIJournal. Wiley, Aug. 09, 2022. doi: 10.4218/etrij.2021- 0299.
https://doi.org/10.4218/etrij.2021-0299
[32] X. Zang, B. Li, L. Zhao, D. Yan, and L. Yang, "End-to-End Depression Recognition Based on a One-Dimensional Convolution Neural Network Model Using Two-Lead ECG Signal," Journal of Medical and Biological Engineering, vol. 42, no. 2. Springer Science and Business Media LLC, pp. 225-233, Feb. 07, 2022. DOI: 10.1007/s40846-022-00687-7.
https://doi.org/10.1007/s40846-022-00687-7
[33] Y. Long, Y. Lin, Z. Zhang, R. Jiang, and Z. Wang, "Objective Assessment of Depression Using Multiple Physiological Signals", 2021 14th International Congress on Image and Signal Processing, BioMedicalEngineering and Informatics (CISP-BMEI). IEEE, Oct. 23, 2021. doi: 10.1109/cispbmei53629.2021.9624394.
https://doi.org/10.1109/CISP-BMEI53629.2021.9624394
[34] R. A. Movahed, G. P. Jahromi, S. Shahyad, and G. H. Meftahi, "A major depressive disorder classification framework based on EEG signals using statistical, spectral, wavelet, functional connectivity, and nonlinear analysis", Journal of Neuroscience Methods, vol. 358. Elsevier BV, p. 109209, Jul. 2021. doi: 10.1016/j.jneumeth.2021.109209.
https://doi.org/10.1016/j.jneumeth.2021.109209
[35] C.T. Wu et al., "Resting-State EEG Signal for Major Depressive Disorder Detection: A Systematic Validation on a Large and Diverse Dataset", Biosensors, vol. 11, no. 12. MDPI AG, p. 499, Dec. 06, 2021. doi: 10.3390/bios11120499.
https://doi.org/10.3390/bios11120499
[36] C.-H. Cho, T. Lee, M.-G. Kim, H. P. In, L. Kim, and H.-J. Lee, "Mood Prediction of Patients With Mood Disorders by Machine Learning Using Passive Digital Phenotypes Based on the Circadian Rhythm: Prospective Observational Cohort Study", Journal of Medical Internet Research, vol. 21, no. 4. JMIR Publications Inc., p. e11029, Apr. 17, 2019. DOI: 10.2196/11029.
https://doi.org/10.2196/11029
[37] L. Duan et al., "Machine Learning Approaches for MDD Detection and Emotion Decoding Using EEG Signals", Frontiers in Human Neuroscience, vol. 14. Frontiers Media SA, Sep. 23, 2020. doi: 10.3389/fnhum.2020.00284.
https://doi.org/10.3389/fnhum.2020.00284
[38] Y. Tazawa et al., "Evaluating depression with multimodal wristband-type wearable device: screening and assessing patient severity utilizing machine-learning", Heliyon, vol. 6, no. 2. Elsevier BV, p. e03274, Feb. 2020. doi: 10.1016/j.heliyon.2020.e03274.
https://doi.org/10.1016/j.heliyon.2020.e03274
[39] G. Valenza et al., "Predicting Mood Changes in Bipolar Disorder Through Heartbeat Nonlinear Dynamics", IEEE Journal of Biomedical and Health Informatics, vol. 20, no. 4. Institute of Electrical and Electronics Engineers (IEEE), pp. 1034-1043, Jul. 2016. DOI: 10.1109/jbhi.2016.2554546.
https://doi.org/10.1109/JBHI.2016.2554546