Research update: AI-based recognition of mental health parameters in the workplace

About six months ago, we announced an exciting collaboration with the Fraunhofer Institute for Computer Graphics Research (IGD). Funded by the German Federal Ministry for Economic Affairs and Climate Protection, this research and development project aims to promote technological approaches to preventing stress in the workplace. The main objective is to research and develop methods to recognise the first signs of mental stress in the workplace at an early stage and without collecting personal data. On this basis, an individualised early warning system is to be developed that enables office workers to find out more about their own mental well-being and learn better coping strategies.

Jump directly to the interim results

Self-efficacy as a means of maintaining health

The idea behind Isa has always been to use new technologies to give people low-threshold and sustainable access to healthy behaviours. Although our founders worked in companies that offered numerous prevention programmes, they were not being reached by the usual health programmes. Despite being aware of the problem and endeavouring to integrate sufficient ergonomics and exercise into their office work, they ended up becoming patients due to musculoskeletal complaints. This sobering experience led to the question: "How can I maintain my health despite intensive VDU work?" The answer was clear: "I need someone who keeps an eye on my health and tells me what my body needs when necessary." This vision has not changed. We believe that everyone lives healthiest when their behaviour follows health science recommendations. With the development of Isa, we have successfully realised this and turned numerous people into health experts themselves through personal AI assistance so that they can take responsibility for their own health despite their professional priorities.

Isa - New sensors open up new possibilities

Review of existing technology: The current version of Isa works with a time-of-flight (ToF) sensor, an infrared sensor that emits light points in the invisible range and measures their reflection time. The greater the distance to an object, the longer the reflection time. This creates a depth image of the environment that allows conclusions to be drawn about movement patterns and posture without recording personal data. This enables us to determine precisely how ergonomic a person's behaviour is and what strain is being placed on the body. We can also recognise whether a workplace is ergonomically designed, which means that Isa is already being used in occupational health and safety as part of the risk assessment.

New areas of application for more comprehensive support: With a significant update of new sensors and the optimisation of our AI models, we want to improve MSK prevention by increasing the functionality in the area of occupational health and safety. We also want to transfer our current approach to other areas of health, as 9 out of 10 office workers now confirm the positive effect of Isa on their own health behaviour at work. In view of the rising number of cases of mental stress at work, we have therefore set ourselves the goal of using Isa to enable employees to learn more about their mental well-being at work in order to help them deal better with mental stress.

Challenges and approaches to recognising stress

Assessing mental stress is much more complex than identifying physical stress. There are many causes that cannot be measured directly, such as work stress, social conflicts or personal problems. The common approach is therefore to measure physical symptoms of mental stress, such as an elevated pulse or blood pressure. However, this data is difficult to interpret, as different causes can manifest themselves in similar symptoms. For example, heart rate variability can be used to recognise whether a person is "stressed", but it is not known where the stress is coming from and whether it is good (eustress) or bad (distress) [What is Eu- & Distress?]. For example, a highly focussed person could show the same physical symptoms of stress as a person who is working overtime to submit a report before the deadline. The former would experience the stress as enjoyable or "flow", while the other person would release high levels of stress hormones such as cortisol, putting them at high risk of burnout in the long term. Many wearable technologies such as smartwatches and fitness trackers promise a lot, but are very imprecise in identifying and differentiating mental stress. Their hit rate is usually around 60%, which is hardly better than simply guessing.

Approaches to the assessment of mental stress:

• Neurophysiological methods: These include, for example, the electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI), which essentially measure the electrical activity of the brain.
• Physiological measurements: e.g. heart rate variability (HRV), electrocardiogram (ECG), skin conductance (EDA/GSR), blood pressure and respiratory rate.
• Behavioural analysis: Measurement of movement patterns and activity levels that correlate with stress. A popular example of this is the approach of predicting stress at a computer workstation based on mouse movements.
• AI and machine learning: Analysis of large amounts of physiological and behavioural data, e.g. speech analysis to identify the mental state.

How Isa should recognise mental stress in the work context

Our confidence that Isa can do what established brands have not yet achieved is based on the integration of promising approaches to recognising psychological stress and stress detection - all but neurophysiological methods are taken into account. In addition to existing sensors and sensors for air, noise and light, the new Isa version will also be expanded to include a radar sensor that can measure vital data such as heart rate, heart rate variability and respiration without contact. By combining this physiological data with behavioural analyses (e.g. movement, ergonomics, blinking) and AI algorithms, we expect to be able to achieve a significantly higher level of predictive accuracy than conventional smartwatches. As this is sensitive data, we would like to emphasise that Isa will continue to work completely offline, ensuring the protection of personal data.

Interim results after the first few months of intensive research to recognise mental stress in the workplace at an early stage

We are very pleased with the results of the joint research project with Fraunhofer IGD so far. After just a few months, we have developed a sound concept that looks very promising. After reviewing numerous studies, we were able to narrow down the methodology of stress detection to certain methods that offer the greatest potential. We have started to simulate stressful situations and record data with Isa. With the help of our AI models, we are now trying to recognise patterns that emerge in different stress situations. In this way, we not only want to recognise psychological stress symptoms, but also be able to differentiate between different causes. Initial studies already indicate that increased levels of work stress (perceived stress) correlate with lower levels of physical activity (e.g. sitting interruptions, standing desk use, movement exercises) and poorer posture (body tension).

Possible connection between work stress, posture and movement behaviour

As mentioned, the data only show a snapshot of initial data analyses. This is why we do not want to speak of significant and reliable data here. Nevertheless, the initial data analyses show some interesting patterns, which we will briefly interpret below.

1. Relationship between stress levels, posture and movement: Our data indicate a relationship between stress level, posture and movement level. At no stress, ergonomic postures dominate (64%) and the level of movement is very high (99.64%). As stress increases, the level of movement decreases dramatically and the stooped posture increases, indicating that less movement at work correlates with poorer posture and higher stress.
2. Discrepancy between extreme values: There are significant differences between the extreme stress levels. At no stress, both the ergonomic posture (64%) and the level of movement (99.64%) are very high. In contrast, at high levels of stress, the majority of people adopt a stooped posture (81.54%) and the level of movement is very low (11.35%). This indicates that high levels of stress lead to both physical inactivity and poorer posture.
3. Fluctuations at medium stress levels: At low (83.47% ergonomic posture, 38.11% level of movement) and moderate stress (88.86% ergonomic posture, 13.43% level of movement), the majority of individuals remain in an ergonomic posture, while the level of movement decreases. This could indicate that, despite decreasing levels of movement in moderate stress, good posture is maintained until a certain stress threshold is reached.
4. Possible misclassification at low stress levels: A surprisingly high percentage of bent postures in non-stressful situations (higher than in low and medium stress) could indicate a possible misclassification or individual differences in stress perception and impact. This could mean that the subjective stress level does not always correlate linearly with posture.

Why early recognition of mental stress in the workplace is so important

Recognising mental stress in the workplace at an early stage is crucial in order to avoid long-term negative effects on the health and productivity of employees. Mental stress can permanently lead to serious mental illnesses such as burnout, depression and anxiety. But even if these only occur occasionally, they have a significant negative impact on the individual and the organisation as a whole. Through early intervention and support, companies can promote the well-being of their employees and create a healthy working environment.

How Isa is designed to reduce workplace stress and promote mental health

By combining various sensors and analytical methods, Isa will in future provide a detailed picture of mental and physical stress in the workplace and provide support on three levels:

1. Early warning system: Employees are made aware of increased stress or stressful situations, sensitised and encouraged to self-reflect.
2. Targeted advice: Isa gives instructions on how to cope with stress, e.g. meditation or breathing exercises, and offers suggestions for external help.
3. Corporate reporting: Anonymous analyses can provide company health management and HR with information about mental well-being in the company or in different departments.

Isa's advanced sensor technology and AI analyses offer an innovative way of keeping an eye on and promoting physical and, in future, mental health in the workplace in compliance with data protection regulations. By recognising mental stress at work at an early stage and making individual recommendations, employees can learn to deal with stress better and improve their health in the long term. Companies benefit from healthier, happier and more productive employees, resulting in a win-win situation for everyone involved.

Further information on the research project:

• Healthier working at the screen - Fraunhofer IGD
• AI-supported stress intervention in the workplace - Deep Care (deep-care.de)