WP1 – Definition of User Requirements and Architecture
Objectives:
(1) User requirements analysis;
In order to determine the user requirements for the development of a system to assist people with fragility, a questionnaire was developed. The research results were published in a scientific article published at the ISI conference, CSCS2021, Romania, 2021. The objective of the questionnaire was to highlight the important requirements for potential users. The questionnaire was distributed to people working in / or assisted persons with fragility.
The research involved 58 participants (66.5% women and 34.5% men) between 30 and 80 years old. The age range is wide, because we wanted to extract information about the perception of potential users, as well as those who need assistance. 74.1% of respondents were in a position to have someone in care, and 25.9% were cared for by someone.
The questionnaire also included a series of questions about some indicators related to fragility, such as body weight, loss of balance, level of physical activity, feeling exhausted and the presence of dizziness.
There is a visible interest among respondents about some features that should be provided by the proposed system. These are listed below, taking into account the order of priority for the development of the system:
- Non-intrusive
- Confidentiality
- Ease of use
- Security
- Performance
- Reliability
(2) System architectural design specification.
The solution for assisted living proposed in this project, improves the living conditions of the elderly by intelligent automation of the environment (home) and monitoring their vital parameters. Assistance for autonomy at home thus ensures an independent living for the elderly and / or suffering from chronic or mental illness. The project will provide a monitoring, automation and event notification / alerting platform based on heterogeneous IoT devices, a data access services platform, offline analysis based on artificial intelligence and notifications, software utilities for system configuration / adaptation to the needs of the beneficiary , scaling services for designer and integrator.
WP2 – Initial cINnaMON prototype
Objectives:
(1) Initial smart bulb prototype specification report;
The research was based on an efficient positioning system for indoor lighting, based on electronic sensors. The system is ubiquitous and inexpensive and ensures constant monitoring at home. Lighting fixtures use current technologies and can be used to monitor the actions of monitored persons. The main functionality of the platform associated with lighting fixtures is to monitor people at home where they carry out their daily activities and to constantly find individuals and place them on the map of the house.
(2) Initial smart bracelet prototype specification report;
To detect the activities and energy consumed by the people who will wear the bracelets, we have integrated in the project architecture one of the Fitbit bracelets, so that we can process the data provided by the sensors in the bracelet. To measure the relevant indicators to determine the level of emotional fragility, the data taken from the bracelet sensors were: accelerometer, gyroscope, orientation sensor and heart rate sensor.
(3) Report on the intelligent location module in a building;
The transmission and reception of RSSI values is done through the Arduino Mini Pro microcontroller that contains the Bluetooth module. For testing purposes, smart bulbs were considered to be placed in three different rooms to place the monitored person inside his or her home or building. The RSSI value provides information about the distance between the user’s smartphone and the smart light bulb.
Each MAC address of each smart light bulb is known, as well as the dimensions of the rooms in the house or building where the smart light bulbs are located. Smart bulb positions are indicated on the map using red circles. Their coordinates are known based on their position on the map.
The position of the monitored person’s mobile phone is characterized by the coordinates (x, y, z) which are taken over after the application of the trilateral algorithm. This is done in relation to the three nearby light bulbs belonging to the house or building of the monitored person.
(4) Report on the recognition of human activity;
In order to determine the level of physical activity of a person, the research activity focused on determining a mathematical formula, which with the help of data taken from the accelerometer sensor of the bracelet, to determine an approximate level. In this regard, an algorithm for recording the sampled data was implemented, which we later used to determine certain activities.
(5) Report on the user’s personal profile module;
Users of the cINnAMON system can create an account and log in to access the services offered by the digital platform. The administrator is responsible for granting access to users and must give a user credentials when requested. Setting up a project is a very important part of our system, as it allows the patient’s collected data to be available in the application for his doctor and the patient himself.
(6) Telemonitoring system report;
One project is the digital representation of a patient’s configuration. The designer must go to the patient’s location, install the sensors, the gateway and then insert them into the application. Each patient account will have an associated location.
(7) cINnAMON web application report;
Doctors can view the data collected for patients in tabular or chart form. In the Monitoring and Alert Scenarios module, patients can set thresholds for their monitored conditions. Notifications are sent to practitioners and the patient. The system sends an alert to a patient and their doctor when a value for a monitored condition is out of range.
(8) Data privacy report;
Establishing and preventing security requirements is extremely important for every system that processes confidential personal data. The system must ensure the confidentiality of data on patients, healthcare professionals, end-users and visitors. To ensure this, you must comply with European Union regulations on the protection of personal data: Law no. 190/2018 transposes into Romanian legislation the regulations of the General Regulation on Data Protection (GDPR). It introduces new provisions on liability, expressing consent to the processing of personal data, extensive regulations on data security breaches and the right to erasure. This law repeals the existing provisions at EU level by Directive 95/46/EC.
(9) Initial prototype test report.
The test system has been designed so that the functionalities can be tested independently of the others with which it communicates. The second level of testing is functional testing of the entire system. The functionalities are interconnected and complex scenarios that provide real use cases are verified. Although locating faults is difficult, system-wide functional tests can detect errors that are impossible to detect by unit functional testing, such as connection errors, communication errors, security breaches. The last component of the functional test is the testing of the client component. This ensures that the interaction between the user and the application takes place as specified.
WP3 – cINnAMON Alpha prototype
Objectives:
(1) Intelligent bulb prototype – alpha variant;
The report proposed a system efficient positioning of lighting for indoor, based on sensors electronics. Three Philips Hue bulbs and other three Zipato 2 bulbs were used to be compared with the bulb intelligent accomplished in the project. The left side of Figure 7 shows a device that was created as part of the research project and includes sensors for measuring temperature, humidity and volatile compound levels. In the center it is a light bulb intelligent available in the trade, Philips Hue, while on the right side is a different smart commercial light bulb, Zipato Bulb 2 .
Figure 7 – Intelligent lighting device developed as part of the project (left), Philips Hue bulb (center) and Zipato 2 bulb (right)
Testing has been done in a house where the external walls were 35 cm thick and they were built of brick together with cement (Figure 8) . The interior walls were 17 cm thick and they were made of brick. Because the corners of the rooms were made of armed concrete, the absorption of the signal occurred. During the time when the motion of the studied person was monitored, interference appeared and there were found consequences in the results obtained by calculating the person’s position.
Figure 8 – The plan of the apartment for the tests that included lighting devices, namely our bulb, Philips Hue bulbs and Zipato 2 for the technical validation of the system. The arrows show the movements of the user (a) from bed to wardrobe (blue); (b) from closet to bed (green); (c) from bed to desk (magenta)
(2) Initial smart bracelet prototype report – alpha version;
For the prototype, some relevant indicators for determining the level of fragility by analyzing data from sensors were determined, such as: accelerometer, gyroscope, orientation sensor and heart rate sensor (Figure 9).
Figure 9 – General architecture of the prototype
At the level of the smart bracelet, an application is installed that allows the collection of raw data directly from the built-in sensors. Accelerometer data is extracted to measure the acceleration of the device along three orthogonal axes: X, Y, and Z. The X axis is parallel to the device screen, aligned with the top and bottom edges in the left-right direction. The Y axis is parallel to the device screen, aligned with the left and right edges in the up-down direction, and the Z axis is perpendicular to the device’s screen. Accelerometer data is collected with gravitational acceleration included, which required some extra server-side processing to remove it. When the device is placed on a table, the acceleration along the Z-axis indicates the value of the gravitational acceleration (approx. 9.8 m/ ), and the acceleration along the other two axes is 0. As mentioned above, additional data comes from sensors, such as the gyroscope, the orientation sensor (very useful in finding a patient’s position, or identifying the type of activity they are doing), and the heart rate sensor. All this data is collected and transmitted to the cINnAMON system via web sockets at a frequency of 100 Hz.
(3) Report on the intelligent location module in a building – alpha version;
The experiments were conducted in an apartment, using trilateration in n directions, with Least Squares optimization. The input data of the algorithm is represented by the positions of known intelligent light bulbs and the distances between these for the aim to monitor the person. These distances are calculated based on of registered RSSI values, and a loss exponent that depends on the distance traveled and the measured signal power.
The result of the previous mentioned method is the centroid position of the monitored person. There are necessary at least three smart bulbs, along with their locations to enable the calculation of the targeted person positions. The Levenberg-Marquardt algorithm is used to solve the non- linear Least Squares problem.
(4) Report on the recognition of human activity – alpha variant;
To classify the activities using the data acquired by the sensors of a bracelet, the accelerations x, y and z obtained from the accelerometer, the angular velocities X, Y and Z obtained from the gyroscope, q0, q1, q2 , q3 are used, where q0 is the scalar part of quaternion and q1, q2, q3 are the quaternion factors, from the orientation sensor and heart rate variability obtained from the pulse sensor (Figure 10).
The accelerometer mounted on the smartwatch is used to measure the instantaneous accelerations of a moving element along 3 orthogonal axes: X, Y and Z. The default accelerometer bandwidth is 100 Hz, which means it can be read 100 times per second.
Figure 10 – General data collection architecture
(5) Report on the user’s personal profile module – alpha version;
The cINnAMON system users can register to have access to offered services. The administrator grants user access depending on their role (Figure 11). Projects can be added in the system to allow the collection of data coming from patients, which can later be analyzed by doctors who are associated to them.
Figure 11 – User registration by the administrator
(6) Telemonitoring system report – alpha variant;
Every new project created on the cINnAMON platform is a configuration representation of sensors in one patient’s house based on associated locations (Figure 12). The designer travels to the location of the patient, installs the sensors, the gateway and then inserts them into the application, such that the data from them to can be taken over.
Figure 12 – Viewing the monitored patient within a project
(7) cINnAMON web application report – alpha version;
Doctors can view patient data collections in tabular form or as diagrams (Figure 13). In the module Monitorization scenarios and alerts, patients can set thresholds for their surveilled conditions. Notifications are sent to practitioners and the patient. The system sends an alert to the patient and the doctor when a value for a monitored condition is outside the normal range.
Figure 13 – Graphs regarding the average value of temperature, CO2, air quality, humidity, smoke and ambient light
(8) Prototype test report – alpha variant;
In order to verify the realization of the functionalities from the initial design phase of the cINnAMON prototype, a functional test plan was created. Functional testing is a type of software testing that validates the software system against functional requirements/specifications. The purpose of functional testing is to test each function of the software application by providing appropriate input, checking the output against the functional requirements.
The main objective of testing is to ensure that the system complies with all specified requirements, including non-functional ones, and that all operating scenarios are satisfied. The secondary objective of testing is to identify and report all defects and risks, communicate them to the development team and ensure that all these issues are dealt with in an appropriate manner before the implementation is completed.
Achieving quality objectives requires careful and methodical testing of the system, clear and complete reporting of all detected problems, and appropriate handling of these problems.
(9) Stage 3 dissemination report.
They were presented and published 3 articles at international conferences.
The Consortium aims to keep the information on the website up-to-date, as well as to maintain the website during and after the completion of the project implementation.
The website of the cINnAMON project has the following structure:
- Home Page – includes a brief presentation of the project, including public information related to the budget and the funding received;
- Consortium – brief presentation of project partners;
- Approach – information related to the main objective of the project, and the way in which it is to be fulfilled;
- Activities – activities foreseen within the project, organized in work packages;
- Outputs – information related to the expected results, as well as the achieved public results of the project;
- Events – current events within the project, especially those that benefit from a public component;
- Publication – list of scientific publications developed within the project;
- Contact – project consortium’s contact form.
WP4 – CINnAMON Final Prototype
Objectives:
(1) Smart bulb prototype report – final version;
This report described an efficient positioning system for indoor lighting fixtures based on electronic sensors. This system uses smart bulbs built around a Raspberry Pi 3 Model B, which manage the light intensity of the LEDs and collect environmental data. Smart bulbs are equipped with sensors to measure temperature, humidity, concentration of CO2 and volatile organic compounds, ambient brightness and the level of dust in the environment (Figure 14). They also include motion and location sensors. These smart bulbs can perform Bluetooth Low Energy (BLE) scans and connect to central servers using Wi-Fi and Ethernet connections.
Smart bulbs use the MQTT protocol to communicate with the central server. They are connected to a printed circuit board (PCI), which includes a transformer for direct mains power. The PCI is divided into two parts, one for power supply and the other for environmental monitoring. This flexible configuration allows the placement of components according to the bulb type (Figure 15).
There are three main types of sensors in smart lighting:
- Environmental/ambient sensors for temperature, humidity, ambient light, dust, volatile organic compounds and CO2.
- Presence sensors for motion detection.
- Location sensors for determining the position of people inside the room.
All sensors have different interfaces and require different voltages, and their data is collected and transmitted to a central server for analysis and monitoring. The system uses a MongoDB database to store information about homes, smart bulbs, users and environmental data. The communication between the smart bulbs and the server is done through the MQTT protocol, and the data is transmitted in JSON format.
(2) Smart bracelet prototype report – final version;
A Fitbit smart bracelet was used within the cINnAMON system to monitor and detect early signs of frailty in the elderly (Figure 16). This development comes in the context of the growth of the elderly population and the need to improve their quality of life. This is a system that uses sensors and technology to collect and process relevant data about users’ health, with the aim of identifying early signs of frailty.
The main aspects of the prototype include:
– Data collection: Sensors such as accelerometer, gyroscope, orientation sensor and heart rate sensor are used to monitor various health related parameters and user activities.
– Communication and security: Data is collected from the smart bracelet and transferred to a secure cloud server via a companion app. Communication is secured by SSL encryption and OAuth 2.0 for authorization.
– Storage and processing: Collected data is stored in a MongoDB database and processed to identify early signs of fragility.
– User interface: Users can access their data and track their health status through a web interface.
– Testing and performance: The prototype was successfully tested under laboratory conditions to evaluate its performance in terms of data collection, server resource utilization and reliability.
– Comparison with other systems: The prototype was compared with other similar fragility monitoring systems and found to provide a predictive and reliable solution for detecting early signs of fragility.
The test results showed that the prototype is able to work reliably and provide useful data for monitoring the health of users. This system has the potential to improve the quality of life of older people by early detection of frailty risks and appropriate intervention to prevent negative consequences.
(3) Report on the intelligent location module in a building – final version;
The intelligent way of locating in a building was achieved by using Bluetooth devices and data processing techniques. The experimental methodology was similar in two different locations. Devices have been installed and connected to a server. One person remained stationary in various places in the rooms to collect data. A Bluetooth enabled mobile phone was placed on the monitored person and the RSSI signal values were recorded and sent to the server for further analysis.
The evaluation was based on the Euclidean distance measure and used Kalman filtering and a heuristic called “look-back-k” to improve the localization accuracy. The results showed that Kalman filtering and look-back-k heuristics reduced the localization errors on average.
An artificial neural network was also tested for localization, and the results showed that with an increased number of hidden layers and training epochs, the localization accuracy increased significantly (Figure 17). However, training the network for multiple locations would require more time.
Compared with other indoor localization methods, such as trilateration and using WiFi signal, the proposed method achieved competitive results with lower average localization errors. This intelligent way of locating within a building has the potential to significantly improve localization accuracy in indoor environments.
(4) Report on the recognition of human activity – final version;
The human activity recognition mode collected data for four types of activities: fast walking, slow walking, resting, and stair climbing. The data was recorded by sensors in a Fitbit Versa device and processed in a MongoDB server. Then, a machine learning module was developed to recognize the activity types (Figure 18).
The results showed that machine learning models, especially Gradient Boosting and Random Forest, achieved good performance in activity recognition. Lighter datasets have shown that sparse data such as acceleration vector magnitude and orientation data can be effectively used for activity recognition.
However, limitations were also identified, such as the variability of data collected between different individuals and the need to measure and analyze multiple physiological parameters. Ethical considerations related to the collection of personal data and their security were also emphasized.
The human identification algorithm is based on temperature and motion (Figure 19), with special attention to the position and orientation of a person standing or lying down. Falling is detected when the human position changes from standing to lying down and remains still for a defined period of time.
Finally, the developed module could be used to detect the daily activities of the elderly, having the potential to identify early signs of frailty. However, further research is needed to overcome the limitations and develop a more robust and ethical system.
(5) Report on the user’s personal profile module – final version;
The administrator can register new users, including doctors, patients, designers and other system administrators, using an intuitive interface. Users log in by entering the email address and password provided by the administrator, thus gaining access to the application. The administrator can manage user access and provide their credentials to ensure they have the correct permissions and can access the required functionality. This administrator role is crucial to maintaining system security and integrity by ensuring that only authorized individuals can access sensitive application resources and data.
(6) Telemonitoring system report – final version;
Designers can create and configure projects, including adding patients, locations, and sensors relevant to each project. This process is essential to facilitate the collection and access of patient data through the application. Practitioners, in turn, have the ability to monitor patients and access detailed reports on their health status. The doctor can add observations and instructions for patients based on the report data (Figure 20). The system also generates notifications for practitioners in the event of alerts related to monitored patients. Patients can access recorded medical observations and receive clear guidance on the actions needed to maintain their health.
(7) cINnAMON web application report – final version;
cINnAMON Web Application Module – Final Version is a comprehensive platform developed on an Apache 2.4 server and MySQL database, using multiple instances of NodeJS for real-time information processing. This module integrates several essential components, including user registration and authentication, daily status and weight reporting, automatic data visualization such as heart rate and sleep history, and real-time motion and energy consumption monitoring functionality based on on machine learning models. It also includes modules for user location monitoring, fall detection and user frailty assessment.
(8) Prototype test and validation report – final version;
During the testing and validation of the cINnAMON prototype – Final Version, a meticulous functional testing plan was implemented to evaluate the software functionalities against the initial requirements and specifications. Testing included verifying the connection, validating the security certificate, ensuring the web app is displayed correctly, testing the FitBit server connection, installing and using the app on the wristband, checking multiple connection and platform stability, and extensive data monitoring and functionality testing user interface.
A pilot study was also conducted to validate the Fitbit wristband and the cINnAMON system in a group of participants aged 65 years or older with no history of neurodegenerative diseases. Participants completed questionnaires to assess frailty and satisfaction with bracelet use. Preliminary results indicated a positive perception and acceptable performance of the system.
Artificial intelligence models have also been developed and tested to detect the type of physical activity, achieving results with an accuracy of over 90% (Figure 22). These models were used to determine energy expenditure and assess sedentary time. The system’s front-end interface allowed real-time data to be displayed and was updated every 5 seconds to provide users with current information.
In conclusion, the prototype cINnAMON – Final Version has successfully gone through the testing and validation process, resulting in a reliable and functional system, ready to be used for its intended purposes, especially in monitoring the health status of the elderly and in promoting a healthy lifestyle.
(9) Dissemination report stage 4.
Papers published in conferences and journals have covered various aspects related to the project, including security of IoT devices, depression detection and support for people with frailty. These articles have been published in conferences and specialized journals, thus contributing to the exchange of knowledge in the field of medical technology.
The project website, accessible in English and Romanian, provided a platform for the presentation of the project, the project partners and the results obtained. It included information about the project’s objectives, outputs and activities, as well as contact details for the project consortium.
Other forms of dissemination included collaboration with the media and participation in dissemination events (Figure 23). This effort was aimed at attracting the attention of the general public and promoting the solutions developed within the project.
Pentru viitor, se planifică continuarea diseminării prin participarea la conferințe și evenimente, publicarea de articole și studii, organizarea de webinar-uri și prezentări online, utilizarea platformelor de socializare, crearea de materiale video și demo-uri, și participarea la târguri și expoziții pentru atragerea colaboratorilor și partenerilor din industrie. Toate aceste acțiuni vor contribui la creșterea vizibilității proiectului și la promovarea soluțiilor dezvoltate în cadrul acestuia.