How to use sensor data to create animation for physiotherapy application?

[MDabh]

I am using Blue Coin for developing a motion sensor for physiotherapy application.

I am not able to figure out how to use the sensor (accelerometer, gyroscope) data (logged in csv files) to develop an animation to show the performed exercise, based on it's velocity and angles achieved.

I greatly appreciate your assistance.

[Eleon BORLINI]

Hi @MDabh​ , the Blue Coin app already includes the activity recognition feature. However, I think you are interested in something more specific and versatile related to the physiotherapy activity recognition. A way to start could be starting from the existing libraries of the X-CUBE-MEMS1 package, available for STM32F4 (same MCU family as BlueCoin core), such as the MotionGR (gesture recognition) library, the MotionFA (fitness activity) library, or also the MotionAR (activity recognition) one. You can mix these activity recognition to try to get a specific gesture recognized.

Another way to implement your application, although without using the BlueCoin, is to use a LSM6DSO or LSM6DSOX embedded features, such as Finite State Machine feature (FSM) and/or Machine Learning Core (MLC). Especially for the MLC, you should acquire lots of different gestures (and lots of data for each gesture) and then generate a personalized decision tree where the leaves are the activities you want to recognize. Regards

[MDabh]

Hello Eleon! Thank you for your response. However, I wish to perform an animation from the BlueCoin sensor data and display it on the smartphone by modifying the existing ST Blue_MS Android application.

eg., when i put the sensor on my wrist and get the raw data from the BlueCoin, i wish to animate this data to move an object in the space, e.g. a leg or an arm performing a physiotherapy exercise.

[Eleon BORLINI]

Hi @MDabh​ , please note that there is no ad-hoc FW for your purpose... However, using the BlueCoin tool, you could start from the MEMS sensor fusion demo: this is based on the MotionFX_Manager.c and the related algorithm you can find in STM32CubeFunctionPack_ALLMEMS1_V3.4.0\Projects\Multi\Applications\ALLMEMS1\Src after downloaded ALLMEMS1 package.

The main function used is the following one, which translates the acceleration and rotation data into linear distance and angles, that can be plotted in 3D space.

void MotionFX_update(MFX_output_t *data_out, MFX_input_t *data_in, float eml_deltatime, float *eml_q_update);
 
/**
 * @brief Run the Kalman filter propagate
 * @param data_out pointer to the MFX_output_t structure
 * @param data_in pointer to the MFX_input_t structure
 * @param eml_deltatime delta time between two propagate calls [sec]
 * @retval none
 */

If you use also a magnetometer, you can check if the use described for example in Unicleo-GUI UM2128 at p. 15 can be a starting point for your application. Regards