Real-Time Torque Prediction for Adaptive Exoskeleton Control

ROS / Machine Learning / Python

As part of Professor Jose Pons’ Neurorehabilitation and Neural Engineering Laboratory I helped develop the framework for realtime torque prediction for an adaptive ankle exoskeleton controller intended for stroke rehabilitation. Specifically, I built the pipeline to predict joint torque in real-time from high-density EMG measurements.

Project Goals

  • Build the data pipeline to stream high-density EMG signals from the OTB Quattrocento and torque measurements from the H3 ankle exoskeleton and perform the necessary processing steps such that a model can be fit to predict torque from the processed EMG readings
  • Integrate the existing convolutional neural net (CNN) into the real-time framework and use the output cumulative spike train (CST) predictions to predict torque. Compare this method with the simple root-mean-square EMG (RMS EMG) method
  • Prepare a user-friendly and modular ROS package so future researchers can continue to develop more complex protocols

Procedure

There are three primary stages involved in the pipeline for real-time torque prediction.

  1. Data Stream: Manage raw signals from the OTB Quattrocento high-density EMG bioelectrical amplifier and the Technaid H3 ankle exoskeleton.

    • The exoskeleton is ROS compatible and publishes sensor data to a topic with custom message types, allowing for the ROS nodes to access this information directly. To access the data from the Quattrocento, there is an intermediate ROS node that sets up a socket for the data stream and then publishes to a topic used by other nodes.

  2. Calibration: The core of this project. Receive, organize, and process raw data streams to fit a model that estimates torque from muscle activity.

    • The calibration procedure starts with the subject sitting comfortably with the right ankle resting with a roughly 90 degree hip angle and 180 degree knee angle.

    The subject follows a modular set of tasks that consist of a sequence of 2 maximum voluntary contractions and then a torque trajectory to follow. The ankle exoskeleton is locked at a specified joint angle, and the subject pushes or pulls to generate a joint torque following the target trajectory. The trajectory is configurable in its shape (trapezoidal or sinusoidal), directionality (plantarflexion or dorsiflexion), joint angle, effort level (percentage of MVC), and duration. The modularity of this step is intended to support future work looking into the optimal combination of these parameters. Throughout the calibration, the subject follows a series of prompts guiding them through the process and receives real-time feedback on a plot tracking applied torque overlaid on the target trajectory.

    At the end of the calibration sequence, the node will finish post-processing steps and calculate optimal coefficients to fit a model from the torque and muscle activity data.

    Root Mean Square EMG data, Torque, and Angles across a baseline period and 5 tasks

  3. Real-Time Prediction: Apply the model

    • Once the calibration sequence is finished, the exoskeleton can be switched to torque control mode so it begins following the torque commands predicted from continuous EMG data collection following similar processing steps to the calibration procedure.

Data Processing

Depending on which method (CST or RMS EMG) is selected for prediction, the data processing requirements differ significantly.

Raw data sampled at approximately 100 Hz

Method 1: Root-Mean-Square EMG

After the intermediate processing steps, a final data frame is constructed from data recorded during each trajectory task including RMS EMG from each muscle, joint torque, and the fixed joint angle for the task.

Root Mean Square EMG data, torque, and angles across a baseline period and 5 tasks

The final data frame is fed into an optimization algorithm that calculates coefficients to minimizes the error between the predicted torque and recorded torque values.

Predicted torque measurements plotted on top of measured values to show model performance

RMS EMG is the more forgiving of the methods due to the several layers of filtering and processing, so the model fit is less susceptible to noise.

Method 2: Cumulative Spike Train Estimation

A key processing step for accurate motor unit activation predictions is normalizing the raw data to the maximum of each muscle. As shown in the plot below, this can be difficult with the noise and artifacts even after an outlying peak-removing step.

Normalized Raw EMG Data from 3 Channels

Motor unit activation predicted from the neural net should gradually increase from 0 to 4, mapping to no activation through maximum activation. The neural net is primarily looking at the shape of the signal, so it is not surprising that the spike predictions do not follow a regular 0 to 4 to 0 gradual increase/decrease pattern if there is a low signal to noise ratio. Ideally, we would look for patterns more similar to the end of the channel in the topmost chart below.

Neural Net Output

However, despite the low signal quality, after the convolution across the 400ms Hanning window, trends do actually appear.

Cumulative spike train estimation, torque, and angles across a baseline period and 5 tasks.

The final data frame containing CST predictions, torque, and angles are fed into an identical optimization algorithm using the same model as the RMS EMG method.

Predicted torque measurements plotted on top of measured values to show model performance

Despite the signal quality, the data pipeline shows promise for accurately modeling joint torque from cumulative spike train estimations.

Model Evaluation

The model used for all methods is a non-linear physiologically informed equation accounting for each muscle, joint angle, and the interaction of each muscle with the joint angle. To understand the state of predictions, I evaluated the model performance across six possible calibration procedures using the RMS EMG method. This initial analysis shows that there is significant room for improvement with the generalizability of the calibration. However, for a foundational framework, this is a promising first step towards a more robust calibration procedure.

R2 of model fit for each calibration task set training and testing on the same data

To evaluate the generalizability, I tested the model trained on each calibration procedure on every task. This heatmap shows that in general each calibration procedure performs well for the tasks included in the training, but will struggle as the tasks become more disparate, especially between differently shaped trajectories.

R2 of fit for each calibration task set tested on each individual task using Trial 1 Data. All three trials showed similar trends.

A box plot of the same data further shows the variation in the R2 value. While the distribution tends toward more acceptable R2 values, each procedure contains extreme outliers where the model is effectively useless, if not counter-productive (a negative R2 indicates the model would perform better with a constant value).

Outcomes

  • Proof of Concept
    • This package establishes a foundation for real-time torque prediction from high-density EMG. It will support many future studies looking into optimizing various aspects of this pipeline.
  • Adaptation of existing methodologies to the quirks of this system
    • The data processing and modeling in this project are not novel concepts. However, combining procedures into a functional ROS package that could operate in real-time was a new challenge that required adaptions to existing methodologies to account for the increased signal noise and processing speed requirements.
  • Understanding tradeoffs of CST and RMS EMG Methods
    • At its current state, the CST method requires less data processing but is more sensitive to signal quality, while RMS EMG requires more processing but can handle lower signal to noise ratios. Further developments will look into strategies to mitigate noise and decrease processing time in order to improve the effectiveness of both methods.
  • Piloting with n=4
    • I tested the system on 4 individuals including myself. These trials demonstrated the usability of the package and provided a varied data set to evaluate the current state of prediction methods.

View This Project on Github