Comprehensive upper-limb human-machine interaction strategies for post-stroke upper-limb rehabilitation

doi:10.21203/rs.3.rs-5002266/v1

Download PDF

Research Article

Comprehensive upper-limb human-machine interaction strategies for post-stroke upper-limb rehabilitation

https://doi.org/10.21203/rs.3.rs-5002266/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background

Stroke and its related complications, place significant burdens on human society in the 21st century, and lead to substantial demands for rehabilitation. To fulfill rehabilitation needs, human-machine interaction (HMI) technology strives continuously. Traditional passive HMI strategy requires device to be dynamically well-fitting, and executes command precisely. Advanced active HMI strategy also demands the device to react fast and accurately based on user’s intention. However, current study usually focuses on limited aspect of HMI, a complete HMI study which addresses the complexity of stroke related complications and provide the possibility for personalized post-stroke upper-limb rehabilitation is lacking.

Method

An Up-limb Rehabilitation Device and Utility System (UarDus) is proposed along with 3 HMI strategies namely robot-in-charge, therapist-in-charge and patient-in-charge. Based on physiological structure of human upper-limb and scapulohumeral rhythm (SHR) of shoulder, a base exoskeleton with 14 degrees of freedoms (DoFs) is designed as foundation of the 3 strategies. Passive robot-in-charge and therapist-in-charge strategies provides fully-assisted rehabilitation options. The active patient-in-charge strategy incorporates data acquisition matrices and a new deep learning model, which is developed based on CNN and Transformer structure, aims to provide partially-assisted rehabilitation.

Results

Kinematically, the work space of the base exoskeleton is presented first. Utilizing motion capture technology, the GH center of both human and exoskeleton is compared the well-matched curves suggesting comfortable dynamic wear experience. For robot-in-charge and therapist-in-charge strategy, the desired and measured angle-time curve present good correlation, with low phase difference, which serve the purpose of real-time control. Featuring the patient-in-charge strategy, Kernel Density Estimation (KDE) result suggesting reasonable sensor-machine-human synergy. Applying K-fold (K = 10) cross-validation method, the classification accuracy of the proposed model achieves an average of 99.6% for the designated 15 actions. The proposed model is also evaluated on public datasets, demonstrating outstanding response time and accuracy comparing with state-of-art models.

Conclusions

A validated exoskeleton hardware system named UarDus is constructed, along with the 3 HMI strategies proposed. This work provides possibility for people with stroke to engage in complex personalized rehabilitation training with dynamic comfortable wear experience.

Exoskeleton

Rehabilitation robotics

Human-machine interaction strategies

In the 21st century, aging population increases rapidly. Among the elderly, considerable portion of individuals suffer from strokes and related complications, which often lead to inability to perform daily activities [1]. Based on statistical analysis in 2022, the figure of stroke survivors reaches 101 million globally and the predicted increasement of new strokes each year is more than 12.2 million [2]. Previous study also suggests approximately 40% stroke survivors possess functional impairments, and physical rehabilitation is demanded to improve motor deficits [3, 4].

The clinical manifestation of post-stroke individual is complex, common symptoms include clonus, flexor spasms, spasticity, rigidity, Babinski sign, increased deep tendon reflexes, weakness, fatigue, coordination impairment, and loss of dexterity [5]. Post-stroke individual possesses different level of inability, which leads to personalized rehabilitation need. To tackle the issue, conventional post-stroke rehabilitation techniques include constraint-induced movement therapy, physical therapy and occupational therapy are adopted worldwide [6, 7], which require post-stroke patients to repeat fully or partially-assisted actions under professional supervision. However, due to financial factor and accessibility issue, it is often impractical for patients to receive enough doses of rehabilitation therapy, especially for patients in the chronic phase of stroke [6].

To address this issue, human-machine interaction (HMI) strategy develops continuously [8]. Passive HMI strategies usually focus on fully-assisted approach, which requires exoskeleton device to be physically well-fitting, and executes motion command precisely [9–11]. As artificial intelligence technology thrives, efforts are devoted for user-led, self-rehabilitation. Thus, active HMI strategies also demand device to react fast and accurately based on user’s intention [12–15].

Mechanical design criterion is the foundation for both passive and active HMI strategy. In rehabilitation training process, both static and dynamic joint alignment are vital, as to provide comfortable wear experience and avoid the possible harm to user such as skin abrasion and joint dislocation. Anatomically, the human upper limb can be divided into 3 major joints, which are shoulder, elbow, and wrist [16]. Consider the large distance between the major joints, it is easy to achieve static alignment for human joints and exoskeleton joints with translational mechanism. Due to simple anatomic structure, elbow and wrist can be regarded as joint with 1 degree of freedom (DoF) [4, 17]. In contrast, the motion of human shoulder requires the synergy of bones named thorax, clavicle, scapula, humerus as well as the 3 corresponding joints namely sternoclavicular joint (SC), acromioclavicular joint (AC), glenohumeral joint (GH). Most study treats shoulder as a single ball-and-socket joint [18], however, the scapulohumeral rhythm (SHR) of shoulder and dynamics wear fitness [19–22] is often neglected or poorly validated, which may leads to discomfort and injury in rehabilitation with exoskeleton.

Intention recognition is the most important aspect of active HMI strategy, which refers to the prediction of human activities based on sensor feedbacks [11]. In consideration of the data robustness and precision, muti-channel sensor setup is essential. In the recent decade, multi-channel electromyography (EMG) and electroencephalography (EEG) technology are extensively studied due to intention prediction potential [23]. However, the state-of-art non-invasive EMG and EEG signal usually requires significant data manipulation efforts due to the environment noises and signal complexity, which lead to noticeable delays in real-time control [24], and low prediction accuracy [24–26]. On the other hand, ethic consideration and cost-effectiveness impede the deployment of invasive bioelectrical signal technology in rehabilitation practices. Consider spasticity, rigidity and weakness symptoms of post-stroke patients [27, 28] gear ratio up to 1:100 is common in exoskeleton actuator [29]. As a result, significant torque is required for user to drive stationary exoskeleton joint. Thus, gather data via strain gauges [30], inertial measurement units (IMUs) [31], lasers [32], and computer vision approach [33], are questionable in real practices. Force transducer array for force-admittance control and is widely adopted [29, 34–36], however, most studies focus on control logarithm, the sensor-machine-human synergy is often neglected, which may lead to low accuracy in intention recognition, especially in dealing with complex postures/actions.

In addition to the sensor-machine-human synergy, data processing and intention prediction models are also crucial for active HMI strategy. Studies on intention prediction mainly focus on statistical approaches [37, 38] machine learning [39], and deep learning method [40, 41]. Statistical and machine learning methods are suitable for simple linear regression tasks [37–39]. For sophisticated classification and prediction tasks, two deep learning architectural are developed for spatial and sequential data format respectively. Representative model are Convolutional Neural Networks (CNN) [42, 43] and Recurrent Neural Networks (RNN) [44]. Although continuous innovation efforts in data modeling methods, feature extraction, and computational time reduction are devoted, leading to new model such as Long Short-Term Memory (LSTM) [45], Temporal Convolutional Neural Network (TCNN) [46], Transformer [47], AutoFormer [48] etc., the classification and prediction performance of each model is still highly sensitive to the format of specific dataset collected [49]. Thus, innovation and parametric study on deep learning architectural is still necessary in order to obtain the optimal balance between prediction accuracy and real-time performance.

Due to realization and manipulation complexity, upper-limb exoskeleton-based studies usually focus on limited aspects of HMI strategy [50–52]. However, consider the complexity of stroke related complication, a specific HMI strategy is hard to fulfill personalized needs. For instance, intention based active HMI strategy is inapplicable to people show symptoms such as clonus, flexor spasms, spasticity [5]. Secondly, real-time feedback from user is usually insufficient for robot-led passive HMI strategy, which may damp down rehabilitation expectation [15]. A complete HMI study which considers the complexity of stroke related complication lacks in literature, thus, aim of the work focus on comprehensive HMI strategies and provides possibility for complex personalized rehabilitation training needs. Mechanical design of the wearable device is the foundation of HMI strategies. Thus, mechanisms to achieve comfortable dynamics wear experience is stressed first. Consider the widely adopted conventional post-stroke rehabilitation techniques [6, 7], 3 HMI strategies namely robot-in-charge, therapist-in-charge and patient-in-charge are proposed, which enables both fully-assisted passive rehabilitation and partially-assisted active rehabilitation. The passive robot-in-charge strategy aims at assisting patients who are unable to move or possess limited mobility. Mimic the mechanism of the base exoskeleton, an additional motorless exoskeleton controller is designed for the passive therapist-in-charge strategy, which allows therapist to adjust rehabilitation therapy base on dynamic behavior of patients. The active patient-in-charge strategy incorporates force acquisition matrices with sensor-machine-human synergy consideration and a new deep learning model specified for sequential data format produced by the sensor matrices.

Consider the complexity of stroke related complications, individual needs for stroke rehabilitation varies. To provide possibility for personalized rehabilitation, Up-limb Rehabilitation Device and Utility System (UarDus) is proposed along with 3 HMI strategies. The UarDus is composed of four main subsystems which are mechanical, actuation, sensory and control (Fig. 1a). Mechanical, actuation and control subsystem are the hardware foundation of the 3 HMI strategies, thus is explained first. Sensory subsystem requirement for each HMI strategy is different, robot-in-charge strategy is purely passive and no sensory subsystem is required, for therapist-in-charge and patient-in-charge strategy distinct sensory subsystems are developed. Hence. sensory subsystems are discussed in each HMI strategy subsection.

The base mechanical structure of UarDus contains a wearable upper-limb exoskeleton (which allows to power the motion of shoulder, arm, forearm and hand) and a 3-axis translational motion platform which supports the weight of upper-limb exoskeleton and helps to coordinates patient’s SHR with the exoskeleton during rehabilitation process. Arm, forearm and hand can be adjusted in size for different personnel. The whole system has DoF up to 14 and provides sufficient dynamic wear comfortableness to user of different body sizes (Fig. 1b).

Mechanical structure and actuation mechanism of the base exoskeleton

Understand and mimic the working principle of human up-limb is the challenging aspect in the exoskeletons design. Anatomically, the human upper limb can be divided into 3 major joints, which are shoulder, elbow, and wrist (Fig. 2a-c) [16]. Featuring the human shoulder, its motion is mainly related to the 4 bones (thorax clavicle, scapula, and humerus) and the 3 corresponding joints SC, AC and GH. Though, all the 3 joints share similar working mechanism, which all can be treated as ball-and-socket joint [18]. Consider the overall SHR of shoulder [19] and the possible rotation range of shoulder (Table 1), a 6-DoF mechanism is proposed for shoulder. This mechanism contains 3 translational joints and 3 rotational joints. The 3 perpendicular translational joints (P1, P2, and P3 joints in Fig. 2d) are designed to compensate the motion caused by AC and SC and assist the movement of the GH center, while the 3 rotation joints (R6, R7, and R8 in Fig. 2d) are used to mimic the rotation in GH.

The angle between the clavicle and the Z-axis in Fig. 2c is defined as $\:{\gamma\:}_{s}$. The $\:{\gamma\:}_{a}$ is donated as the angles between the AC-GH line and the clavicle, while $\:{\gamma\:}_{g}$ is the angle between the humerus and the AC-GH line, respectively. The angle between the humerus and the Z-axis is denoted as $\:{\theta\:}_{1}$. The length of the clavicle is represented as $\:{l}_{c}$, and the distance between the center of the AC joint and the GH joint (Fig. 2c) is denoted as $\:{l}_{s}$. By establishing a mathematical relationship between the displacement of the GH joint center and $\:{\theta\:}_{1}$, it is possible to control three translational joints to move the exoskeleton arm, thereby compensating for the wearer's SHR [19].

Bone length can be assumed to be proportional to height (represented by h) of exoskeleton user [53]. Therefore, the ratio of the wearer's height to the measured volunteer's height is used as a scaling factor in the formula, making the identified patterns universally applicable. The value of $\:{\theta\:}_{1}$ can be obtained through an IMU worn on the wearer's forearm. Thus, the functional relationship expressions are derived as shown in equations (1)-(3). Adopting the empirical data [19], the $\:{\gamma\:}_{s0}=99^\circ\:$, $\:{\gamma\:}_{a0}=143^\circ\:$, $\:{h}_{0}=170\:\text{c}\text{m}$, $\:{\alpha\:}_{1}=70^\circ\:$, $\:{\alpha\:}_{2}=150^\circ\:$, $\:{\alpha\:}_{3}=180^\circ\:$, $\:{\beta\:}_{1}=80^\circ\:$, $\:{\beta\:}_{2}=140^\circ\:$, $\:{\beta\:}_{3}=180^\circ\:$ For more accurate personalized parameters such as$\:{\gamma\:}_{s0}$, $\:{\gamma\:}_{a0},{l}_{s},{l}_{c}$,etc., CT scans can be performed in order to enhance the accuracy of the GH joint center displacement trajectory expressed by the formula (1–3) shown as follows:

$$\:\begin{array}{c}\left(\genfrac{}{}{0pt}{}{{Y}_{GH}}{{z}_{GH}}\right)=\left(\genfrac{}{}{0pt}{}{{l}_{c}sin\left({\gamma\:}_{s}\left({\theta\:}_{1}\right)+{\gamma\:}_{s0}\right)-{l}_{s}sin({\gamma\:}_{a}\left({\theta\:}_{1}\right)+{\gamma\:}_{s}\left({\theta\:}_{1}\right)+{{\gamma\:}_{a0}+\gamma\:}_{s0})}{-{l}_{c}cos\left({\gamma\:}_{s}\left({\theta\:}_{1}\right)+{\gamma\:}_{s0}\right)+{l}_{s}cos\left(\right({\gamma\:}_{a}\left({\theta\:}_{1}\right)+{\gamma\:}_{s}\left({\theta\:}_{1}\right)+{{\gamma\:}_{a0}+\gamma\:}_{s0})}\right)\frac{h}{{h}_{0}}\:\#\left(1\right)\end{array}$$

$$\:\begin{array}{c}{\gamma\:}_{a}\left({\theta\:}_{1}\right)=\left\{\begin{array}{c}0^\circ\:\#for\:0^\circ\:\le\:{\theta\:}_{1}<{\alpha\:}_{1}\\\:35(10{\left(\frac{{\theta\:}_{1}-{\alpha\:}_{1}}{{\alpha\:}_{2}-{\alpha\:}_{1}}\right)}^{3}-15{\left(\frac{{\theta\:}_{1}-{\alpha\:}_{1}}{{\alpha\:}_{2}-{\alpha\:}_{1}}\right)}^{4}+6{\left(\frac{{\theta\:}_{1}-{\alpha\:}_{1}}{{\alpha\:}_{2}-{\alpha\:}_{1}}\right)}^{5}\#for\:{\alpha\:}_{1}\le\:{\theta\:}_{1}<{\alpha\:}_{2}\#\\\:35^\circ\:\#for\:{\alpha\:}_{2}\le\:{\theta\:}_{1}<{\alpha\:}_{3}\end{array}\right.\#\left(2\right)\end{array}$$

$$\:\begin{array}{c}{\gamma\:}_{s}\left({\theta\:}_{1}\right)=\left\{\begin{array}{c}15\left(10{\left(\frac{{\theta\:}_{1}}{{\beta\:}_{1}}\right)}^{3}-15{\left(\frac{{\theta\:}_{1}}{{\beta\:}_{1}}\right)}^{4}+6{\left(\frac{{\theta\:}_{1}}{{\beta\:}_{1}}\right)}^{5}\right)\#\#for\:0^\circ\:\le\:{\theta\:}_{1}<{\beta\:}_{1}\\\:15^\circ\:\#for\:{\beta\:}_{1}\le\:{\theta\:}_{1}<{\beta\:}_{2}\#\\\:15^\circ\:+9\left(10{\left(\frac{{\theta\:}_{1}-{\beta\:}_{2}}{{\beta\:}_{3}-{\beta\:}_{2}}\right)}^{3}-15{\left(\frac{{\theta\:}_{1}-{\beta\:}_{2}}{{\beta\:}_{3}-{\beta\:}_{2}}\right)}^{4}+6{\left(\frac{{\theta\:}_{1}-{\beta\:}_{2}}{{\beta\:}_{3}-{\beta\:}_{2}}\right)}^{5}\right)\#\#for\:{\beta\:}_{2}\le\:{\theta\:}_{1}<{\beta\:}_{3}\end{array}\right.\#\left(3\right)\end{array}$$

Compared to shoulder joints, the mechanism of the elbow (EL) joint is simple. As shown in the Fig. 2b, the EL joint connects of the distal end of the humerus with the proximal end of both radius and ulna, which is regarded as 1-DoF uniaxial hinge joint in this work (joint R10). The rotation of the forearm occurs at the distal end of the radius and ulna. In addition to the 2 DoFs introduced by wrist, these 3 uniaxial joints together (presented by R12, R13, and R14 in Fig. 2d) is mechanically equivalent to a ball-and-socket joint. Consider the mechanism for GH Joint, similar design is adopted (Fig. 3).

Table 1

The rotation range of upper-limb joint [54]
Joint	Shoulder	Elbow	Wrist
Flexion	+ 180°	+ 145°	+ 85°
Extension	-80°	-10°	-70°
Abduction	+ 180°	/	20°
Adduction	-50°	/	-40°
Internal rotation	+ 90°	/	/
External rotation	-90°	/	/
Supination	/	/	+ 90°
Pronation	/	/	-90°

Consider the needs of personalized training for both arms, the presence of passive joint R4 and the R5 (in Fig. 2d), allows bilateral training and the translational joint P9 and P11 enables the lengths adjustment of the exoskeleton linkages. To further improve the wear comfortableness and training result, joint P1, P2 and P3 are also adjustable for user with distinct body dimension. To provide a better kinematic description, the DH parameters for the upper-limb is established (Table 2).

Table 2

The DH parameters of the exoskeleton
link$\:\left(\varvec{i}\right)$	$\:{\varvec{\alpha\:}}_{\varvec{i}}$ (deg)	$\:{\varvec{a}}_{\varvec{i}}$ $\:\left(\varvec{m}\right)$	$\:{\varvec{\theta\:}}_{\varvec{i}}$ (deg)	$\:\mathbf{o}\mathbf{f}\mathbf{f}\mathbf{s}\mathbf{e}\mathbf{t}$ (deg)	$\:{\varvec{\theta\:}}_{\varvec{m}\varvec{i}\varvec{n}}$ (deg)	$\:{\varvec{\theta\:}}_{\varvec{m}\varvec{a}\varvec{x}}$ (deg)
1	60	0	$\:{\theta\:}_{1}$	-24.8	0	120
2	-60	0	$\:{\theta\:}_{2}$	85.49	0	38
3	0	$\:{a}_{1}$	$\:{\theta\:}_{3}$	114.8	-35	120
4	0	$\:{a}_{2}$	$\:{\theta\:}_{4}$	0	0	120
5	-90	0	$\:{\theta\:}_{5}$	90	-40	40
6	90	0	$\:{\theta\:}_{6}$	90	-90	90
7	0	0	$\:{\theta\:}_{7}$	0	-90	90

Control hardware overview

In the high-level design, the entire control system is composed of 4 major components: the upper limb rehabilitation exoskeleton, a master a controller (PC), a slave controller (STM32-F429 microcontroller, manufactured by Zhengdianyuanzi Ltd.), and the sensor matrix (Fig. 4). The master controller processes the sensor data collected by the slave controller and sends direct commands to the exoskeleton. An interface program is developed in the QT environment, which allows for mode selection, data collection (CSV format file), and data visualization.

For the sensors, considering the high real-time and high-resolution requirements, pressure thin-film sensor (RP-C18.3-ST, manufactured by Aodong Ltd.) and inertial sensor (IMU901, manufactured by Zhengdianyuanzi Ltd.) are chosen for HMI. The thin-film sensor module obtains real-time pressure values through the resistance-voltage conversion relationship:

$$\:\begin{array}{c}{\text{U}}_{0}=\left(1+{\text{R}}_{\text{A}\text{O}-\text{R}\text{E}\text{S}}\times\:\frac{1}{{\text{R}}_{\text{x}}}\right)\times\:0.1\#\left(4\right)\end{array}$$

Where ${{\text{R}}_{{\text{AO}} - {\text{RES}}}}$ is an adjustable resistor, and ${{\text{R}}_{\text{x}}}$ varies with respect to pressure. The real-time pressure data collected by the sensor is converted into analog voltage (0 ~ 3.3 V) using the ADC module on the slave. The IMU integrates a gyroscope, accelerometer, magnetometer, and barometer. The IMU outputs changes in pitch, roll, and yaw angles through the Universal Synchronous Asynchronous Receiver Transmitter (USART) module. To minimize interference from "abnormal data" caused by arm shaking and random arm movements and to ensure data reliability, a clamping filter algorithm (integrated into the STM32) is used to constrain abrupt data changes.

HIM strategies

Stroke patients require personalized rehabilitation therapy to recover from complicated complications. Even the same patient possess different needs in different rehabilitation stages [55]. Therefore, distinct rehabilitation treatment should be adopted accordingly. To address this issue, HMI strategies are employed to adjust rehabilitation therapy based on individual needs. Three different modes are designed for rehabilitation and daily life assistance, which are robot-in-charge, therapist-in-charge, and patient-in-charge mode.

The robot-in-charge training strategy aims at assisting patients who are unable to move or possess limited mobility. In this mode, the upper-limb exoskeleton guides the patient's limb along a pre-defined path recommended by a physician. The therapist-in-charge training strategy is suitable for patients at all stages. The patient-in-charge training strategy is intended for patients who are capable of low-intensity exercises. In this mode, both rehabilitation and daily activities assistance can be achieved by patient him/herself. The following sections provides more details about these 3 modes.

Robot-in-charge strategy

Figure 5 provides the control logic flowchart for the 3 rehabilitation modes. The Robot-in-charge mode is presented by green blocks. In this mode, the rehabilitation treatment therapy (motion of each joint) suggested by the therapist is collected, and the motion trajectories for each exoskeleton joint are planned and executed. The modes involve a dual-closed-loop control logic (Fig. 6 upper panel). Firstly, the servo motors used in this study integrates internal PID control loops, allowing precise control of the motor's position. An additional closed-loop control system is constructed with the aid of IMUs placed at each end of joints (shown in Fig. 8b) to eliminate errors caused by the mechanical structures (such as harmonic gear). The discrete PID control can be represented as follows:

$$\:\begin{array}{c}{\text{U}}_{\text{k}}={\text{k}}_{\text{p}}{\text{e}}_{\text{k}}+{\text{k}}_{\text{i}}\sum\:_{\text{j}=0}^{\text{k}}{\text{e}}_{\text{j}}+{\text{k}}_{\text{d}}\left({\text{e}}_{\text{k}}-{\text{e}}_{\text{k}-1}\right)\#\left(5\right)\end{array}$$

Where, ${U_k}$represents the system's output at time k, ${e_k}$ is the error at time k between the expected value and the actual value, and ${k_p}$,${k_i}$ and ${k_d}$ are the proportional, integral, and derivative coefficients, respectively. The lower left panel in Fig. 6 suggests this control system is capable of accurately tracking the joint angles.

Therapist-in-charge strategy

The control logic of therapist-in-charge mode is represented by the orange blocks in Fig. 5. In this mode, the therapist is required to wear a master, motor-less exoskeleton, which possess similar mechanism as the base exoskeleton. As shown in Fig. 7, 2 possible mapping methods, which utilizes IMU and encoder matrices, are proposed.

IMU based mapping:

IMUS capture the arm's motion state in real-time through IMU 1–3, as illustrated in Fig. 7. At the beginning of the therapist-in-charge mode, all IMU values are reset to zero. During system operation, the roll, pitch, and yaw angles obtained by IMU1 control the exoskeleton joints 1–3, the roll angle obtained by IMU 2 controls joint 4, and the roll, pitch, and yaw angles obtained by IMU 3 control the rehabilitation exoskeleton joints 5–7 accordingly. This process can be represented by the following equation:

$$\:\begin{array}{c}\left\{\begin{array}{l}{\left[\begin{array}{c}\text{J}\text{o}\text{i}\text{n}\text{t}1\\\:\text{J}\text{o}\text{i}\text{n}\text{t}2\\\:\text{J}\text{o}\text{i}\text{n}\text{t}3\end{array}\right]}_{\text{S}\text{h}\text{o}\text{u}\text{l}\text{d}\text{e}\text{r}}=\:{\left[\begin{array}{c}\text{R}\text{o}\text{l}\text{l}\\\:\text{P}\text{i}\text{t}\text{c}\text{h}\\\:\text{Y}\text{a}\text{w}\end{array}\right]}_{\text{I}\text{m}\text{u}1}\\\:{\left[\text{J}\text{o}\text{i}\text{n}\text{t}4\right]}_{\text{E}\text{l}\text{b}\text{o}\text{w}}=\:{\left[\text{P}\text{i}\text{t}\text{c}\text{h}\right]}_{\text{I}\text{M}\text{U}2}-\:{\left[\text{P}\text{i}\text{t}\text{c}\text{h}\right]}_{\text{I}\text{M}\text{U}1}\\\:{\left[\begin{array}{c}\text{J}\text{o}\text{i}\text{n}\text{t}5\\\:\text{J}\text{o}\text{i}\text{n}\text{t}6\\\:\text{J}\text{o}\text{i}\text{n}\text{t}7\end{array}\right]}_{\text{w}\text{r}\text{i}\text{s}\text{t}}=\:\:{\left[\begin{array}{c}\text{R}\text{o}\text{l}\text{l}\\\:\text{P}\text{i}\text{t}\text{c}\text{h}\\\:\text{Y}\text{a}\text{w}\end{array}\right]}_{\text{I}\text{m}\text{u}3}-{\left[\begin{array}{c}\text{R}\text{o}\text{l}\text{l}\\\:\text{P}\text{i}\text{t}\text{c}\text{h}\\\:\text{Y}\text{a}\text{w}\end{array}\right]}_{\text{I}\text{m}\text{u}2}\end{array}\right.\#\left(6\right)\end{array}$$

Encoder based mapping:

Encoders capture the arm's motion state in real-time utilizing Encoder 1–7, as illustrated in Fig. 7. The positions of the therapist exoskeleton and upper limb rehabilitation exoskeleton are calibrated based on the absolute value of encoders. During system operation, the real-time values from Encoder 1–7 are loaded to control the exoskeleton joints 1–7. This process can be represented by the following formula:

$$\:\begin{array}{c}{\text{J}\text{o}\text{i}\text{n}\text{t}}_{\text{I}}\:=\:{\text{E}\text{n}\text{c}\text{o}\text{d}\text{e}\text{r}}_{\text{i}}\#\left(7\right)\end{array}$$

Patient-in-charge strategy

Deep learning models are renowned for its robust predictive capabilities, making it adaptable for wider range of user. In this study, a patient-in-charge training strategy is developed based on deep learning technology, targeting post-stroke patients in chronic phase and without symptoms such as clonus, flexor spasms (because these two symptoms trends to generate fake intentions). To make a reasonable intention prediction, 4 steps are followed, which are data acquisition, data pre-processing, intention recognition model construction, and model performance evaluation.

Sensor Matrices Design:

Consider the geometry complexity of human upper-limb, to capture and measure HMI forces effectively is crucial for the exoskeleton operation. To address the challenge, an interaction sensor matrix for arm and a forearm is designed (in Fig. 8a-f), which contains 3 IMU units and 21 piezoelectric pressure sensors. The essential part of the mechanism is the sensor block (Fig. 8d), which contains 3 pressure senor units. The installed position of sensor blocks in arm/ forearm holder can be largely adjusted in one direction (0–30 mm), and each pressure sensor unit can be compressed up to 8mm, so as to fit people with diverse arm size.

Data acquisition and pre-processing:

Based on the Arm Motor Ability Test (AMAT) [56] and the upper-limb joint range of presented in Table 1, 15 rehabilitation actions for training are designed (Fig. 9). In this study, motion intention is captured and analyzed based on triggering patterns of sensor matrix. The correlation between action and active sensors are also highlighted.

For data collection, 4 healthy volunteers (authors of this work) are participated in the process, with each action repeated 250 times for each volunteer. This resulted in a total 15,000 action data sets. For these data sets, 80% are randomly selected for training, while the remaining 20% are allocated as test set. The repeated number for each of the 15 actions is equal in both training and test set.

Prior to data collection, calibration is performed on both the IMU and pressure sensors. The data collection frequency is fixed at 40 Hz. Notably, in the real practice, the recorded data pattern of individual sensors varied significantly even for the same posture/action. In order for a decent learning process, individual action data set with diversity are obtained on purpose

Data processing techniques, such as normalization and feature extraction, are essential for deep learning. Consider the different measurement ranges of sensors and the distribution patterns of each dataset, training without normalization may reduce training efficiency and the convergency of the created model might be more challenging. Normalization involves two steps. Firstly, scaling all numbers within each column to the range of [0,1] utilizing$\:\frac{\left(\varvec{x}-{x}_{Min}\right)}{{x}_{Max}-{x}_{Min}}$. Secondly, adjust the mean value of dataset to 0 based on $\:\frac{\left(\varvec{x}-\mu\:\right)}{\sigma\:}$. Figure 10 (b) shows the triggering patterns of the pressure sensors for the 15 different actions, with the aid of the aforementioned normalization process.

Intention recognition models:

In the field of intention recognition, deep learning methods are extensively exploited. However, most of these methods such as MobileNet [57] are predominately optimized for computer vision related applications. In addition, model size and real-time performance are also neglected, which largely constrains their deployment in wearable devices [10]. For these reasons, the model is proposed as depicted in Fig. 11. This model is designed to meet the requirements of high prediction accuracy, decent real-time performance, and lightweight deployment.

As shown in Fig. 11a, the model takes a tensor $\:Xϵ{R}^{N\times\:L\times\:W}$ as input, where N, L, and W represent batch size, length, and width, respectively. The tensor X goes through Blocks1, Blocks2, and Blocks3, resulting in the final output tensor $\:Xϵ{R}^{N\times\:M}$, where M represents the number of classes. Based on the data format, it can be noted that L = 200, W = 27, and M = 15. The hyperparameters for each module of the model are determined by the ‘Genetic Algorithm’ (Appendix Table 1).

To tackle time-series data in convolutional neural networks, it is necessary to preprocess the data and perform feature construction [13, 14]. Feature construction is fundamental for deep learning models to extract relevant features, which significantly influences the model's performance. The Block 0 module integrates the feature construction process into the proposed deep learning model, capable of learning feature from the data and construct parameters based on feature. As shown in Fig. 11b, Block 0 is the model's first layer, it processes 2 dimensional $\:200\times\:27$ raw data matrix and its output being a $\:3\times\:200\times\:27$ feature matrix. Specifically, Block 0 employs 3 parallel branches to obtain the data's embedding [47], local features and global features. Block 0 output data via a channel-wise concatenation. This approach serves as a "rough filter" for the data, helping the model quickly learn significant features (e.g., local features) and speeding up model convergence. The following section elaborates on the working principles of the 3 branches of the Block 0 module.

In the first branch, local features of the original data are obtained using the TimesBlock [58] module. The core idea of TimesBlock here is to decompose complex time-series variations into different periods using Fourier transformation. In the second branch, the original data tensor is projected to a high-dimensional space using embedding. Embedding comprises Token Embedding, Segment Embedding, and Position Embedding [47]. In the third branch, global features of the original data are obtained using the Encoder module [47]. The mathematical basis of this module is the self-attention mechanism, with the deployment of Encoder module, linear complexity is achieved, which accelerates the calculation [59]. The point convolution operation on this branch is used to adjust dimensions only. The output tensor of the 3 branches is concatenated along the channel dimension, allowing the input tensor to construct both local and global information. The output tensor of the Block 0 module can be represented as $\:{X}_{O}ϵ{R}^{C\times\:H\times\:W}$.

The feature extraction structure designed for this dataset should balance accuracy with lightweight requirements. To meet this goal, Block 1 and Block 2 modules are designed based on two classic feature extraction structures, which are ResNet and MobileNet [57]. In addition, Squeeze-and-Excitation (SE) attention mechanism is also incorporated in the above 2 blocks to further reduce parameters redundancy. The output tensor of the Block1 module can be represented as $\:{X}_{O}ϵ{R}^{C\times\:H\times\:W}$. Block 2 (Fig. 11b) is an inverted residual structure composed mainly of the SE module and depth wise separable convolution module [60]. The core idea of SE is to evaluate the importance of each feature channel. Based on the evaluation, vital features are excited while less important features are squeezed for the task. The squeeze and excitation operations in SE is represented as follows:

$$\:\begin{array}{c}\left\{\begin{array}{l}{Z}_{c}={F}_{sq}\left({u}_{c}\right)=\frac{1}{W\times\:H}\sum\:_{i=1}^{W}\sum\:_{j=1}^{H}{u}_{c}\left(i,j\right)\\\:S={F}_{ex}\left(z,W\right)=\sigma\:\left(g\left(z,W\right)\right)=\sigma\:\left({W}_{2}\delta\:\left({W}_{1}z\right)\right)\\\:{X}_{c}={F}_{scale}\left({u}_{c},\:{s}_{c}\right)={s}_{c}{u}_{c}\end{array}\right.\#\left(8\right)\end{array}$$

Where $\:{Z}_{C}ϵ{R}^{C\times\:1\times\:1}$, while $\:{u}_{c}$ $\:S$, $\:{X}_{c}$ represents input tensor, computed weight vector of channel dimension, output of particular channel. Besides, $\:g$ and $\:\sigma\:$ are donated as fully connected layer, and sigmoid activation function respectively. The core idea of depthwise separable convolution is to split a complete convolution process into two steps: depthwise convolution and pointwise convolution. Compared with standard convolution, it reduces the parameter number by 30% [57]. The output tensor of the Block 2 module can be represented as $\:{X}_{O}ϵ{R3}^{C\times\:H\times\:W}$.

Block 3 consists of convolution layers, batch normalization layers, and pooling layers. The convolution layer is used to extract features from specific data segments, while batch normalization ensures proper backpropagation gradients and alleviates the vanishing gradient problem. The pooling layer is employed to reduce the H and W dimensions of the tensor to 1. Point convolution is used instead of fully connected layers. The activation functions used in this study are as follows:

$$\:\begin{array}{c}\left\{\begin{array}{l}Relu\left(x\right)=max\left(0,\:x\right)\\\:swish\left(x\right)=xsigmoid\left(\beta\:x\right)\end{array}\right.\#\left(9\right)\end{array}$$

There are 4 hardware subsystems in UarDus, which are mechanical, actuation, sensory and control. The unique, dynamically fitting mechanical subsystem is the vital basis of all the HMI strategies, as such, the work space of the base exoskeleton is present and compared with human upper-limb first. Secondly, with the aid of Opensim software, the complex SHR of both human and exoskeleton are illustrated.

The simple control method and control effect of the two passive HMI strategies are explained in Fig. 6. Intention recognition results of patient-in-charge strategy, which involves the synergy of the proposed sensory sub-system and the new intention recognition model is mainly presented.

Base exoskeleton kinematic analysis

The AMAT is a tool used to assess activities of daily living (ADL) which includes 15 tasks for both unilateral and bilateral upper limbs. Comparing Fig. 12a and 12b, the exoskeleton is able to cover the necessary work space, and serve the purpose of assisting patients in ADL. Consider SHR and comfortable wear experience, with aid of Opensim software, the simulated displacement curve of the GH joint center for both human and exoskeleton are presented in Fig. 12c and 12d. As can be seen the two curves agrees well with each other.

Deep learning model performance assessment

To assess the performance of our model, 4 key indexes namely best prediction accuracy, average accuracy of K-fold cross-validation (where K = 10), response time, and model size are mainly stressed. The 4 key indexes of the proposed model are also compared with 3 representative classification model, including SVM (Support Vector Mechine, a machine learning model), ST-GCN (Spatial Temporal Graph Convolutional Networks, a graph network model), ResNet34 (a heavyweight performance model), and MobileNetV3 (a lightweight model) [57].

The confusion matrices presented in Fig. 13 suggest the proposed model and the ResNet34 possess superior prediction accuracy. Compare with SVM and ST-GCN, the proposed model achieved ~ 2.5% higher ‘best prediction accuracy’, which is significant for HMI application as false intention prediction may injure exoskeleton users. With random training and test sets, K-fold cross-validation method (K = 10) is performed. The average prediction accuracy of the proposed model reaches 99.7%. Moreover, consider the size of the proposed model, the computation time is also competitive with the famous light weight Mobilenet [57], which is reflected by the response time shown in Table 3.

Table 3

The performance of different models on the 4 evaluation indexes
Model	Best Prediction Accuracy	K-Fold cross-validation (K = 10)	Response Time (ms)	Size (MB)
SVM	0.9753	0.976	64	/
ST-GCN	0.9760	0.982	12	10
ResNet34	1.0	0.993	13	81
MobilelNet	0.9990	0.997	12	8.55
OurModel	0.9997	0.997	11	7.09

An exoskeleton hardware system named UarDus is constructed, along with the 3 HMI strategies proposed, which includes 2 passive (robot-in-charge, therapist-in-charge) and 1 active (patient-in-charge) rehabilitation strategies. Function of UarDus and the 3 HMI strategies requires the synergy of four hardware subsystems which are mechanical, actuation, sensory and control. In the discussion section the performances of the unique mechanical and sensory subsystems (for patient-in-charge strategy) are validated utilizing experiment and statistical approach. In addition, the new intention prediction model targeting time-series data is assessed via public data sets.

Dynamics fitness validation for the base exoskeleton

Base exoskeleton and the master exoskeleton (for therapist-in-charge strategy) share the same trajectory for each joint, thus, mechanical design of the exoskeleton is essential for both passive and active HMI strategies. Dynamics fitness is the foundation for comfortable wear experience and rehabilitation performance. Poor dynamics fitness may cause harm to user, for instance, skin abrasion and joint dislocation. As such, dynamics fitness of the base exoskeleton is validated first.

Camera array (PrimeX 13) along with the software platform (Motive 3.1) produced by OptiTrack is deployed to capture the motion of grey markers, which is highlighted in Fig. 14. This setup allows high speed, precise tracking with accuracy of ~ 0.20 mm. For up-limb rehabilitation, SHR is the major factor for dynamic fitness, as such, 1 marker is placed on top of the human GH center and additional 2 are placed in the vicinity for calibration purpose. For the base exoskeleton, markers are placed on top of each joint (geometry center). The rotation axis of each shoulder joint meets at a single point, which is the GH center of exoskeleton. Utilizing the robot-in-charge strategy, trajectories of human and exoskeleton GH points for action 2 (subplot number 3 in Fig. 9) captured. Compare the position of GH points along Y and Z axis, curves match well in general. However, with comfortable user feedback, slight mismatch between the curves can still be captured, which can be caused by multiple factors such as skin deformation, accuracy of the motion capture system, inaccurate human data measurement etc. The misalignment can be improved utilizing advance CT technology as well as additional passive DoF design.

Effectiveness validation for the data acquisition matrices

Sensor-machine-human synergy is essential for patient-in-charge mode. Consider the complex postures/actions performed in Fig. 9, bad synergy may lead to low accuracy in intention recognition, which jeopardizes the rehabilitation scheme. A total 15000 action sets are conducted by 4 volunteers. Therefore, statistic approach is suitable to evaluated the large amount of data and validate the effectiveness of the sensor-machine-human synergy.

Kernel Density Estimation (KDE) method is adopted to illustrate the probability density distribution of random training and testing datasets. It is observed that these two distribution patterns are consistent with each other (Fig. 15), suggesting high efficacy of the data acquisition matrices setup. With this setup, prediction models are more likely to achieve good training and intention estimation.

Performance validation for the proposed prediction model

Regarding the high prediction accuracy behavior, performance of the proposed model is also validated and compared using public datasets UEA [61]. The UEA contains 4 sub-datasets which are ECG 5000, UWGLA (UWaveGestureLibraryAll), StandWalkJump and NATOPS. In the validation test, training parameters are consistent with the previous experiment and no pretraining/optimization is applied to any parameters. The overall prediction accuracy of all the models is close on single-variable time-series datasets (ECG 5000, UWGLA). However, regarding multivariate time-series datasets (StandWalkJump, NATOPS), ResNet [62] and MobileNet [60] are struggled to capture complex temporal features and unable to achieve convergence, thus the proposed model demonstrated clear superiority (Fig. 16). The above validation agrees well with the results presented in Table 3.

The superior performance of the proposed model in terms of overall prediction accuracy, response time, and model size compared to ResNet [62] and ST-GCN [63] can be attributed to the following two main reasons. Firstly, the dataset collected in this study is focused on time-series data, which is distinct from general-purpose models like ResNet34 [64]. Block 0 is specifically designed for feature extraction from time-series data, which enhances model's accuracy. Secondly, the design of Block 1 and Block 2 reduced the model's size while ensuring response time and training convergence speed. All the reason above makes the proposed model well-suited for the deployment on wearable devices.

Targeting personalized rehabilitation training for post-stroke patient, UarDus is proposed, along with 3 rehabilitation modes namely robot-in-charge, therapist-in-charge, and patient-in-charge mode. Aims to provided dynamic comfortable wear experience for user, a 14 DoF base exoskeleton is designed and validated via motion capture system. Robot-in-charge strategy targets patient with limited mobility and utilizes therapist’s treatment prescription for rehabilitation. Therapist-in-charge strategy serve the purpose of high feasibility. An additional motorless master exoskeleton is designed, which allows patient to accurately mimic and repeat the movement of therapist. The patient-in-charge strategy aims for chronic phase post-stroke patients with no clonus and flexor spasms history (to avoid fake intentions) and provides support based on detected motion intention. Data acquisition matrices with good sensor-machine-human synergy is built for this strategy. For intention recognition, targeting the time-series data structure, this study puts forth a new light-weight, deep learning model based on the existing CNN and Transformer structure. Applying K-fold (K = 10) cross-validation method, the classification accuracy of the proposed model achieves an average of 99.6% for the designated 15 actions. The proposed model is also evaluated on public datasets, demonstrating outstanding response time and accuracy by comparing with the state-of-art prediction models. Enlighted by conventional post-stroke rehabilitation techniques, this works stresses both fully and partially assisted approach, and provides possibility for personalized post-stroke rehabilitation

HMI	Human-machine interaction
UarDus	Up-limb Rehabilitation Device and Utility System
DoF	Degrees of Freedom
SHR	Scapulohumeral rhythm
EMG	Electromyography
EEG	Electroencephalography
SC	Sternoclavicular joint
AC	Acromioclavicular joint
GH	Glenohumeral joint
EL	Elbow
CNN	Convolutional Neural Networks
RNN	Recurrent Neural Networks
LSTM	Long Short-Term Memory
TCNN	Temporal Convolutional Neural Network
SVM	Support Vector Mechine
ST-GCN	Spatial Temporal Graph Convolutional Networks
IMU	Inertial measurement unit
KDE	Kernel Density Estimation
PID	Proportion Integration Differentiation
USART	Universal Synchronous Asynchronous Receiver Transmitter
ADL	Activities of daily living

Ethics approval and consent to participate

Prior to participation, all subjects (were authors of this study) were informed of the experimental procedure, and each provided written consent. This study was approved by the Institutional Review Board of Hohai university.

Consent for publication

All participants provided written and informed consent for the publication of these data.

Competing interests

The authors declare that they have no competing interests.

Funding

This research is funded by National Natural Science Foundation of China (12102127), China Postdoctoral Science Foundation (2021M690872), Changzhou Health Commission Technology Projects (ZD202103), 2021 Annual Performance Evaluation of Technology Transfer in Jiangning District (521053812/2093).

Author Contribution

KX, XDC, CSL, HS and YMW conceived the conception and the design of the study. KX, XDC, CSL and XWW performed the experiment. KX, XDC and CSL analyzed the data. KX XDC and CSL drafting the article. All authors reviewed and approved the final manuscript.

Acknowledgements

Not applicable

Data Availability

The datasets referenced within the article are available from the corresponding author on reasonable request.

Zhang L, Zhang H, Zhou X, Zhao J, Wang X. Bibliometric analysis of research on migraine-stroke association from 2013 to 2023. J Pain Res. 2023;16:4089–112.
Thayabaranathan T, Kim J, Cadilhac DA, Thrift AG, Donnan GA, Howard G, et al. Global stroke statistics 2022. Int J Stroke. 2022;17(9):946–56.
Duncan PW, Zorowitz R, Bates B, Choi JY, Glasberg JJ, Graham GD, et al. Management of adult stroke rehabilitation care: a clinical practice guideline. Stroke. 2005;36(9):e100–43.
Grimm F, Kraugmann J, Naros G, Gharabaghi A. Clinical validation of kinematic assessments of post-stroke upper limb movements with a multi-joint arm exoskeleton. J Neuroeng Rehabil. 2021;18(1):92.
Alzahrani WM, Aljundi ZE, Sulaiman AA, Bagadood RM. Acute spasticity secondary to ischemic stroke involving superior frontal gyrus and anterior cingulate gyrus. J Neurosci Rural Pract. 2023;14(4):741–3.
Cramer SC. Intense rehabilitation therapy produces very large gains in chronic stroke. J Neurol Neurosurg Psychiatry. 2019;90(5):497–497.
Huo Y, Wang X, Zhao W, Hu H, Li L. Effects of EMG-based robot for upper extremity rehabilitation on post-stroke patients: a systematic review and meta-analysis. Front Physiol. 2023;14:1172958.
Frisoli A, Barsotti M, Sotgiu E, Lamola G, Procopio C, Chisari C. A randomized clinical control study on the efficacy of three-dimensional upper limb robotic exoskeleton training in chronic stroke. J Neuroeng Rehabil. 2022;19(1):14.
Guo L, Lu Z, Yao L. Human-machine interaction sensing technology based on hand gesture recognition: a review. IEEE T Hum-Mach Syst. 2021;51(4):300–9.
Nasri N, Orts-Escolano S, Cazorla M. An sEMG-controlled 3D game for rehabilitation therapies: real-time time hand gesture recognition using deep learning techniques. Sensors-Basel. 2020;20(22):6451.
Gordleeva SY, Lobov SA, Grigorev NA, Savosenkov AO, Shamshin MO, Lukoyanov MV, et al. Real-time EEG–EMG human–machine interface-based control system for a lower-limb exoskeleton. IEEE Access. 2020;8:84070–81.
Mörtl A, Lawitzky M, Kucukyilmaz A, Sezgin M, Basdogan C, Hirche S. The role of roles: physical cooperation between humans and robots. Int J Robot Res. 2012;31(13):1656–74.
Zhu M, Sun Z, Zhang Z, Shi Q, He T, Liu H, et al. Haptic-feedback smart glove as a creative human-machine interface (HMI) for virtual/augmented reality applications. Sci Adv. 2020;6(19):eaaz8693.
Xia K, Chen X, Chang X, Liu C, Guo L, Xu X, et al. Hand exoskeleton design and human–machine interaction strategies for rehabilitation. Bioengineering. 2022;9(11):682.
Xu P, Xia D, Li J, Zhou J, Xie L. Execution and perception of upper limb exoskeleton for stroke patients: a systematic review. Intel Serv Robot. 2022;15(4):557–78.
Pang Z, Wang T, Wang Z, Yu J, Sun Z, Liu S. Design and analysis of a wearable upper limb rehabilitation robot with characteristics of tension mechanism. Appl Sci-Basel. 2020;10(6).
Kim B, Deshpande AD. An upper-body rehabilitation exoskeleton Harmony with an anatomical shoulder mechanism: design, modeling, control, and performance evaluation. Int J Robot Res. 2017;36(4):414–35.
Majidi Fard Vatan H, Nefti-Meziani S, Davis S, Saffari Z, El-Hussieny H. A review: a comprehensive review of soft and rigid wearable rehabilitation and assistive devices with a focus on the shoulder joint. J Intel Robot Syst. 2021;102(1).
Nef T, Guidali M, Riener R. ARMin III – arm therapy rxoskeleton with an ergonomic shoulder actuation. Appl Bionics Biomech. 2009;6(2):127–42.
Vitiello N, Cempini M, Crea S, Giovacchini F, Cortese M, Moise M, et al. Functional design of a powered elbow orthosis toward its clinical employment. IEEE-ASME T Mech. 2016;21(4):1880–91.
Trigili E, Crea S, Moise M, Baldoni A, Cempini M, Ercolini G, et al. Design and experimental characterization of a shoulder-elbow exoskeleton with compliant joints for post-stroke rehabilitation. IEEE-ASME T Mech. 2019;24(4):1485–96.
Hsu HY, Koh CL, Yang KC, Lin YC, Hsu CH, Su FC, et al. Effects of an assist-as-needed equipped tenodesis-induced-grip exoskeleton robot (TIGER) on upper limb function in patients with chronic stroke. J Neuroeng Rehabil. 2024;21(1):5.
Hortal E, Planelles D, Resquin F, Climent JM, Azorin JM, Pons JL. Using a brain-machine interface to control a hybrid upper limb exoskeleton during rehabilitation of patients with neurological conditions. J Neuroeng Rehabil. 2015;12(1):92.
Zhang X, Zhou P. High-density myoelectric pattern recognition toward improved stroke rehabilitation. IEEE Trans Biomed Eng. 2012;59(6):1649–57.
Xiao F, Chen Y, Zhu Y. GADF/GASF-HOG:feature extraction methods for hand movement classification from surface electromyography. J Neural Eng. 2020;17(4):046016.
Tang Z, Zhang L, Chen X, Ying J, Wang X, Wang H. Wearable supernumerary robotic limb system using a hybrid control approach based on motor imagery and object detection. IEEE Trans Neural Syst Rehabil Eng. 2022;30:1298–309.
GE SL, WZ F. A new definition of poststroke spasticity and the interference of spasticity with motor recovery from acute to chronic stages. Neurorehabilit Neural Repair. 2021;35(7):601–10.
Roots J, Trajano GS, Fontanarosa D. Ultrasound elastography in the assessment of post-stroke muscle stiffness: a systematic review. Insights into imaging. 2022;13(1):67.
Zarrin RS, Zeiaee A, Langari R. A variable-admittance assist-as-needed controller for upper-limb rehabilitation exoskeletons. IEEE Robot Autom Lett. 2024;9(6):5903–10.
Ramli NA, Nordin AN, Zainul Azlan N. Development of low cost screen-printed piezoresistive strain sensor for facial expressions recognition systems. Microelectron Eng. 2020;234(3):111440.
Qiu S, Zhao H, Jiang N, Wang Z, Liu L, An Y, et al. Multi-sensor information fusion based on machine learning for real applications in human activity recognition: state-of-the-art and research challenges. Informa Fusion. 2022;80:241–65.
Zhang H, Liu Y, Wang C, Fu R, Sun Q, Li Z. Research on a pedestrian crossing intention recognition model based on natural observation data. Sensors-Basel. 2020;20(6):1776.
Vélez-Guerrero MA, Callejas-Cuervo M, Álvarez JC, Mazzoleni S. Assessment of the mechanical support characteristics of a light and wearable robotic exoskeleton prototype applied to upper limb rehabilitation. Sensors-Basel. 2022;22(11):3999.
Huang J, Huo W, Xu W, Mohammed S, Amirat Y. Control of upper-limb power-assist exoskeleton using a human-robot interface based on motion intention recognition. IEEE T Autom Sci Eng. 2015;12(4):1257–70.
Tian D, Li W, Li J, Li F, Chen Z, He Y, et al. Self-balancing exoskeleton robots designed to facilitate multiple rehabilitation training movements. IEEE T Neur Sys Reh. 2024;32:293–303.
Ahmed T, Islam MR, Brahmi B, Rahman MH. Robustness and tracking performance evaluation of PID motion control of 7 DoF anthropomorphic exoskeleton robot assisted upper limb rehabilitation. Sensors-Basel. 2022;22(10):3747.
Tryon J, Trejos AL. Classification of task weight during dynamic motion using EEG–EMG fusion. IEEE Sens J. 2021;21(4):5012–21.
Tunca C, Pehlivan N, Ak N, Arnrich B, Salur G, Ersoy C. Inertial sensor-based robust gait analysis in non-hospital settings for neurological disorders. Sensors-Basel. 2017;17(4):825.
Tahir S, Jalal A, Kim K. Wearable inertial sensors for daily activity analysis based on adam optimization and the maximum entropy markov model. Entropy-Switz. 2020;22(5):579.
Liu L. Objects detection toward complicated high remote basketball sports by leveraging deep CNN architecture. Future Gener Comp Sy. 2021;119:31–6.
Zhou X, Liang W, Wang KIK, Wang H, Yang LT, Jin Q. Deep-learning-enhanced human activity recognition for internet of healthcare things. IEEE Internet Things. 2020;7(7):6429–38.
Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM. 2017;60(6):84–90.
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D et al. Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition. 2015. pp. 1–9.
Lipton ZC, Berkowitz J, Elkan C. A critical review of recurrent neural networks for sequence learning. arXiv preprint. 2015. arXiv:1506.00019
Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997;9(8):1735–80.
Bai S, Kolter JZ, Koltun V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint. 2018. arXiv:1803.01271.
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN et al. Attention is all you need. In: 31st International Conference on Neural Information Processing Systems. 2017 p. 6000–6010.
Wu H, Xu J, Wang J, Long M. Autoformer: decomposition transformers with auto-correlation for long-term series forecasting. Adv Neural Inf Process Syst. 2021;34:22419–30.
Benidis K, Rangapuram SS, Flunkert V, Wang Y, Maddix D, Turkmen C, et al. Deep learning for time series forecasting: tutorial and literature survey. ACM Comput Surv. 2022;55(6):1–36.
Jalal MFA, Harith HH, Hasan WZW, Salim MS, Lin T-TJIJIE. Exoskeletons for elderly activity of daily living assistance: a review of upper limb exoskeletons and assessments. Int J Integr Eng. 2024;16(1):87–105.
Khalid S, Alnajjar F, Gochoo M, Renawi A, Shimoda S. Robotic assistive and rehabilitation devices leading to motor recovery in upper limb: a systematic review. Disabil Rehabil-Assi. 2023;18(5):658–72.
Ochieze C, Zare S, Sun Y. Wearable upper limb robotics for pervasive health: a review. Prog Biomed Eng. 2023;5(3):032003.
Cheng JCY, Leung SSF, Chiu BSK, Tse PWT, Xu YYY. Can we predict body height from segmental bone length measurements? a study of 3,647 children. J Pediatr Orthoped. 1998;18(3):387–93.
Perry JC, Rosen J. Case study: an upper limb powered exoskeleton. In: Pons JL, editor. Wearable robots: biomechatronic exoskeletons. Chichester: John Wiley & Sons Ltd; 2008. pp. 259–69.
Jaramillo-Yanez A, Benalcazar ME, Mena-Maldonado E. Real-time hand gesture recognition using surface electromyography and machine learning: a systematic literature review. Sensors-Basel. 2020;20(9):2467.
Kopp B, Kunkel A, Flor H, Platz T, Rose U, Mauritz K-H, et al. The arm motor ability test: reliability, validity, and sensitivity to change of an instrument for assessing disabilities in activities of daily living. Arch Phys Med Rehab. 1997;78(6):615–20.
Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T et al. Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint. 2017. arXiv:1704.04861.
Wu H, Hu T, Liu Y, Zhou H, Wang J, Long M. Timesnet: temporal 2d-variation modeling for general time series analysis. arXiv preprint. 2022. arXiv:2210.02186.
Zhou H, Zhang S, Peng J, Zhang S, Li J, Xiong H et al. Informer: beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of the AAAI conference on artificial intelligence. 2021. pp. 11106–11115.
Howard A, Sandler M, Chu G, Chen LC, Chen B, Tan M et al. Searching for mobilenetv3. In: Proceedings of the IEEE/CVF international conference on computer vision. 2019. pp. 1314–1324.
Bagnall A, Dau HA, Lines J, Flynn M, Large J, Bostrom A et al. The UEA multivariate time series classification archive. arXiv preprint. 2018. arXiv:1811.00075.
Szegedy C, Liu W, Jia Y, Sermanet P, Rabinovich A. Going Deeper with Convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2014. pp. 1–9.
Yan S, Xiong Y, Lin D. Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition. In: Proceedings of the AAAI conference on artificial intelligence. 2018. p.
Jefferson KB. ResNet 34. In: Koonce B, editor. Convolutional neural networks with swift for tensorflow: image recognition dataset categorization. Berkeley: A; 2021. pp. 51–61.

No competing interests reported.

Download PDF

Reviewers agreed at journal
21 Sep, 2024
Reviewers invited by journal
16 Sep, 2024
Editor assigned by journal
02 Sep, 2024
Submission checks completed at journal
02 Sep, 2024
First submitted to journal
30 Aug, 2024

You are reading this latest preprint version

Comprehensive upper-limb human-machine interaction strategies for post-stroke upper-limb rehabilitation

Status:

Version 1

Abstract

Background

Method

Results

Conclusions

Figures

Introduction

Methods

Mechanical structure and actuation mechanism of the base exoskeleton

Control hardware overview

HIM strategies

Results

Base exoskeleton kinematic analysis

Deep learning model performance assessment

Discussion

Dynamics fitness validation for the base exoskeleton

Effectiveness validation for the data acquisition matrices

Performance validation for the proposed prediction model

Conclusion

Abbreviations

Declarations

Funding

Author Contribution

Acknowledgements

Data Availability

References

Additional Declarations

Status:

Version 1

link\(\:\left(\varvec{i}\right)\)	\(\:{\varvec{\alpha\:}}_{\varvec{i}}\) (deg)	\(\:{\varvec{a}}_{\varvec{i}}\) \(\:\left(\varvec{m}\right)\)	\(\:{\varvec{\theta\:}}_{\varvec{i}}\) (deg)	\(\:\mathbf{o}\mathbf{f}\mathbf{f}\mathbf{s}\mathbf{e}\mathbf{t}\) (deg)	\(\:{\varvec{\theta\:}}_{\varvec{m}\varvec{i}\varvec{n}}\) (deg)	\(\:{\varvec{\theta\:}}_{\varvec{m}\varvec{a}\varvec{x}}\) (deg)
1	60	0	\(\:{\theta\:}_{1}\)	-24.8	0	120
2	-60	0	\(\:{\theta\:}_{2}\)	85.49	0	38
3	0	\(\:{a}_{1}\)	\(\:{\theta\:}_{3}\)	114.8	-35	120
4	0	\(\:{a}_{2}\)	\(\:{\theta\:}_{4}\)	0	0	120
5	-90	0	\(\:{\theta\:}_{5}\)	90	-40	40
6	90	0	\(\:{\theta\:}_{6}\)	90	-90	90
7	0	0	\(\:{\theta\:}_{7}\)	0	-90	90