Participant Characteristics for the Survey and Intervention
A total of 600 eligible participants fulfilled the inclusion criteria, after confirming their willingness to intervene through telephone communication, 107 participants willing to receive intervention were randomly assigned to the biofeedback or waiting list control (WLC) group using a web-based randomization procedure (https://www.random.org/lists/) implemented by an independent research assistant. Symptom scale and vocal acoustic measurements were performed at baseline and after the intervention for the biofeedback (n=54) and WLC groups (n=53) separately. The biofeedback group performed 12 intervention sessions over a 4-week period, with the control group waiting for 4 weeks. After the voice quality control, the biofeedback intervention group consisting of 52 biofeedback participants (n=18, female, 34.62%; mean age, 19.02; SD, 0.94) and the waiting group (n=52) consisting of 52 WLC participants (n=18, female, 34.62%; mean age, 18.81; SD, 0.79) finally completed the voice data collection before and after treatment.
At baseline, the biofeedback and waiting-list group did not differ in age (T=1.241, P=0.218), sex (C2=0.000, P=1.000), and depression (T=1.014, P=0.313), anxiety (T=0.141, P=0.888), insomnia symptoms (T=0.641, P=0.523) or perceived stress (T=0.676, P=0.501).
Biofeedback Intervention Effect of Clinical Symptoms, Acoustic Speech Features, and Physiological Features
The Mann-Whitney U test showed that the biofeedback group at post-treatment had improvements in the PHQ-9, GAD-7, ISI, and PSS scores compared with the WLC group. There were significant differences between the biofeedback and WLC groups in ISI changes after 4 weeks (Table 1).
Table 1. Primary and secondary clinical measures of symptom severity at baseline and 4 weeks in the Biofeedback and WLC groups
Measure
|
Group
|
Baseline Week 4 Change
|
Mann–Whitney U test (Change)a
|
Wilcoxon signed rank testb
|
Z3
|
P3
|
Z4
|
P4
|
PHQ-9
|
Biofeedback
|
6
|
4.5
|
-2
|
-1.659
|
0.097
|
-3.317
|
0.001
|
WLC
|
5
|
4.5
|
-1
|
GAD–7
|
Biofeedback
|
4
|
1.5
|
-1.5
|
-1.647
|
0.100
|
-3.385
|
0.001
|
WLC
|
3
|
2
|
0
|
ISI
|
Biofeedback
|
4
|
2
|
-1
|
-2.743
|
0.006
|
-2.491
|
0.013
|
WLC
|
3
|
3
|
0
|
PSS
|
Biofeedback
|
25
|
21
|
-2
|
-1.000
|
0.317
|
-2.909
|
0.004
|
WLC
|
22.5
|
22
|
-1
|
Abbreviations: WLC, waitlist control; PHQ-9, 9-item Patient Health Questionnaire; GAD-7, 7-item Generalized Anxiety Disorder Scale; ISI, Insomnia Severity Index; PSS, Perceived Stress Scale; SD Standard Deviation.
a. Mann–Whitney U test for the difference between Biofeedback and WLC groups;
b. Wilcoxon signed rank test within Biofeedback treatment group (Baseline vs. Week 4).
At baseline and week 4, there were no significant differences in acoustic speech features between the biofeedback intervention and WLC groups. However, the changes in speech before and after 4 weeks in the intervention group were significantly different from those in the WLC group, mainly reflected in the changes in the energy parameters and Mel-Frequency Cepstral Coefficients (MFCC), the details of which are shown in Table 2.
Table 2. Speech acoustic measure at baseline and 4 weeks in the biofeedback and WLC groups
Speech acoustic measure
|
Mann–Whitney U test (Baseline)a
|
Mann–Whitney U test (Week 4)b
|
Mann–Whitney U test (Change)c
|
Z1
|
P1
|
Z2
|
P2
|
Z3
|
P3
|
energy_de_skew
|
-0.559
|
0.576
|
-3.410
|
0.001
|
-2.694
|
0.007
|
spl_de_skew
|
-0.599
|
0.549
|
-3.658
|
<0.001
|
-3.089
|
0.002
|
mfcc_para7_skew
|
-1.081
|
0.280
|
-2.085
|
0.037
|
-2.547
|
0.011
|
mfcc_de_para1_skew
|
-0.559
|
0.576
|
-3.410
|
0.001
|
-2.694
|
0.007
|
mfcc_de_para9_min
|
-1.590
|
0.112
|
-2.078
|
0.038
|
-2.259
|
0.024
|
mfcc_de_para12_std
|
-0.780
|
0.436
|
-2.413
|
0.016
|
-2.580
|
0.01
|
f2_de_kur
|
-0.519
|
0.604
|
-2.634
|
0.008
|
-2.045
|
0.041
|
qingyin_max
|
-1.825
|
0.068
|
-1.199
|
0.231
|
-2.522
|
0.012
|
qingyin_ptp
|
-1.825
|
0.068
|
-1.199
|
0.231
|
-2.522
|
0.012
|
energy_max
|
-1.683
|
0.092
|
-1.235
|
0.217
|
-2.306
|
0.021
|
ste_max
|
-1.429
|
0.153
|
-1.362
|
0.173
|
-2.841
|
0.004
|
ste_ptp
|
-1.416
|
0.157
|
-1.362
|
0.173
|
-2.828
|
0.005
|
ste_mean
|
-1.496
|
0.135
|
-1.482
|
0.138
|
-2.868
|
0.004
|
ste_std
|
-1.503
|
0.133
|
-1.523
|
0.128
|
-2.921
|
0.003
|
ste_de_min
|
-1.402
|
0.161
|
-1.201
|
0.230
|
-2.647
|
0.008
|
ste_de_max
|
-1.402
|
0.161
|
-1.201
|
0.230
|
-2.694
|
0.007
|
ste_de_ptp
|
-1.416
|
0.157
|
-1.268
|
0.205
|
-2.647
|
0.008
|
ste_de_mean
|
-0.980
|
0.327
|
-0.131
|
0.896
|
-2.132
|
0.033
|
ste_de_std
|
-1.402
|
0.161
|
-1.462
|
0.144
|
-2.875
|
0.004
|
ste_de2_min
|
-1.650
|
0.099
|
-1.369
|
0.171
|
-2.841
|
0.004
|
ste_de2_max
|
-1.590
|
0.112
|
-1.215
|
0.224
|
-2.727
|
0.006
|
ste_de2_ptp
|
-1.636
|
0.102
|
-1.362
|
0.173
|
-2.794
|
0.005
|
ste_de2_std
|
-1.529
|
0.126
|
-1.476
|
0.140
|
-3.022
|
0.003
|
spl_max
|
-1.429
|
0.153
|
-1.362
|
0.173
|
-2.219
|
0.027
|
mfcc_para1_max
|
-1.683
|
0.092
|
-1.235
|
0.217
|
-2.306
|
0.021
|
mfcc_para13_mean
|
-0.954
|
0.340
|
-1.516
|
0.130
|
-2.326
|
0.02
|
mfcc_de2_para4_min
|
-1.904
|
0.057
|
-0.566
|
0.572
|
-2.419
|
0.016
|
mfcc_de2_para4_ptp
|
-1.476
|
0.140
|
-0.519
|
0.604
|
-2.165
|
0.03
|
mfcc_de2_para9_kur
|
-0.023
|
0.981
|
-1.917
|
0.055
|
-1.998
|
0.046
|
b2_max
|
-0.144
|
0.886
|
-1.296
|
0.195
|
-1.978
|
0.048
|
shimmer_abs
|
-1.690
|
0.091
|
-1.021
|
0.307
|
-2.567
|
0.01
|
a. Mann–Whitney U test on baseline differences between the biofeedback and WLC groups;
b. Mann–Whitney U test for the difference in the fourth week between the biofeedback and WLC groups;
c. Mann–Whitney U test for the difference between the biofeedback and WLC groups. Abbreviations: WLC, waitlist control; max, maximum; ptp, range.
The changes in physiology before and after 4 weeks in the biofeedback intervention group were significantly different from those in the WLC group, mainly reflected in the changes in the body relaxation index and surface electromyography (EMG). The main changes in the biofeedback intervention group after 4 weeks compared to baseline were changes in body relaxation index, surface EMG, heart rate, standard deviation of beat-to-beat intervals, standard deviation of the difference between adjacent RR intervals, and total energy, the details of which are shown in Table 3.
Table 3. Physiological features measurement at baseline and 4 weeks in the biofeedback and WLC groups
Physiological features measure
|
Mann–Whitney U test (Baseline)a
|
Mann–Whitney U test (Week 4)b
|
Mann–Whitney U test (Change)c
|
Wilcoxon signed rank testd
|
|
Z1
|
P1
|
Z2
|
P2
|
Z3
|
P3
|
Z4
|
P4
|
|
body_data
|
-3.624
|
<0.001
|
-0.181
|
0.856
|
-3.097
|
0.002**
|
-3.414
|
0.001
|
|
|
EMG
|
-3.624
|
<0.001
|
-0.181
|
0.856
|
-3.097
|
0.002**
|
-3.414
|
0.001
|
|
|
HR
|
-0.060
|
0.952
|
-1.450
|
0.147
|
-1.430
|
0.153
|
-3.136
|
0.002
|
|
|
LN(sdnn)
|
-1.801
|
0.072
|
-0.522
|
0.602
|
-1.148
|
0.251
|
-2.327
|
0.02
|
|
|
LN(rmssd)
|
-2.326
|
0.02
|
-0.167
|
0.867
|
-0.870
|
0.384
|
-2.226
|
0.026
|
|
|
LN(pnn50)
|
-2.219
|
0.027
|
-0.689
|
0.491
|
-2.406
|
0.016*
|
-3.890
|
<0.001
|
|
|
LN(tp)
|
-0.532
|
0.595
|
-0.954
|
0.340
|
-1.001
|
0.317
|
-1.978
|
0.048
|
|
Abbreviations. WLC, waitlist control; max, maximum; ptp, range.
a. Mann–Whitney U test on baseline differences between the biofeedback and WLC groups;
b. Mann–Whitney U test for the difference between the biofeedback and WLC groups in the fourth week.
c. Mann–Whitney U test for the difference between the biofeedback and WLC groups after four weeks.
d. Mann–Whitney U test for the difference between the biofeedback and WLC groups on the fourth week;
e. Wilcoxon signed rank test for the difference after intervention in the biofeedback group;
Association between Clinical Symptoms and Acoustic Speech Features
The results of Spearman correlation analysis showed that at the baseline level, the total PHQ-9, GAD-7, and ISS scores of the intervention group were correlated with the energy parameters and MFCC of the speech features. PHQ-9 was correlated with energy parameters (energy_max: R=0.349, P=0.012) and MFCC (mfcc_para1_max: R=0.349, P=0.012). The GAD-7 was correlated with energy parameters (ste_max: R=0.286, P=0.042; ste_ptp: R=0.286, P=0.042; ste_std: R=0.281, P=0.046; ste_de_std: R=0.288, P=0.040; ste_de2_std: R=0.288, P=0.041). ISS was correlated with energy parameters (ste_de2_max: R=0.160, P=0.262; ste_de2_ptp: R=0.122, P=0.392; ste_de2_std: R=0.107, P=0.455) and MFCC (mfcc_para7_skew: R=0.119, P=0.406).
The changes in the energy parameters (energy_max: R=0.280, P=0.046) and MFCC (mfcc_para1_max: R=0.280, P=0.046) were correlated with the improvement in depression before and after the intervention. Changes in the energy parameters (ste_max: R=0.279, P=0.047; ste_ptp: R=0.279, P=0.047; ste_std: R=0.324, P=0.020; ste_de_std: R=0.294, P=0.036; ste_de2_std: R=0.363, P=0.009) were correlated with improvement in anxiety before and after the intervention. The changes in the energy parameters (ste_de2_max: R=0.305, P=0.029; ste_de2_ptp: R=0.329, P=0.018; ste_de2_std: R=0.288, P=0.040) and MFCC (mfcc_para7_skew: R=0.286, P=0.042) were correlated with the improvement in insomnia before and after the intervention.
Association between Clinical Symptoms and Physiological Features
The results of the Spearman correlation analysis showed that at the baseline level, the total PHQ-9 score of the intervention group was correlated with the physical relaxation index (P=0.001), mental relaxation index (P=0.015), surface EMG (P=0.001), and standard deviation of NN intervals (SDNN) (P=0.041). The total GAD-7 score of the intervention group correlated with surface EMG (P=0.016), low frequency (LF) (P=0.021), high frequency (HF) (P=0.019), and LF/HF (P=0.020).
Association between Acoustic Speech Features and Physiologic Features
The results of the Spearman correlation analysis showed the following. The changes in the energy parameters (spl_max) and MFCC (mfcc_de2_para12_ptp) were correlated with the improvement in EMG before and after the intervention. The changes in the energy parameters (energy_de_skew) and MFCC (mfcc_para7_skew) were correlated with the improvement in heart rate before and after the intervention.
The changes in the energy parameters (energy_de_skew) correlated with the improvement in SDNN before and after the intervention. The changes in the energy parameters (energy_de_skew and ste_max) were correlated with the improvement in RMSSD before and after the intervention. The changes in the energy parameters (ste_ptp, ste_std, and ste_de_std) were correlated with the improvement in RMSSD before and after the intervention. The changes in the energy parameters (energy_max) and MFCC (mfcc_para1_max) correlated with the improvement in LF and HF before and after the intervention. The changes in the energy parameters (energy_max, ste_max, and ste_ptp) and MFCC (mfcc_para1_max) were correlated with the improvement in LF/HF before and after the intervention (Table 4).
Table 4. Correlation between acoustic speech features and physiological features in the biofeedback group
Physiological features
|
Speech acoustic
|
Spearman correlation (Change)
|
R value
|
P value
|
emg
|
spl_max
|
0.276
|
0.05
|
mfcc_de2_para12_ptp
|
0.298
|
0.034
|
hr
|
energy_de_skew
|
-0.358
|
0.010
|
mfcc_para7_skew
|
0.298
|
0.034
|
LN(sdnn)
|
energy_de_skew
|
0.425
|
0.002
|
LN(rmssd)
|
energy_de_skew
|
0.347
|
0.013
|
LN(pnn50)
|
ste_max
|
0.345
|
0.013
|
ste_ptp
|
0.345
|
0.013
|
ste_std
|
0.286
|
0.042
|
ste_de_std
|
0.286
|
0.042
|
LN(lf)
|
energy_max
|
-0.323
|
0.021
|
mfcc_para1_max
|
-0.323
|
0.021
|
LN(hf)
|
energy_max
|
0.299
|
0.033
|
mfcc_para1_max
|
-0.323
|
0.021
|
LF/HF
|
energy_max
|
-0.285
|
0.043
|
mfcc_para1_max
|
-0.323
|
0.021
|
ste_max
|
-.0277
|
0.049
|
ste_ptp
|
-.0277
|
0.049
|
Artificial Neural Network Prediction Model by Speech Acoustic Features
Because the LR model only considers linear classification, an ANN-based classification model and a regression model were constructed for verification.
This model, based on ANN binary classification to detect anxiety and insomnia, uses speech feature differences between the responder and non-responder groups as input features. Figure 1 and Figure 2 show the prediction results of this model for 51 participants, in which 0 means “Non-responder,” 1 means “Responder,” the horizontal axis represents the predicted result, and the vertical axis represents the original value of the label. For anxiety, the correct diagnosis rate of this classification model for the responding group was 0.62, indicating a low misdiagnosis rate; the overall accuracy rate was 62% (Figure 1). For insomnia, the correct diagnosis rate of this classification model for the responding group was 0.58, indicating a low misdiagnosis rate, and the overall accuracy rate was 58% (Figure 2). This indicates that the classification model can predict the population that will respond well to treatment with biofeedback.
At the same time, Figures 3 and 4 show the performance of the classifier per fold when the model was subjected to 5-fold cross-validation. From the ROC curve, the area of the ROC curve of the model in each fold was greater than 0.5, which proves that the training of this classifier in each fold was effective.