稳态视觉诱发电位(SSVEP)识别| Extended Canonical Correlation Analysis, eCCA

Note： 在我有空的时候尽量把SSVEP流行的方法都开源出来，一是方便自己复习和巩固，二是希望广大网友能够指出我思路中存在的一些问题。本人接触这个领域时间较短，有什么纰漏请多指教。

前言

如果要说目前SSVEP识别中最流行的几个方法，那么Extended Canonical Correlation Analysis(eCCA)¹绝对是其中之一。目前来看，如果研究SSVEP算法，那么对比算法基本是TRCA或者eCCA² ³。此外，很多在线BCI系统也是采用该方法进行频率识别。与TRCA这种具有自己数学模型的方法不同，eCCA仅是对CCA的一种计算结构调整。我们知道受试者的template中包含着一些潜在可利用的分类信息（相位等），eCCA在CCA基础上考虑到了受试者template项。

算法

我们知道典型相关分析(CCA)是一种降维和评估两组多维数据相关性的方法，它通过一对典型变量将两组数据变为一维，使得两者之间的相关系数最大。可以参考博客1、2。
CCA应用在SSVEP识别上得到了很好的效果，当已有受试者训练数据的时候，我们能够获得三种多通道信号：测试数据

X (t) \in R^{N_{c} \times N_{s} \times N_{t}} X(t)\in R^{N_c\times N_s\times N_t}

X(t)∈RNc?×Ns?×Nt?，受试者训练数据的平均

{\hat{X}}_{k}^{N_{c} \times N_{s}} \hat X_k^{N_c\times N_s}

X^kNc?×Ns??以及构造的正余弦参考信号

Y_{f_{k}} Y_{f_k}

Yfk??，其中

N_{c} N_c

Nc?，

N_{s} N_s

Ns?，

N_{t} N_t

Nt?分别表示通道数、采样点数和测试集trial数。任意两种信号可以根据CCA计算出空间滤波器。
按照排列组合方式来看，如果我们要根据其中两个信号求空间滤波器，那么会有六种空间滤波器形式。六种空间滤波器会产生10个典型变量，然后每两个典型变量之间又可以计算相关系数，那么就有45个相关系数。这个在Mohammad Hadi MehdizavarehI等人2020年发表在PLOS ONE上的文章⁴有详述。但实际应用过程中，我们通常只取了三种空间滤波器形式（这里以Chen X等人PANS上的文章为例，第三项做了修正）分别为

$W_{X} (X {\hat{X}}_{k}) W_X(X\hat X_k)$
WX?(XX^k?)，
$W_{{\hat{X}}_{k}} (X {\hat{X}}_{k}) W_{\hat X_k}(X\hat X_k)$
WX^k??(XX^k?)：测试数据和template之间；
$W_{X} (X Y_{f_{k}}) W_X(XY_{f_k})$
WX?(XYfk??)：测试数据和参考信号之间；
$W_{{\hat{X}}_{k}} ({\hat{X}}_{k} Y_{f_{k}}) W_{\hat X_k}(\hat X_kY_{f_k})$
WX^k??(X^k?Yfk??)：template和参考信号之间。

由上述空间滤波器，我们可以得到5个相关系数组合（这里的数量在不同的论文中不一致，最初提出eCCA的时候并没有对为什么采用下述相关系数组合做解释。），如下

注：上述公式来自Chen X等人的论文，其中

r_{k} (4) r_k(4)

rk?(4)所对应的空间滤波器形式有误，应该为

W_{{\hat{X}}_{k}} ({\hat{X}}_{k} Y_{f_{k}}) W_{\hat X_k}(\hat X_kY_{f_k})

WX^k??(X^k?Yfk??)。如果仅考虑第一项，那么算法变为标准的CCA；如果对相关系数进行融合，便可以得到k-th刺激下的相关系数。最后通过确认最大相关系数对应的刺激便可以实现分类

整个算法的计算结构为

Matlab代码

CCA

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

function [output1, output2] = CCA(signal1, signal2)
% Canonical Correlation Analysis, CCA
% Input:
% signal: #channels, #points
% Output:
% output1: spatial filter of signal1
% output2: spatial filter of signal2
%
X=signal1;
Y=signal2;
T=size(signal2, 2);
%compute covariance matrix
meanx=mean(X,2);
meany=mean(Y,2);
s11=0;s22=0;s12=0;s21=0;
for i1=1:T
s11=s11+(X(:,i1)-meanx)*(X(:,i1)-meanx)';
s22=s22+(Y(:,i1)-meany)*(Y(:,i1)-meany)';
s12=s12+(X(:,i1)-meanx)*(Y(:,i1)-meany)';
s21=s21+(Y(:,i1)-meany)*(X(:,i1)-meanx)';
end
s11=s11/(T-1);
s22=s22/(T-1);
s12=s12/(T-1);
s21=s21/(T-1);
%compute eigvalue and eigvector
[eigvectora,eigvaluea]=eig(inv(s11)*s12*inv(s22)*s21);
[eigvectorb,eigvalueb]=eig(inv(s22)*s21*inv(s11)*s12);

evaluea= diag(eigvaluea);
evalueb = diag(eigvalueb);
% correlation coefficient & canonical variates of signal1
[corrcoef1, index]= max(sqrt(evaluea));
output1 = eigvectora(:, index);
% correlation coefficient & canonical variates of signal2
[corrcoef2, index]= max(sqrt(evalueb));
output2 = eigvectorb(:, index);
end

对上述程序做简要解释

上述CCA求解采用的是特征值分解的方法，另外还可以参考SVD分解的做法；
通过CCA的推导过程可以看到，corrcoef1和corrcoef2相等，表示两个输入多通道信号的相关系数。在SSVEP中，标准CCA可采用该值直接输出；
这个函数文件输出的是两个多通道信号的典型变量，即空间滤波器。

eCCA

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109

% -------------------------------------------------------------------------
% Main for Extended Canonical Correlation Analysis[1]
%
% Dataset (Sx.mat):
% A 40-target SSVEP dataset recorded from a single subject. The stimuli
% were generated by the j oint frequency-phase modulation (JFPM)
% - Stimulus frequencies : 8.0 - 15.8 Hz with an interval of 0.2 Hz
% - Stimulus phases : 0pi, 0.5pi, 1.0pi, and 1.5pi
% - # of channels : Oz
% - # of recording blocks : 6
% - Data length of epochs : 1.5 [seconds]
% - Sampling rate : 250 [Hz]
% - Data format : # channels, # points, # targets, # blocks
%
% See also:
% CCA.m
%
% Reference:
% [1] Chen X, Wang Y, Nakanishi M, Gao X, Jung TP, Gao S (2015)
% High-speed spelling with a noninvasive brain-computer interface.
% PNAS 112:E6058-6067.
% -------------------------------------------------------------------------

clear all
close all

load ('Freq_Phase.mat')
load('subject1.mat')
eeg = subject1;
[N_channel, N_point, N_target, N_block] = size(eeg);
% sample rate
fs = 250;
t=1/fs:1/fs:N_point/fs;
%% ------------classification-------------
tic
% LOO cross-validation
for loocv_i = 1:N_block
Testdata = eeg(:, :, :, loocv_i);
Traindata = eeg;
Traindata(:, :, :, loocv_i) = [];
% number of harmonics
N_harmonic = 2;
for targ_i = 1:N_target
% Template
Template(:, :, targ_i) = mean(squeeze(Traindata(:,:,targ_i,:)),3);
% Reference
Y=[];
for har_i=1:N_harmonic
Y=cat(1,Y,cat(1, sin(2*pi*freqs(targ_i)*har_i*t), ...
cos(2*pi*freqs(targ_i)*har_i*t)));
end
Reference(:, :, targ_i) = Y;
end

% labels assignment according to testdata
truelabels=freqs;

N_testTrial=size(Testdata, 3);
for trial_i=1:N_testTrial
Allcoefficience = [];
for targ_j=1:length(freqs)
% ρ1 (Filter: Test data & Reference)
[wn1, wn2]= CCA(Testdata(:,:,trial_i), Reference(:, :, targ_j));
weighted_train = wn2'*Reference(:,:,targ_j);
weighted_test = wn1'*Testdata(:,:,trial_i);
coefficienceMatrix = corrcoef(weighted_test,weighted_train);
coefficience(1) = abs(coefficienceMatrix(1,2));
% ρ2 (Filter: Test data & Template)
[wn, ~] = CCA(Testdata(:,:,trial_i), Template(:, :, targ_j));
weighted_train = wn'*Template(:,:,targ_j);
weighted_test = wn'*Testdata(:,:,trial_i);
coefficienceMatrix = corrcoef(weighted_test,weighted_train);
coefficience(2) = coefficienceMatrix(1,2);
% ρ3 (Filter: Test data & Reference)
[wn, ~] = CCA(Testdata(:,:,trial_i), Reference(:, :, targ_j));
weighted_train = wn'*Template(:,:,targ_j);
weighted_test = wn'*Testdata(:,:,trial_i);
coefficienceMatrix = corrcoef(weighted_test,weighted_train);
coefficience(3) = coefficienceMatrix(1,2);
% ρ4 (Filter: Template & Reference)
[wn, ~] = CCA(Template(:, :, targ_j), Reference(:, :, targ_j));
weighted_train = wn'*Template(:,:,targ_j);
weighted_test = wn'*Testdata(:,:,trial_i);
coefficienceMatrix = corrcoef(weighted_test,weighted_train);
coefficience(4) = coefficienceMatrix(1,2);
% % ρ5 (Filter: Test data & Template)
% [wn1, wn2] = CCA(Testdata(:,:,trial_i), Template(:, :, targ_j));
% weighted_train = wn2'*Template(:,:,targ_j);
% weighted_test = wn1'*Template(:,:,targ_j);
% coefficienceMatrix = corrcoef(weighted_test,weighted_train);
% coefficience(5) = coefficienceMatrix(1,2);
% compute correlation values
Allcoefficience(targ_j) = sum(sign(coefficience).*coefficience.^2);
end % end targ_i
% target detection
[~, index] = max(Allcoefficience);
outputlabels(trial_i) = freqs(index);

end % end trial_i
trueNum = sum((outputlabels-truelabels)==0);
acc(loocv_i) = trueNum/length(truelabels);
fprintf('The %d-th CV accuracy is: %.4f, samples: %d/%d\n',loocv_i,...
acc(loocv_i),trueNum, N_testTrial)
end % end looCv_i
t=toc;
% data visualization
fprintf('\n-----------------------------------------\n')
disp(['total time: ',num2str(t),' s']);
fprintf('6-fold CV average accuracy is: %.4f\n',mean(acc))

对上述程序做简要解释

采用的数据为清华benchmark dataset 受试者1，数据已经经过滤波等预处理，数据长度为1.5s；
实验中发现第五项的加入使得正确率降低，故只采用了四项的融合，和万峰老师论文保持一致² ；
完整代码见我的仓库。

前四项相关系数融合运行结果为

1
2
3
4
5
6
7
8
9
10

The 1-th CV accuracy is: 0.9250, samples: 37/40
The 2-th CV accuracy is: 0.9750, samples: 39/40
The 3-th CV accuracy is: 0.9750, samples: 39/40
The 4-th CV accuracy is: 0.9750, samples: 39/40
The 5-th CV accuracy is: 0.9750, samples: 39/40
The 6-th CV accuracy is: 0.9750, samples: 39/40

-----------------------------------------
total time: 175.7503 s
6-fold CV average accuracy is: 0.9667

五项相关系数融合运行结果为

1
2
3
4
5
6
7
8
9
10

The 1-th CV accuracy is: 0.5750, samples: 23/40
The 2-th CV accuracy is: 0.5750, samples: 23/40
The 3-th CV accuracy is: 0.7750, samples: 31/40
The 4-th CV accuracy is: 0.4750, samples: 19/40
The 5-th CV accuracy is: 0.6000, samples: 24/40
The 6-th CV accuracy is: 0.6500, samples: 26/40

-----------------------------------------
total time: 216.9206 s
6-fold CV average accuracy is: 0.6083

关于取哪些相关系数进行融合的问题，本文参考了四篇较新的文献，其中采用5个相关系数融合是常用的选择（都是清华的文章…）。但是在本文实验后发现，第五项加入使正确率下降了…所以采用了万峰老师的做法，只取了前四项，如果有问题希望各位能及时提出来。

References

Chen X, Wang Y, Nakanishi M, Gao X, Jung TP, Gao S (2015) High-speed spelling with a noninvasive brain-computer interface. PNAS 112:E6058-6067.
Wong CM, Wan F, Wang B, Wang Z, Nan W, Lao KF, Mak PU, Vai MI, Rosa A (2020) Learning across multi-stimulus enhances target recognition methods in SSVEP-based BCIs. Journal of neural engineering 17:016026.
Xu M, Han J, Wang Y, Jung TP, Ming D (2020) Implementing over 100 command codes for a high-speed hybrid brain-computer interface using concurrent P300 and SSVEP features. IEEE Transactions on Biomedical Engineering.
Mehdizavareh MH, Hemati S, Soltanian-Zadeh H (2020) Enhancing performance of subject-specific models via subject-independent information for SSVEP-based BCIs. PloS one 15:e0226048.