TS2Vec 改造为时间序列基础模型：1.6M 参数逼近 SOTA
Turning TS2Vec into a Foundation Model for Time Series Classification
2022 年的老方法 TS2Vec 改三个地方就变成基础模型，1.6M 参数冻住就能打平 MantisV2，还不用注意力，思路很值得看。
TS2Vec 是 2022 年发布的自监督时间序列分类方法，原本需要在每个数据集上单独训练。研究者通过三项改动将其转化为基础模型 TS2Vec-FM：在合成语料上预训练、用双 stem 分别输入原始幅值视图和实例归一化形状视图、再把两个专家编码器蒸馏为单一编码器。TS2Vec-FM 只有 1.6M 参数，采用膨胀卷积结构，冻结后线性探测在 UCR-128 上达到 83.3%，UEA-27 上达到 75.3%，与当前最优的 MantisV2 差距在 0.3 个百分点以内。它不使用注意力机制、patch tokenizer 和中间层提取，仅靠卷积编码器和两层 MLP stem 就接近了 SOTA 水平。
Turning TS2Vec into a Foundation Model for Time Series Classification
TS2Vec is one of the strongest self-supervised methods for time series classification, introduced in 2022. It is designed to be trained anew on every dataset: self-supervised pre-training followed by linear probing. Since then, the field has shifted towards foundation models that reuse one encoder across datasets. Yet TS2Vec offers a compact convolutional architecture that operates directly on temporal samples and accepts variable-length inputs without resampling. We ask whether these properties can be retained when the encoder is pre-trained only once, and show that they can: TS2Vec can essentially be turned into a foundation model. Three changes suffice: pre-train on a synthetic corpus, feed the encoder a raw amplitude view and an instance-normalized shape view of the input through two small stems, and distill one specialist encoder per view into a single encoder. The result, TS2Vec-FM, is a 1.6M-parameter dilated convolutional network that, kept frozen, reaches 83.3% on UCR-128 and 75.3% on UEA-27 under linear probing, within 0.3 pp of MantisV2, the current state of the art, and above TS2Vec trained separately on each target dataset. What we find most interesting is how different TS2Vec-FM is from other foundation models in design: near-parity is reached without attention, without a patch tokenizer, and without intermediate-layer extraction, using an "old-school" convolutional encoder and a two-layer MLP stem.
