自 2020 年底视觉 Transformer（Vision Transformer, ViT）问世以来，它几乎重塑了整个计算机视觉的编码范式。然而，一个有趣的现象是，虽然大语言模型（LLM）领域的架构演进如火如荼，从 LLaMA 到 Qwen 再到 Gemma，各种新组件层出不穷，但视觉骨干网络的设计却似乎陷入了某种“停滞”。即便是一些最先进的视觉模型，其核心依然守着五年前的原始设计。

研究团队发现，这个看似简单的指标实际上与模型的学习能力密切相关。当一个组件具有高可塑性时，它能够产生较大的梯度值，从而在反向传播过程中推动更显著的参数更新。这就像一个敏感的温度计能够快速响应环境温度的细微变化一样，高可塑性组件能够敏锐地捕捉到数据中的细微模式，并迅速调整自身的行为来适应新的任务需求。

实验表明，能够「早退」的样本准确率高达 83.33%，而需要完整迭代的困难样本准确率为 45.80%。这与人类的认知资源分配策略惊人地一致——简单问题快速解决，复杂问题投入更多时间。

It’s 2023 and transformers are having a moment. No, I’m not talking about the latest installment of the Transformers movie franchise, “Transformers: Rise of the Beasts”; I’m talking about the deep ...

In the last decade, convolutional neural networks (CNNs) have been the go-to architecture in computer vision, owing to their powerful capability in learning representations from images/videos.

Want smarter insights in your inbox? Sign up for our weekly newsletters to get only what matters to enterprise AI, data, and security leaders. Subscribe Now Transformer-based large language models ...

Computer vision continues to be one of the most dynamic and impactful fields in artificial intelligence. Thanks to breakthroughs in deep learning, architecture design and data efficiency, machines are ...

Transformers, first proposed in a Google research paper in 2017, were initially designed for natural language processing (NLP) tasks. Recently, researchers applied transformers to vision applications ...