论文标题:Swin Transformer: Hierarchical Vision Transformer using ShiftedWindows
swin transformer的主要有特点有三个:
- 第一,把图像划分为一个个窗口,只在窗口内部计算self-attention。这样带来的优势是,self-attention的计算复杂度只与图像尺寸呈线性 系,而非平方关系。(Swin Transformer builds hierarchical feature maps by merging image patches in deeper layers and has linear computation complexity to input image size due to computation of self-attention only within each local window.)
- 第二,后面layer的patch会合并前面layer的patch,所以越深的layer,它的patch size越大,视野越大,从而构建出hierarchical feature maps。(Swin Transformer constructs a hierarchical representation by starting from small-sized patches (outlined in gray) and gradually merging neighboring patches in deeper Transformer layers.)
- 第三个特点是shifted window,就是前后两层的window划分之间有偏移。每一个swin transformer block都包含两层,第一层是W-MSA (window multi-head self-attention),第二层是SW-MSA (shifted window multi-head self-attention)。前后层这种shifted window分别为对方的被拆开的window带来了联结。(The shifted windows bridge the windows of the preceding layer, providing connections among them that significantly enhance modeling power)
论文讲解资料: