{"id":355,"date":"2022-03-27T05:12:51","date_gmt":"2022-03-27T05:12:51","guid":{"rendered":"https:\/\/blog.liguanxin.cn\/?p=355"},"modified":"2022-03-27T05:12:51","modified_gmt":"2022-03-27T05:12:51","slug":"%e8%ae%ba%e6%96%87%e7%ac%94%e8%ae%b0-cmt-convolutional-neural-networks-meet-vision-transformers","status":"publish","type":"post","link":"https:\/\/blog.liguanxin.cn\/index.php\/2022\/03\/27\/%e8%ae%ba%e6%96%87%e7%ac%94%e8%ae%b0-cmt-convolutional-neural-networks-meet-vision-transformers\/","title":{"rendered":"\u8bba\u6587\u7b14\u8bb0\u2014\u2014CMT: Convolutional Neural Networks Meet Vision Transformers"},"content":{"rendered":"

\u521b\u65b0\u70b9\uff1a
\n\u2460\u63d0\u5347\u6027\u80fd\uff0c\u76f8\u6bd4ViT-S\u53c2\u6570\u66f4\u5c11\u7684\u60c5\u51b5\u4e0b\u7cbe\u5ea6\u66f4\u9ad8
\n\u2461transformer\u548cCNN\u7684\u6df7\u5408\u7f51\u7edc\uff0c\u5229\u7528transformer\u6765\u6355\u83b7\u8fdc\u7a0b\u7279\u5f81\u5173\u7cfb\uff0cCNN\u6355\u83b7\u5c40\u90e8\u7279\u5f81\u5173\u7cfb<\/strong><\/p>\n

CMT (CNNs meet transformers)<\/p>\n

\u6574\u4f53\u7ed3\u6784<\/h1>\n

\"\"<\/p>\n

\u56fe(a)\u662fResNet-50\u6a21\u578b<\/p>\n

\u56fe(b)\u662fViT-S\u7ed3\u6784\uff0cMHSA\u4ee3\u8868\u591a\u5934\u81ea\u6ce8\u610f\u529b\u673a\u5236\uff0c\u4f46\u662f\u8f93\u5165\u56fe\u50cf\u53ea\u80fd\u7ebf\u6027\u6295\u5f71\u5230token\uff0c\u4e3a\u4e86\u89e3\u51b3\u8fd9\u4e00\u9650\u5236\uff0c\u63d0\u51fa\u4e86\u56fe(c)\u7684stem\u7ed3\u6784<\/p>\n

\u56fe(c)\u7684stem\u7ed3\u6784\u91c7\u7528\u6b65\u957f\u4e3a2\u76843*3<\/span>\u5377\u79ef\u8f93\u51fachannel\u4e3a32\uff0c\u7136\u540e\u518d\u8ddf\u4e24\u4e2a\u6b65\u957f\u4e3a1\u76843*3<\/span>\u5377\u79ef\u6765\u66f4\u597d\u7684\u63d0\u53d6\u5c40\u90e8\u4fe1\u606f\u3002
\n\u6bcf\u4e00\u4e2aStage\u524d\u9762\u6709\u4e00\u4e2a\u6b65\u957f\u4e3a2\u76842*2<\/span>\u5377\u79ef\u8ba9\u5206\u8fa8\u7387\u51cf\u5c114\u500d\uff0cchannel\u589e\u52a02\u500d\u3002
\n\u91c7\u7528\u7684CMT\u5757\u65e2\u53ef\u4ee5\u6355\u83b7\u5c40\u90e8\u7279\u5f81\uff0c\u4e5f\u80fd\u6355\u83b7\u8fdc\u7a0b\u7279\u5f81\u4f9d\u8d56\u3002
\n\u6700\u540e\u91c7\u7528\u4e00\u4e2a\u5e73\u5747\u6c60\u5316\u5c42\u548c\u4e00\u4e2aFC\u5c42\u6295\u5f71\u52301000\u4e2a\u5206\u7c7b\uff0c\u7136\u540e\u7ecf\u8fc7\u4e00\u4e2asoftmax\u6765\u8f93\u51fa\u3002<\/p>\n

CMT Block<\/h1>\n

\u628a\u7edd\u5bf9\u4f4d\u7f6e\u7f16\u7801\u6539\u6210\u5c40\u90e8\u611f\u77e5\u5355\u5143(LPU)<\/strong>
\n\"\"
\nDWConv\u4ee3\u8868depth-wise convolution
\n\"\"<\/p>\n

\u8f7b\u91cf\u7ea7\u7684\u591a\u5934\u81ea\u6ce8\u610f\u529b\u673a\u5236<\/strong>
\n\"\"
\n\u6240\u8c13\u8f7b\u91cf\u7ea7\u5c31\u662f\u5bf9H*W*C<\/span>\u8f6c\u5316\u4e3aQ\uff0cK\u4e4b\u524d\u52a0\u4e86\u4e00\u4e2adepth-wise\u5377\u79ef\u4f7f\u5c3a\u5bf8\u53d8\u6210H\/k*W\/k*C<\/span>\uff0c\u518d\u8f6c\u6362\u4e3atransformer\u7684n\/k^2*d<\/span>\uff0c\u5176\u4e2dn\/k^2=H\/k*W\/k<\/span>\uff0cd=C<\/span>\uff0c\u7136\u540e\u628ad\u5212\u5206\u4e3ah\u4e2a\uff0c\u505ah\u4e2a\u5934\u7684\u81ea\u6ce8\u610f\u529b\u3002\"\"<\/p>\n

\u53cd\u5411\u5269\u4f59\u524d\u9988\u7f51\u7edc(Inverted Residual Feed-forward Network)<\/strong>
\n\"\"
\n\u7b2c\u4e00\u4e2a1*1<\/span>\u5377\u79ef\u628a\u7ef4\u5ea6\u6269\u59274\u500d\uff0c\u7136\u540e\u7ecf\u8fc7\u4e00\u4e2a\u5e26\u6b8b\u5dee\u7684DW\u5377\u79ef\u5c42\uff0c\u7136\u540e\u6700\u540e\u4e00\u4e2a1*1<\/span>\u5377\u79ef\u518d\u628a\u7ef4\u5ea6\u51cf\u5c114\u500d\u3002
\n\"\"<\/p>\n","protected":false},"excerpt":{"rendered":"

\u521b\u65b0\u70b9\uff1a \u2460\u63d0\u5347\u6027\u80fd\uff0c\u76f8\u6bd4ViT-S\u53c2\u6570\u66f4\u5c11\u7684\u60c5\u51b5\u4e0b\u7cbe\u5ea6\u66f4\u9ad8 \u2461transformer\u548cCNN\u7684\u6df7\u5408\u7f51\u7edc\uff0c\u5229\u7528 […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[6],"tags":[14,13,19,17,11],"_links":{"self":[{"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/posts\/355"}],"collection":[{"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/comments?post=355"}],"version-history":[{"count":0,"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/posts\/355\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/media?parent=355"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/categories?post=355"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/tags?post=355"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}