{"id":565,"date":"2022-11-05T15:26:57","date_gmt":"2022-11-05T15:26:57","guid":{"rendered":"https:\/\/blog.liguanxin.cn\/?p=565"},"modified":"2022-11-12T07:30:09","modified_gmt":"2022-11-12T07:30:09","slug":"%e8%ae%ba%e6%96%87%e7%ac%94%e8%ae%b0-cvpr-2022-oralrestormer-efficient-transformer-for-high-resolution-image-restoration","status":"publish","type":"post","link":"https:\/\/blog.liguanxin.cn\/index.php\/2022\/11\/05\/%e8%ae%ba%e6%96%87%e7%ac%94%e8%ae%b0-cvpr-2022-oralrestormer-efficient-transformer-for-high-resolution-image-restoration\/","title":{"rendered":"\u8bba\u6587\u7b14\u8bb0\u2014\u2014[CVPR 2022 Oral]Restormer: Efficient Transformer for High-Resolution Image Restoration"},"content":{"rendered":"<p><strong>\u521b\u65b0\u70b9\uff1a<br \/>\n\u2460\u89e3\u51b3\u9ad8\u5206\u8fa8\u7387\u56fe\u50cf\u7684\u56fe\u50cf\u6062\u590d\u4efb\u52a1\u4e2dtransformer\u590d\u6742\u5ea6\u9ad8\u7684\u95ee\u9898<br \/>\n\u2461\u5728\u56fe\u50cf\u8fd0\u52a8\u53bb\u6a21\u7cca\uff0c\u53bb\u7126\u53bb\u6a21\u7cca\uff0c\u56fe\u50cf\u53bb\u566a\uff08\u9ad8\u65af\u7070\u5ea6\/\u989c\u8272\u53bb\u566a\uff0c\u548c\u771f\u5b9e\u56fe\u50cf\u53bb\u566a\uff09\u4efb\u52a1\u4e2d\u53d6\u5f97sota<br \/>\n\u2462self-attention\u4e2d\u91c7\u7528\u901a\u9053\u505a\u4e58\u6cd5<br \/>\n\u2463\u524d\u9988\u7f51\u7edc\u7684\u521b\u65b0\uff0c\u91c7\u7528\u95e8\u63a7\u7f51\u7edc<br \/>\n\u2464\u7f51\u7edc\u65e9\u671f\u7528\u5c0fpatch\u5927batch\uff0c\u9010\u6e10\u5230\u540e\u671f\u5927patch\u5c0fbatch\uff0c\u5f88\u597d\u5730\u5e2e\u52a9\u7f51\u7edc\u4ece\u5927\u56fe\u50cf\u4e2d\u5b66\u4e60\u4e0a\u4e0b\u6587\uff0c\u540e\u671f\u80fd\u63d0\u9ad8\u6027\u80fd<\/strong><\/p>\n<h2>\u7f51\u7edc\u7ed3\u6784<\/h2>\n<p><img src=\"https:\/\/blog.liguanxin.cn\/wp-content\/uploads\/2022\/11\/\u7f51\u7edc.png\" alt=\"\" \/><\/p>\n<ul>\n<li>\u603b\u4f53\u91c7\u7528UNet\u7ed3\u6784<\/li>\n<\/ul>\n<h2>\u591aDW\u5377\u79ef\u5934\u8f6c\u7f6e\u6ce8\u610f\u529b\uff08Multi-Dconv Head Transposed Attention\uff09<\/h2>\n<p><img src=\"https:\/\/blog.liguanxin.cn\/wp-content\/uploads\/2022\/11\/\u56fe1.png\" alt=\"\" \/><br \/>\n\u7279\u70b9\uff1a\u901a\u8fc7\u53ea\u5bf9\u901a\u9053\u505a\u4e58\u6cd5\uff0c\u5927\u5927\u51cf\u5c11\u4e86\u8ba1\u7b97\u91cf\u3002<br \/>\n\u5bf9\u4e8e\u8f93\u5165X\u9996\u5148\u7ecf\u8fc7layer norm\uff0c\u7136\u540e\u590d\u5236\u6210\u4e09\u4efd\u4f5c\u4e3aQKV\uff0c\u7ecf\u8fc71<em>1\u5377\u79ef\u589e\u5f3a\u901a\u9053\u4e0a\u4e0b\u6587\uff0c\u7136\u540e\u7ecf\u8fc73<\/em>3\u7684DW\u5377\u79ef\u589e\u5f3a\u50cf\u7d20\u95f4\u7684\u4e0a\u4e0b\u6587\uff0c\u6700\u540ereshape\u6210\u8bcd\u5411\u91cf\u7684\u5f62\u5f0f\uff0cQK\u76f8\u4e58\u5f97\u5230c<em>c\u7684\u5b50\u6ce8\u610f\u529b\u77e9\u9635\uff0csoftmax\u540e\u5728\u4e0eV\u76f8\u4e58\uff0c\u518d\u7ecf\u8fc71<\/em>1\u5377\u79ef\u52a0\u4e0a\u539f\u672c\u7684X\u5f97\u5230\u8f93\u51fa\u3002<br \/>\n<img src=\"https:\/\/blog.liguanxin.cn\/wp-content\/uploads\/2022\/11\/\u5fae\u4fe1\u622a\u56fe_20221105224527.png\" alt=\"\" \/><\/p>\n<h2>\u95e8\u63a7DW\u5377\u79ef\u524d\u9988\u7f51\u7edc\uff08Gated-Dconv Feed-Forward Network\uff09<\/h2>\n<p><img src=\"https:\/\/blog.liguanxin.cn\/wp-content\/uploads\/2022\/11\/\u56fe2.png\" alt=\"\" \/><br \/>\n\u7c7b\u4f3c\u4e8e\u539f\u59cb\u7684CA\u6ce8\u610f\u529b\uff0c\u4e0b\u65b9\u7684\u901a\u9053\u88ab\u62d3\u5c55\u5230\u03b3\u500d\uff08\u901a\u5e38\u662f4\u500d\uff09\uff0c\u901a\u8fc7GELU\u975e\u7ebf\u6027\u540e\u4e0e\u4e0a\u65b9\u5206\u652f\u70b9\u4e58\u3002<br \/>\n<img src=\"https:\/\/blog.liguanxin.cn\/wp-content\/uploads\/2022\/11\/\u5fae\u4fe1\u622a\u56fe_20221105225503.png\" alt=\"\" \/><\/p>\n<h2>\u6e10\u8fdb\u5f0f\u5b66\u4e60<\/h2>\n<p>\u57fa\u4e8eCNN\u7684\u7f51\u7edc\u901a\u5e38\u4f1a\u7528\u56fa\u5b9apatches\u7684\u7f51\u7edc\u6765\u8bad\u7ec3\u3002\u4f46\u662f\u7528\u5c0fpatches\u8bad\u7ec3transformer\u7f51\u7edc\u7684\u8bdd\uff0c\u4f1a\u5bfc\u81f4\u65e0\u6cd5\u6709\u6548\u7f16\u7801\u5168\u5c40\u4fe1\u606f\uff0c\u4ece\u800c\u5bfc\u81f4\u8868\u73b0\u4e0d\u662f\u6700\u4f18\u3002<br \/>\n\u4e8e\u662f\u672c\u7f51\u7edc\u5728\u65e9\u671f\u5728\u8f83\u5c0f\u7684patches\u4e0a\u8bad\u7ec3\uff0c\u5728\u540e\u671f\u5728\u66f4\u5927\u7684patches\u4e0a\u8bad\u7ec3\u3002<br \/>\n\u901a\u8fc7\u6e10\u8fdb\u5b66\u4e60\u5728\u6df7\u5408\u5c3a\u5bf8\u7684patches\u4e0a\u8bad\u7ec3\u7684\u6a21\u578b\u5728\u6d4b\u8bd5\u65f6\u663e\u793a\u51fa\u66f4\u597d\u7684\u8868\u73b0\u3002<br \/>\n\u7531\u4e8e\u5728\u5927patches\u4e0a\u7684\u8bad\u7ec3\u7684\u65f6\u95f4\u66f4\u957f\uff0c\u6211\u4eec\u968f\u7740patch size\u589e\u52a0\u800c\u51cf\u5c11batch size\uff0c\u4ee5\u6bcf\u4e2astep\u7684\u4f18\u5316\u65f6\u95f4\u76f8\u540c\u3002<\/p>\n<h2>\u5b9e\u9a8c\u7ed3\u679c<\/h2>\n<p><img src=\"https:\/\/blog.liguanxin.cn\/wp-content\/uploads\/2022\/11\/\u5fae\u4fe1\u622a\u56fe_20221105230159.png\" alt=\"\" \/><\/p>\n<p><strong>\u5728\u53bb\u96e8\u4efb\u52a1\u4e2dpsnr\u6bd4\u5176\u4ed6\u7f51\u7edc\u5e73\u5747\u9ad8\u4e861dB<\/strong><\/p>\n<p><img src=\"https:\/\/blog.liguanxin.cn\/wp-content\/uploads\/2022\/11\/\u5fae\u4fe1\u622a\u56fe_20221105230424.png\" alt=\"\" \/><\/p>\n<p><strong>\u53bb\u6a21\u7cca\u4efb\u52a1\u4e2d\u4e5f\u8fbe\u5230\u4e86sota<\/strong><\/p>\n<p><img src=\"https:\/\/blog.liguanxin.cn\/wp-content\/uploads\/2022\/11\/\u5fae\u4fe1\u622a\u56fe_20221105230626.png\" alt=\"\" \/><\/p>\n<p><strong>\u503c\u5f97\u6ce8\u610f\u7684\u662f\uff0c\u5728\u53bb\u566a\u4efb\u52a1\u4e2d\uff0c\u672c\u6587\u65b9\u6cd5\u57fa\u672c\u4e0a\u4e0eSwinIR\u6301\u5e73<\/strong><\/p>\n<h2>\u8ba1\u7b97\u91cf<\/h2>\n<p><img src=\"https:\/\/blog.liguanxin.cn\/wp-content\/uploads\/2022\/11\/\u5fae\u4fe1\u622a\u56fe_20221105231017.png\" alt=\"\" \/><\/p>\n<h2>CODE<\/h2>\n<p>\u591aDW\u5377\u79ef\u5934\u8f6c\u7f6e\u6ce8\u610f\u529b\uff08Multi-Dconv Head Transposed Attention\uff09<\/p>\n<pre><code class=\"language-python\">class Attention(nn.Module):\n    def __init__(self, dim, num_heads, bias):\n        super(Attention, self).__init__()\n        self.num_heads = num_heads\n        self.temperature = nn.Parameter(torch.ones(num_heads, 1, 1))\n\n        self.qkv = nn.Conv2d(dim, dim*3, kernel_size=1, bias=bias)\n        self.qkv_dwconv = nn.Conv2d(dim*3, dim*3, kernel_size=3, stride=1, padding=1, groups=dim*3, bias=bias)\n        self.project_out = nn.Conv2d(dim, dim, kernel_size=1, bias=bias)\n\n    def forward(self, x):\n        b,c,h,w = x.shape\n\n        qkv = self.qkv_dwconv(self.qkv(x))\n        q,k,v = qkv.chunk(3, dim=1)   \n\n        q = rearrange(q, &#039;b (head c) h w -&gt; b head c (h w)&#039;, head=self.num_heads)\n        k = rearrange(k, &#039;b (head c) h w -&gt; b head c (h w)&#039;, head=self.num_heads)\n        v = rearrange(v, &#039;b (head c) h w -&gt; b head c (h w)&#039;, head=self.num_heads)\n\n        q = torch.nn.functional.normalize(q, dim=-1)\n        k = torch.nn.functional.normalize(k, dim=-1)\n\n        attn = (q @ k.transpose(-2, -1)) * self.temperature\n        attn = attn.softmax(dim=-1)\n\n        out = (attn @ v)\n\n        out = rearrange(out, &#039;b head c (h w) -&gt; b (head c) h w&#039;, head=self.num_heads, h=h, w=w)\n\n        out = self.project_out(out)\n        return out<\/code><\/pre>\n<p>\u95e8\u63a7DW\u5377\u79ef\u524d\u9988\u7f51\u7edc\uff08Gated-Dconv Feed-Forward Network\uff09<\/p>\n<pre><code class=\"language-python\">class FeedForward(nn.Module):\n    def __init__(self, dim, ffn_expansion_factor, bias):\n        super(FeedForward, self).__init__()\n\n        hidden_features = int(dim*ffn_expansion_factor)\n\n        self.project_in = nn.Conv2d(dim, hidden_features*2, kernel_size=1, bias=bias)\n\n        self.dwconv = nn.Conv2d(hidden_features*2, hidden_features*2, kernel_size=3, stride=1, padding=1, groups=hidden_features*2, bias=bias)\n\n        self.project_out = nn.Conv2d(hidden_features, dim, kernel_size=1, bias=bias)\n\n    def forward(self, x):\n        x = self.project_in(x)\n        x1, x2 = self.dwconv(x).chunk(2, dim=1)\n        x = F.gelu(x1) * x2\n        x = self.project_out(x)\n        return x<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>\u521b\u65b0\u70b9\uff1a \u2460\u89e3\u51b3\u9ad8\u5206\u8fa8\u7387\u56fe\u50cf\u7684\u56fe\u50cf\u6062\u590d\u4efb\u52a1\u4e2dtransformer\u590d\u6742\u5ea6\u9ad8\u7684\u95ee\u9898 \u2461\u5728\u56fe\u50cf\u8fd0\u52a8\u53bb\u6a21\u7cca\uff0c\u53bb\u7126\u53bb\u6a21 [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[6],"tags":[13,19,28,11],"_links":{"self":[{"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/posts\/565"}],"collection":[{"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/comments?post=565"}],"version-history":[{"count":0,"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/posts\/565\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/media?parent=565"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/categories?post=565"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.liguanxin.cn\/index.php\/wp-json\/wp\/v2\/tags?post=565"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}