✋ This model is not published yet.

You can claim this model if you're @locuslab on GitHub. Contact us.

locuslab / convmixer

Demo of ConvMixer as described in the paper "Patches Are All You Need?"

  • Public
  • 39 runs
  • GitHub
  • Paper
  • License
Run with an API

Patches Are All You Need? 🤷

This repository contains an implementation of ConvMixer for the ICLR 2022 submission “Patches Are All You Need?” by Asher Trockman and Zico Kolter.

The tweetable version of ConvMixer, which requires from torch.nn import *:

def ConvMixer(h,d,k,p,n):
 S,C,A=Sequential,Conv2d,lambda x:S(x,GELU(),BatchNorm2d(h))
 R=type('',(S,),{'forward':lambda s,x:s[0](x)+x})
 return S(A(C(3,h,p,p)),*[S(R(A(C(h,h,k,groups=h,padding=k//2))),A(C(h,h,1))) for i in range(d)],AdaptiveAvgPool2d(1),Flatten(),Linear(h,n))

Cite

@article{trockman2022patches,
  title={Patches Are All You Need?},
  author={Trockman, Asher and Kolter, J Zico},
  journal={arXiv preprint arXiv:2201.09792},
  year={2022}
}