Commit Graph

20 Commits (a25bf974a9a2c67c72fbb780c056c8638c966a83)

Author SHA1 Message Date
Ross Wightman 87bfb055c0 Merge remote-tracking branch 'origin/master' into bits_and_tpu
2 years ago
Ross Wightman 48e1df8b37 Add norm/norm_act header comments
2 years ago
Ross Wightman f07dfe010a Merge remote-tracking branch 'origin/more_vit' into bits_and_tpu
2 years ago
Ross Wightman 8c9696c9df More model and test fixes
2 years ago
Ross Wightman 43aa84e861 Add 'fast' layer norm that doesn't cast to float32, support APEX LN impl for slight speed gain, update norm and act factories, tweak SE for ability to disable bias (needed by GCVit)
2 years ago
Ross Wightman 1186fc9c73 Merge remote-tracking branch 'origin/master' into bits_and_tpu
2 years ago
Ross Wightman e27c16b8a0 Remove unecessary code for synbn guard
2 years ago
Ross Wightman 879df47c0a Support BatchNormAct2d for sync-bn use. Fix #1254
2 years ago
Ross Wightman 1ba0ec4c18 Merge remote-tracking branch 'origin/master' into bits_and_tpu
2 years ago
Ross Wightman f670d98cb8 Make a few more layers symbolically traceable (remove from FX leaf modules)
2 years ago
Ross Wightman d829858550 Significant norm update
2 years ago
Ross Wightman ab49d275de Significant norm update
2 years ago
Ross Wightman 7bbbd5ef1b EvoNorm and GroupNormAct options for debugging TPU / XLA concerns
2 years ago
Ross Wightman af607b75cc Prep a set of ResNetV2 models with GroupNorm, EvoNormB0, EvoNormS0 for BN free model experiments on TPU and IPU
3 years ago
Ross Wightman 9811e229f7 Fix regression in models with 1001 class pretrained weights. Improve batchnorm arg and BatchNormAct layer handling in several models.
3 years ago
Ross Wightman 231d04e91a ResNetV2 pre-act and non-preact model, w/ BiT pretrained weights and support for ViT R50 model. Tweaks for in21k num_classes passing. More to do... tests failing.
3 years ago
Ross Wightman 3aebc2f06c Switch DPN to use BnAct layer, train a new DPN 68b model with RA to 79.21
4 years ago
Ross Wightman f28170df3f Fix an an untested change, remove a debug print
4 years ago
Ross Wightman eb7653614f Monster commit, activation refactor, VoVNet, norm_act improvements, more
4 years ago
Ross Wightman 14edacdf9a DenseNet converted to support ABN (norm + act) modules. Experimenting with EvoNorm, IABN
4 years ago