14 Commits (16d2db7e4b85b1574cc03694d9c12561d618a0f9)

Author	SHA1	Message	Date
Ross Wightman	48e1df8b37	Add norm/norm_act header comments	2 years ago
Ross Wightman	8c9696c9df	More model and test fixes	2 years ago
Ross Wightman	43aa84e861	Add 'fast' layer norm that doesn't cast to float32, support APEX LN impl for slight speed gain, update norm and act factories, tweak SE for ability to disable bias (needed by GCVit)	2 years ago
Ross Wightman	e27c16b8a0	Remove unecessary code for synbn guard	2 years ago
Ross Wightman	879df47c0a	Support BatchNormAct2d for sync-bn use. Fix #1254	2 years ago
Ross Wightman	f670d98cb8	Make a few more layers symbolically traceable (remove from FX leaf modules) ... * remove dtype kwarg from .to() calls in EvoNorm as it messed up script + trace combo * BatchNormAct2d always uses custom forward (cut & paste from original) instead of super().forward. Fixes #1176 * BlurPool groups==channels, no need to use input.dim[1]	3 years ago
Ross Wightman	ab49d275de	Significant norm update ... * ConvBnAct layer renamed -> ConvNormAct and ConvNormActAa for anti-aliased * Significant update to EfficientNet and MobileNetV3 arch to support NormAct layers and grouped conv (as alternative to depthwise) * Update RegNet to add Z variant * Add Pre variant of XceptionAligned that works with NormAct layers * EvoNorm matches bits_and_tpu branch for merge	3 years ago
Ross Wightman	af607b75cc	Prep a set of ResNetV2 models with GroupNorm, EvoNormB0, EvoNormS0 for BN free model experiments on TPU and IPU	3 years ago
Ross Wightman	9811e229f7	Fix regression in models with 1001 class pretrained weights. Improve batchnorm arg and BatchNormAct layer handling in several models.	4 years ago
Ross Wightman	231d04e91a	ResNetV2 pre-act and non-preact model, w/ BiT pretrained weights and support for ViT R50 model. Tweaks for in21k num_classes passing. More to do... tests failing.	4 years ago
Ross Wightman	3aebc2f06c	Switch DPN to use BnAct layer, train a new DPN 68b model with RA to 79.21	4 years ago
Ross Wightman	f28170df3f	Fix an an untested change, remove a debug print	5 years ago
Ross Wightman	eb7653614f	Monster commit, activation refactor, VoVNet, norm_act improvements, more ... * refactor activations into basic PyTorch, jit scripted, and memory efficient custom auto * implement hard-mish, better grad for hard-swish * add initial VovNet V1/V2 impl, fix #151 * VovNet and DenseNet first models to use NormAct layers (support BatchNormAct2d, EvoNorm, InplaceIABN) * Wrap IABN for any models that use it * make more models torchscript compatible (DPN, PNasNet, Res2Net, SelecSLS) and add tests	5 years ago
Ross Wightman	14edacdf9a	DenseNet converted to support ABN (norm + act) modules. Experimenting with EvoNorm, IABN	5 years ago