13 Commits (a9ecb880e5ff30c4be891651dbd0a11686d69f47)

Author	SHA1	Message	Date
Ross Wightman	372ad5fa0d	Significant model refactor and additions: ... * All models updated with revised foward_features / forward_head interface * Vision transformer and MLP based models consistently output sequence from forward_features (pooling or token selection considered part of 'head') * WIP param grouping interface to allow consistent grouping of parameters for layer-wise decay across all model types * Add gradient checkpointing support to a significant % of models, especially popular architectures * Formatting and interface consistency improvements across models * layer-wise LR decay impl part of optimizer factory w/ scale support in scheduler * Poolformer and Volo architectures added	3 years ago
Ross Wightman	ab49d275de	Significant norm update ... * ConvBnAct layer renamed -> ConvNormAct and ConvNormActAa for anti-aliased * Significant update to EfficientNet and MobileNetV3 arch to support NormAct layers and grouped conv (as alternative to depthwise) * Update RegNet to add Z variant * Add Pre variant of XceptionAligned that works with NormAct layers * EvoNorm matches bits_and_tpu branch for merge	3 years ago
Ross Wightman	78912b6375	Updated EvoNorm implementations with some experimentation. Add FilterResponseNorm. Updated RegnetZ and ResNetV2 model defs for trials.	3 years ago
Ross Wightman	480c676ffa	Fix FX breaking assert in evonorm	3 years ago
Ross Wightman	93cc08fdc5	Make evonorm variables 1d to match other PyTorch norm layers, will break weight compat for any existing use (likely minimal, easy to fix).	3 years ago
Ross Wightman	af607b75cc	Prep a set of ResNetV2 models with GroupNorm, EvoNormB0, EvoNormS0 for BN free model experiments on TPU and IPU	3 years ago
Alexander Soare	b25ff96768	wip - pre-rebase	3 years ago
Alexander Soare	bc3d4eb403	wip -rebase	3 years ago
Ross Wightman	fc8b8afb6f	Fix a silly bug in Sample version of EvoNorm missing x* part of swish, update EvoNormBatch to accumulated unbiased variance.	4 years ago
Ross Wightman	6c17d57a2c	Fix some attributions, add copyrights to some file docstrings	4 years ago
Ross Wightman	eb7653614f	Monster commit, activation refactor, VoVNet, norm_act improvements, more ... * refactor activations into basic PyTorch, jit scripted, and memory efficient custom auto * implement hard-mish, better grad for hard-swish * add initial VovNet V1/V2 impl, fix #151 * VovNet and DenseNet first models to use NormAct layers (support BatchNormAct2d, EvoNorm, InplaceIABN) * Wrap IABN for any models that use it * make more models torchscript compatible (DPN, PNasNet, Res2Net, SelecSLS) and add tests	5 years ago
Ross Wightman	780860d140	Add norm_act factory method, move JIT of norm layers to factory	5 years ago
Ross Wightman	14edacdf9a	DenseNet converted to support ABN (norm + act) modules. Experimenting with EvoNorm, IABN	5 years ago