Commit Graph

14 Commits (88a5b54802b1ab88057744b3925490211621facd)

Author SHA1 Message Date
Ross Wightman 88a5b54802 A few small evonorm tweaks for convergence comparisons
3 years ago
Ross Wightman 66daee4f31 Last change wasn't complete, missed adding full evo_norm changeset
3 years ago
Ross Wightman 7bbbd5ef1b EvoNorm and GroupNormAct options for debugging TPU / XLA concerns
3 years ago
Ross Wightman 78912b6375 Updated EvoNorm implementations with some experimentation. Add FilterResponseNorm. Updated RegnetZ and ResNetV2 model defs for trials.
3 years ago
Ross Wightman 480c676ffa Fix FX breaking assert in evonorm
3 years ago
Ross Wightman 93cc08fdc5 Make evonorm variables 1d to match other PyTorch norm layers, will break weight compat for any existing use (likely minimal, easy to fix).
3 years ago
Ross Wightman af607b75cc Prep a set of ResNetV2 models with GroupNorm, EvoNormB0, EvoNormS0 for BN free model experiments on TPU and IPU
3 years ago
Alexander Soare b25ff96768 wip - pre-rebase
3 years ago
Alexander Soare bc3d4eb403 wip -rebase
3 years ago
Ross Wightman fc8b8afb6f Fix a silly bug in Sample version of EvoNorm missing x* part of swish, update EvoNormBatch to accumulated unbiased variance.
4 years ago
Ross Wightman 6c17d57a2c Fix some attributions, add copyrights to some file docstrings
4 years ago
Ross Wightman eb7653614f Monster commit, activation refactor, VoVNet, norm_act improvements, more
4 years ago
Ross Wightman 780860d140 Add norm_act factory method, move JIT of norm layers to factory
4 years ago
Ross Wightman 14edacdf9a DenseNet converted to support ABN (norm + act) modules. Experimenting with EvoNorm, IABN
4 years ago