Commit Graph

12 Commits (b1b024dfed5870cb1238c8c7c326d6ea8ef67300)

Author SHA1 Message Date
Ross Wightman 43aa84e861 Add 'fast' layer norm that doesn't cast to float32, support APEX LN impl for slight speed gain, update norm and act factories, tweak SE for ability to disable bias (needed by GCVit)
2 years ago
Ross Wightman 78912b6375 Updated EvoNorm implementations with some experimentation. Add FilterResponseNorm. Updated RegnetZ and ResNetV2 model defs for trials.
3 years ago
Ross Wightman 9c78de8c02 Fix #661, move hardswish out of default args for LeViT. Enable native torch support for hardswish, hardsigmoid, mish if present.
4 years ago
Ross Wightman 5f4b6076d8 Fix inplace arg compat for GELU and PreLU via activation factory
4 years ago
Ross Wightman fd962c4b4a Native SiLU (Swish) op doesn't export to ONNX
4 years ago
Ross Wightman e90edce438 Support native silu activation (aka swish). An optimized ver is available in PyTorch 1.7.
4 years ago
Ross Wightman fcb6258877 Add missing leaky_relu layer factory defn, update Apex/Native loss scaler interfaces to support unscaled grad clipping. Bump ver to 0.2.2 for pending release.
4 years ago
Ross Wightman 6c17d57a2c Fix some attributions, add copyrights to some file docstrings
4 years ago
Ross Wightman 298fba09ac Back out some activation hacks trialing upcoming pytorch changes
4 years ago
Ross Wightman 3b9004bef9 Lots of changes to model creation helpers, close to finalizing feature extraction / interfaces
4 years ago
Ross Wightman 151679c2f1 Add custom grad tests, fix cut & paste error with hard_mish ME, add a few more pytorch act fns to factory
4 years ago
Ross Wightman eb7653614f Monster commit, activation refactor, VoVNet, norm_act improvements, more
5 years ago