5 Commits (c55c3a11d3c034e2f3560ed434f75fd260689d80)

Author	SHA1	Message	Date
Taeksang Kim	7f29a46d44	Add gradient accumulation option to train.py ... option: iters-to-accum(iterations to accmulate) Gradient accumulation improves training performance(samples/s). It can reduce the number of parameter sharing between each node. This option can be helpful when network is bottleneck. Signed-off-by: Taeksang Kim <voidbag@puzzle-ai.com>	2 years ago
Ross Wightman	4f49b94311	Initial AGC impl. Still testing.	4 years ago
Ross Wightman	80078c47bb	Add Adafactor and Adahessian optimizers, cleanup optimizer arg passing, add gradient clipping support.	5 years ago
Ross Wightman	fcb6258877	Add missing leaky_relu layer factory defn, update Apex/Native loss scaler interfaces to support unscaled grad clipping. Bump ver to 0.2.2 for pending release.	5 years ago
Ross Wightman	532e3b417d	Reorg of utils into separate modules	5 years ago