pytorch-image-models

Commit Graph

Author	SHA1	Message	Date
Ross Wightman	ac469b50da	Optimizer improvements, additions, cleanup * Add MADGRAD code * Fix Lamb (non-fused variant) to work w/ PyTorch XLA * Tweak optimizer factory args (lr/learning_rate and opt/optimizer_name), may break compat * Use newer fn signatures for all add,addcdiv, addcmul in optimizers * Use upcoming PyTorch native Nadam if it's available * Cleanup lookahead opt * Add optimizer tests * Remove novograd.py impl as it was messy, keep nvnovograd * Make AdamP/SGDP work in channels_last layout * Add rectified adablief mode (radabelief) * Support a few more PyTorch optim, adamax, adagrad	3 years ago
Sangdoo Yun	e93e571f7a	Add `adamp` and 'sgdp' optimizers. Update requirements.txt Update optim_factory.py Add `adamp` optimizer Update __init__.py copy files of adamp & sgdp Create adamp.py Update __init__.py Create sgdp.py Update optim_factory.py Update optim_factory.py Update requirements.txt Update adamp.py Update sgdp.py Update sgdp.py Update adamp.py	4 years ago

Author

SHA1

Message

Date

Ross Wightman

ac469b50da

Optimizer improvements, additions, cleanup

* Add MADGRAD code
* Fix Lamb (non-fused variant) to work w/ PyTorch XLA
* Tweak optimizer factory args (lr/learning_rate and opt/optimizer_name), may break compat
* Use newer fn signatures for all add,addcdiv, addcmul in optimizers
* Use upcoming PyTorch native Nadam if it's available
* Cleanup lookahead opt
* Add optimizer tests
* Remove novograd.py impl as it was messy, keep nvnovograd
* Make AdamP/SGDP work in channels_last layout
* Add rectified adablief mode (radabelief)
* Support a few more PyTorch optim, adamax, adagrad

Sangdoo Yun

e93e571f7a

Add `adamp` and 'sgdp' optimizers.

Update requirements.txt

Update optim_factory.py

Add `adamp` optimizer

Update __init__.py

copy files of adamp & sgdp

Create adamp.py

Update __init__.py

Create sgdp.py

Update optim_factory.py

Update optim_factory.py

Update requirements.txt

Update adamp.py

Update sgdp.py

Update sgdp.py

Update adamp.py

2 Commits (9541f4963bb6188789b65b2902373ec50ccfdc8e)