* Support PyTorch native DDP as fallback if APEX not present
* Support SyncBN for both APEX and Torch native (if torch >= 1.1)
* EMA model does not appear to need DDP wrapper, no gradients, updated from wrapped model
* ModelEma class added to track an EMA set of weights for the model being trained
* EMA handling added to train, validation and clean_checkpoint scripts
* Add multi checkpoint or multi-model validation support to validate.py
* Add syncbn option (APEX) to train script for experimentation
* Cleanup interface of CheckpointSaver while adding ema functionality
* B0-B3 weights ported from TF with close to paper accuracy
* Renamed gen_mobilenet to gen_efficientnet since scaling params go well beyond 'mobile' specific
* Add Tensorflow preprocessing option for closer images to source repo
* Do mixup in custom collate fn if prefetcher enabled, reduces performance impact
* Move mixup code to own file
* Add arg to disable prefetcher
* Fix no cuda transfer when prefetcher off
* Random erasing when prefetcher off wasn't changed to match new args, fixed
* Default random erasing to off (prob = 0.) for train
* factor out data related constants to own file
* move data related config helpers to own file
* add a variant of RandomResizeCrop that randomizes interpolation method
* remove old Numpy version of RandomErasing
* cleanup torch version of RandomErasing and use it in either GPU loader batch mode or single image cpu Transform
* init square_avg with one instead of zero as per TF
* match TF order of ops for square_avg accumulation
* move LR scaling to momentum buffer accumulator as per TF
* add decoupled weight decay flag (not in TF)
Identity is not the same thing as equality in Python. In these instances, we want the latter.
Use ==/!= to compare str, bytes, and int literals.
$ __python__
```python
>>> proj = "pro"
>>> proj += 'j'
>>> proj
'proj'
>>> proj == 'proj'
True
>>> proj is 'proj'
False
>>> 0 == 0.0
True
>>> 0 is 0.0
False
```
[flake8](http://flake8.pycqa.org) testing of https://github.com/rwightman/pytorch-image-models on Python 3.7.1
$ __flake8 . --count --select=E9,F63,F72,F82 --show-source --statistics__
```
./data/loader.py:48:23: F823 local variable 'input' defined as a builtin referenced before assignment
yield input, target
^
./models/dpn.py:170:12: F632 use ==/!= to compare str, bytes, and int literals
if block_type is 'proj':
^
./models/dpn.py:173:14: F632 use ==/!= to compare str, bytes, and int literals
elif block_type is 'down':
^
./models/dpn.py:177:20: F632 use ==/!= to compare str, bytes, and int literals
assert block_type is 'normal'
^
3 F632 use ==/!= to compare str, bytes, and int literals
1 F823 local variable 'input' defined as a builtin referenced before assignment
4
```
__E901,E999,F821,F822,F823__ are the "_showstopper_" [flake8](http://flake8.pycqa.org) issues that can halt the runtime with a SyntaxError, NameError, etc. These 5 are different from most other flake8 issues which are merely "style violations" -- useful for readability but they do not effect runtime safety.
* F821: undefined name `name`
* F822: undefined name `name` in `__all__`
* F823: local variable name referenced before assignment
* E901: SyntaxError or IndentationError
* E999: SyntaxError -- failed to compile a file into an Abstract Syntax Tree
Ross Wightman