81ca323751
Davit update formatting and fix grad checkpointing ( #7 )
...
fixed head to gap->norm->fc as per convnext, along with option for norm->gap->fc
failed tests due to clip convnext models, davit tests passed
1 year ago
927f031293
Major module / path restructure, timm.models.layers -> timm.layers, add _ prefix to all non model modules in timm.models
1 year ago
0862e6ebae
Fix correctness of some group matching regex (no impact on result), some formatting, missed forward_head for resnet
2 years ago
372ad5fa0d
Significant model refactor and additions:
...
* All models updated with revised foward_features / forward_head interface
* Vision transformer and MLP based models consistently output sequence from forward_features (pooling or token selection considered part of 'head')
* WIP param grouping interface to allow consistent grouping of parameters for layer-wise decay across all model types
* Add gradient checkpointing support to a significant % of models, especially popular architectures
* Formatting and interface consistency improvements across models
* layer-wise LR decay impl part of optimizer factory w/ scale support in scheduler
* Poolformer and Volo architectures added
2 years ago
abc9ba2544
Transitioning default_cfg -> pretrained_cfg. Improving handling of pretrained_cfg source (HF-Hub, files, timm config, etc). Checkpoint handling tweaks.
2 years ago
65d827c7a6
rename notrace registration and standardize trace_utils imports
3 years ago
0262a0e8e1
fx ready for review
3 years ago
b25ff96768
wip - pre-rebase
3 years ago
bc3d4eb403
wip -rebase
3 years ago
8880f696b6
Refactoring, cleanup, improved test coverage.
...
* Add eca_nfnet_l2 weights, 84.7 @ 384x384
* All 'non-std' (ie transformer / mlp) models have classifier / default_cfg test added
* Fix #694 reset_classifer / num_features / forward_features / num_classes=0 consistency for transformer / mlp models
* Add direct loading of npz to vision transformer (pure transformer so far, hybrid to come)
* Rename vit_deit* to deit_*
* Remove some deprecated vit hybrid model defs
* Clean up classifier flatten for conv classifiers and unusual cases (mobilenetv3/ghostnet)
* Remove explicit model fns for levit conv, just pass in arg
3 years ago
ba2ca4b464
One codepath for stdconv, switch layernorm to batchnorm so gain included. Tweak epsilon values for nfnet, resnetv2, vit hybrid.
3 years ago
8e4ac3549f
All ScaledStdConv and StdConv uses default to using F.layernorm so that they work with PyTorch XLA. eps value tweaking is a WIP.
3 years ago
742c2d5247
Add Gather-Excite and Global Context attn modules. Refactor existing SE-like attn for consistency and refactor byob/byoanet for less redundancy.
3 years ago
18bf520ad1
Add eca_nfnet_l2/l3 defs for future training
3 years ago
21812d33aa
Add prelim efficientnet_v2s weights from 224x224 train, eval 83.3 @ 288. Add eca_nfnet_l1 weights, train at 256, eval 84 @ 320.
3 years ago
2bb65bd875
Wrong default_cfg pool_size for L1
3 years ago
9071568f0e
Add weights for SE NFNet-L0 model, rename nfnet_l0b -> nfnet_l0. 82.75 top-1 @ 288. Add nfnet_l1 model def for training.
3 years ago
14ac4abf74
Change huggingface hub revision delimiter to '@', add hf_hub reference for eca_nfnet_l0 model as an example.
3 years ago
740f32c96a
Add ECA-NFNet-L0 weights and update model name. Update README and bump version to 0.4.6
3 years ago
317ea3e599
Tweak NFNet, ResNetV2, RexNet feature extraction, use pre-act features for simplicty. Update tests for pytorch 1.8. 0.4.5 version bump.
3 years ago
678ba4e0a2
Add NFNet-F model weights ported from DeepMind Haiku impl and new set of models w/ compatible config.
3 years ago
5f9aff395c
Fix stem width in NFNet-F models, add some more comments, add some 'light' NFNet models for testing.
3 years ago
d86dbe45c2
Update README.md and few more comments
3 years ago
0d253e2c5e
Fix issue with nfnet tests, bit more cleanup.
3 years ago
cb06c7a910
Add NFNet-F models and tweak existing NF models.
3 years ago
e4de077021
Add first 'Normalizer Free' models. nf_regnet_b1 79.3 @ 288x288 test, and nf_resnet50 80.3 @ 256x256 test (80.68 @ 288x288).
3 years ago
dc85e5a237
Add ByobNet w/ GPU-EfficientNets and RepVGG. Also add classic vgg models.
3 years ago
f0e65e37b7
Fix NF-ResNet101 model defs
3 years ago
2de54d174a
Fix pool size defs for NFNet models, add a comment.
3 years ago
90980de4a9
Fix up a few details in NFResNet models, managed stable training. Add support for gamma gain to be applied in activation or ScaleStdConv. Some tweaks to ScaledStdConv.
3 years ago
5a8e1e643e
Initial Normalizer-Free Reg/ResNet impl. A bit of related layer refactoring.
3 years ago
Fredo Guan
Ross Wightman
Alexander Soare