Ross Wightman
6a01101905
Update efficientnet.py and convnext.py to multi-weight, add ImageNet-12k pretrained EfficientNet-B5 and ConvNeXt-Nano.
2 years ago
Fredo Guan
84178fca60
Merge branch 'rwightman:main' into main
2 years ago
Fredo Guan
c43340ddd4
Davit std ( #5 )
...
* Update davit.py
* Update test_models.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* starting point
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update test_models.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Davit revised (#4 )
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
clean up
* Update test_models.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update davit.py
* Update test_models.py
* Update davit.py
2 years ago
Ross Wightman
d5e7d6b27e
Merge remote-tracking branch 'origin/main' into refactor-imports
2 years ago
Ross Wightman
cda39b35bd
Add a deprecation phase to module re-org
2 years ago
Fredo Guan
edea013dd1
Davit std ( #3 )
...
Davit with all features working
2 years ago
Ross Wightman
7c4ed4d5a4
Add EVA-large models
2 years ago
Fredo Guan
434a03937d
Merge branch 'rwightman:main' into main
2 years ago
Ross Wightman
98047ef5e3
Add EVA FT results, hopefully fix BEiT test failures
2 years ago
Ross Wightman
3cc4d7a894
Fix missing register for 224 eva model
2 years ago
Ross Wightman
eba07b0de7
Add eva models to beit.py
2 years ago
Fredo Guan
3bd96609c8
Davit ( #1 )
...
Implement the davit model from https://arxiv.org/abs/2204.03645 and https://github.com/dingmyu/davit
2 years ago
Ross Wightman
927f031293
Major module / path restructure, timm.models.layers -> timm.layers, add _ prefix to all non model modules in timm.models
2 years ago
Ross Wightman
3785c234d7
Remove clip vit models that won't be ft and comment two that aren't uploaded yet
2 years ago
Ross Wightman
755570e2d6
Rename _pretrained.py -> pretrained.py, not feasible to change the other files to same scheme without breaking uses
2 years ago
Ross Wightman
72cfa57761
Add ported Tensorflow MaxVit weights. Add a few more CLIP ViT fine-tunes. Tweak some model tag names. Improve model tag name sorting. Update HF hub push config layout.
2 years ago
Ross Wightman
4d5c395160
MaxVit, ViT, ConvNeXt, and EfficientNet-v2 updates
...
* Add support for TF weights and modelling specifics to MaxVit (testing ported weights)
* More fine-tuned CLIP ViT configs
* ConvNeXt and MaxVit updated to new pretrained cfgs use
* EfficientNetV2, MaxVit and ConvNeXt high res models use squash crop/resize
2 years ago
Ross Wightman
9da7e3a799
Add crop_mode for pretraind config / image transforms. Add support for dynamo compilation to benchmark/train/validate
2 years ago
Ross Wightman
b2b6285af7
Add two more FT clip weights
2 years ago
Ross Wightman
5895056dc4
Add openai b32 ft
2 years ago
Ross Wightman
9dea5143d5
Adding more clip ft variants
2 years ago
Ross Wightman
444dcba4ad
CLIP B16 12k weights added
2 years ago
Ross Wightman
dff4717cbf
Add clip b16 384x384 finetunes
2 years ago
Ross Wightman
883fa2eeaa
Add fine-tuned B/16 224x224 in1k clip models
2 years ago
Ross Wightman
9a3d2ac2d5
Add latest CLIP ViT fine-tune pretrained configs / model entrypt updates
2 years ago
Ross Wightman
42bbbddee9
Add missing model config
2 years ago
Ross Wightman
def68befa7
Updating vit model defs for mult-weight support trial (vit first). Prepping for CLIP (laion2b and openai) fine-tuned weights.
2 years ago
Ross Wightman
0dadb4a6e9
Initial multi-weight support, handled so old pretraing config handling co-exists with new tags.
2 years ago
Wauplin
9b114754db
refactor push_to_hub helper
2 years ago
Wauplin
ae0a0db7de
Create repo before cloning with Repository.clone_from
2 years ago
Ross Wightman
803254bb40
Fix spacing misalignment for fast norm path in LayerNorm modules
2 years ago
Ross Wightman
6635bc3f7d
Merge pull request #1479 from rwightman/script_cleanup
...
Train / val script enhancements, non-GPU (ie CPU) device support, HF datasets support, TFDS/WDS dataloading improvements
2 years ago
Ross Wightman
0e6023f032
Merge pull request #1381 from ChristophReich1996/master
...
Fix typo in PositionalEncodingFourier
2 years ago
Ross Wightman
66f4af7090
Merge remote-tracking branch 'origin/master' into script_cleanup
2 years ago
Ross Wightman
9914f744dc
Add more maxxvit weights includ ConvNeXt conv block based experiments.
2 years ago
Mohamed Rashad
8fda68aff6
Fix repo id bug
...
This to fix this issue #1482
2 years ago
Ross Wightman
1199c5a1a4
clip_laion2b models need 1e-5 eps for LayerNorm
2 years ago
Ross Wightman
e858912e0c
Add brute-force checkpoint remapping option
2 years ago
Ross Wightman
b293dfa595
Add CL SE module
2 years ago
Ross Wightman
a383ef99f5
Make huggingface_hub necessary if it's the only source for a pretrained weight
2 years ago
Ross Wightman
e069249a2d
Add hf hub entries for laion2b clip models, add huggingface_hub dependency, update some setup/reqs, torch >= 1.7
2 years ago
Ross Wightman
9d65557be3
Fix errant import
2 years ago
Ross Wightman
9709dbaaa9
Adding support for fine-tune CLIP LAION-2B image tower weights for B/32, L/14, H/14 and g/14. Still WIP
2 years ago
Ross Wightman
a520da9b49
Update tresnet features_info for v2
2 years ago
Ross Wightman
c8ab747bf4
BEiT-V2 checkpoints didn't remove 'module' from weights, adapt checkpoint filter
2 years ago
Ross Wightman
73049dc2aa
Fix type in dla weight update
2 years ago
Ross Wightman
e11efa872d
Update a bunch of weights with external links to timm release assets. Fixes issue with *aliyuncs.com returning forbidden. Did pickle scan / verify and re-hash. Add TresNet-V2-L weights.
2 years ago
Ross Wightman
fa8c84eede
Update maxvit_tiny_256 weight to better iter, add coatnet / maxvit / maxxvit model defs for future runs
2 years ago
Ross Wightman
c1b3cea19d
Add maxvit_rmlp_tiny_rw_256 model def and weights w/ 84.2 top-1 @ 256, 84.8 @ 320
2 years ago
Ross Wightman
914544fc81
Add beitv2 224x224 checkpoints from https://github.com/microsoft/unilm/tree/master/beit2
2 years ago
Ross Wightman
dc90816f26
Add `maxvit_tiny_rw_224` weights 83.5 @ 224 and `maxvit_rmlp_pico_rw_256` relpos weights, 80.5 @ 256, 81.3 @ 320
2 years ago
Ross Wightman
f489f02ad1
Make gcvit window size ratio based to improve resolution changing support #1449 . Change default init to original.
2 years ago
Ross Wightman
7f1b223c02
Add maxvit_rmlp_nano_rw_256 model def & weights, make window/grid size dynamic wrt img_size by default
2 years ago
Ross Wightman
e6a4361306
pretrained_cfg entry for mvitv2_small_cls
2 years ago
Ross Wightman
f66e5f0e35
Fix class token support in MViT-V2, add small_class variant to ensure it's tested. Fix #1443
2 years ago
Ross Wightman
f1d2160d85
Update a few maxxvit comments, rename PartitionAttention -> PartitionAttenionCl for consistency with other blocks
2 years ago
Ross Wightman
eca6f0a25c
Fix syntax error (extra dataclass comma) in maxxvit.py
2 years ago
Ross Wightman
ff6a919cf5
Add --fast-norm arg to benchmark.py, train.py, validate.py
2 years ago
Ross Wightman
769ab4b98a
Clean up no_grad for trunc normal weight inits
2 years ago
Ross Wightman
48e1df8b37
Add norm/norm_act header comments
2 years ago
Ross Wightman
7c2660576d
Tweak init for convnext block using maxxvit/coatnext.
2 years ago
Ross Wightman
1d8d6f6072
Fix two default args in DenseNet blocks... fix #1427
2 years ago
Ross Wightman
527f9a4cb2
Updated to correct maxvit_nano weights...
2 years ago
Ross Wightman
b2e8426fca
Make k=stride=2 ('avg2') pooling default for coatnet/maxvit. Add weight links. Rename 'combined' partition to 'parallel'.
2 years ago
Ross Wightman
837c68263b
For ConvNeXt, use timm internal LayerNorm for fast_norm in non conv_mlp mode
2 years ago
Ross Wightman
cac0a4570a
More test fixes, pool size for 256x256 maxvit models
2 years ago
Ross Wightman
e939ed19b9
Rename internal creation fn for maxvit, has not been just coatnet for a while...
2 years ago
Ross Wightman
ffaf97f813
MaxxVit! A very configurable MaxVit and CoAtNet impl with lots of goodies..
2 years ago
Ross Wightman
8c9696c9df
More model and test fixes
2 years ago
Ross Wightman
ca52108c2b
Fix some model support functions
2 years ago
Ross Wightman
f332fc2db7
Fix some test failures, torchscript issues
2 years ago
Ross Wightman
6e559e9b5f
Add MViT (Multi-Scale) V2
2 years ago
Ross Wightman
43aa84e861
Add 'fast' layer norm that doesn't cast to float32, support APEX LN impl for slight speed gain, update norm and act factories, tweak SE for ability to disable bias (needed by GCVit)
2 years ago
Ross Wightman
c486aa71f8
Add GCViT
2 years ago
Ross Wightman
fba6ecd39b
Add EfficientFormer
2 years ago
Ross Wightman
ff4a38e2c3
Add PyramidVisionTransformerV2
2 years ago
Ross Wightman
1d8ada359a
Add timm ConvNeXt 'atto' weights, change test resolution for FB ConvNeXt 224x224 weights, add support for different dw kernel_size
2 years ago
Ross Wightman
2544d3b80f
ConvNeXt pico, femto, and nano, pico, femto ols (overlapping stem) weights and model defs
2 years ago
Ross Wightman
13565aad50
Add edgenext_base model def & weight link, update to improve ONNX export #1385
2 years ago
Ross Wightman
8ad4bdfa06
Allow ntuple to be used with string values
2 years ago
Christoph Reich
faae93e62d
Fix typo in PositionalEncodingFourier
2 years ago
Ross Wightman
ec6a28830f
Add DeiT-III 'medium' model defs and weights
2 years ago
Ross Wightman
6f103a442b
Add convnext_nano weights, 80.8 @ 224, 81.5 @ 288
2 years ago
Ross Wightman
4042a94f8f
Add weights for two 'Edge' block (3x3->1x1) variants of CS3 networks.
2 years ago
Ross Wightman
c8f69e04a9
Merge pull request #1365 from veritable-tech/fix-resize-pos-embed
...
Take `no_emb_class` into account when calling `resize_pos_embed`
2 years ago
Ceshine Lee
0b64117592
Take `no_emb_class` into account when calling `resize_pos_embed`
2 years ago
Jasha10
56c3a84db3
Update type hint for `register_notrace_module`
...
register_notrace_module is used to decorate types (i.e. subclasses of nn.Module).
It is not called on module instances.
2 years ago
Ross Wightman
1b278136c3
Change models with mean 0,0,0 std 1,1,1 from int to float for consistency as mentioned in #1355
2 years ago
Ross Wightman
909705e7ff
Remove some redundant requires_grad=True from nn.Parameter in third party code
2 years ago
Ross Wightman
c5e0d1c700
Add dilation support to convnext, allows output_stride=8 and 16 use. Fix #1341
2 years ago
Ross Wightman
dc376e3676
Ensure all model entrypoint fn default to `pretrained=False` (a few didn't)
2 years ago
Ross Wightman
23b102064a
Add cs3sedarknet_x weights w/ 82.65 @ 288 top1. Add 2 cs3 edgenet models (w/ 3x3-1x1 block), remove aa from cspnet blocks (not needed)
2 years ago
Ross Wightman
05313940e2
Add cs3darknet_x, cs3sedarknet_l, and darknetaa53 weights from TPU sessions. Move SE btwn conv1 & conv2 in DarkBlock. Improve SE/attn handling in Csp/DarkNet. Fix leaky_relu bug on older csp models.
2 years ago
nateraw
51cca82aa1
👽 use hf_hub_download instead of cached_download
2 years ago
Ross Wightman
a45b4bce9a
x and xx small edgenext models do benefit from larger test input size
2 years ago
Ross Wightman
a8e34051c1
Unbreak gamma remap impacting beit checkpoint load, version bump to 0.6.4
2 years ago
Ross Wightman
a1cb25066e
Add edgnext_small_rw weights trained with swin like recipe. Better than original 'small' but not the recent 'USI' distilled weights.
2 years ago
Ross Wightman
7c7ecd2492
Add --use-train-size flag to force use of train input_size (over test input size) for validation. Default test-time pooling to use train input size (fixes issues).
2 years ago
Ross Wightman
ce65a7b29f
Update vit_relpos w/ some additional weights, some cleanup to match recent vit updates, more MLP log coord experiments.
2 years ago
Ross Wightman
58621723bd
Add CrossStage3 DarkNet (cs3) weights
2 years ago