ea9c9550b2
Fully move ViT hybrids to their own file, including embedding module. Remove some extra DeiT models that were for benchmarking only.
3 years ago
a5310a3451
Merge remote-tracking branch 'origin/benchmark-fixes-vit_hybrids' into pit_and_vit_update
3 years ago
7953e5d11a
Fix pos_embed scaling for ViT and num_classes != 1000 for pretrained distilled deit and pit models. Fix #426 and fix #433
3 years ago
a760a4c3f4
Some ViT cleanup, merge distilled model with main, fixup torchscript support for distilled models
3 years ago
cf5fec5047
Cleanup experimental vit weight init a bit
3 years ago
cbcb76d72c
Should have included Conv2d layers in original weight init. Lets see what the impact is...
3 years ago
4de57ccf01
Add weight init scheme that's closer to JAX impl
3 years ago
45c048ba13
A few minor fixes and bit more cleanup on the huggingface hub integration.
3 years ago
d584e7f617
Support for huggingface hub via create_model and default_cfgs.
...
* improve consistency of model creation helper fns
* add comments to some of the model helpers
* support passing external default_cfgs so they can be sourced from hub
3 years ago
17cdee7354
Fix C&P patch_size error, and order of op patch_size arg resolution bug. Remove a test vit model.
3 years ago
0706d05d52
Benchmark models listed in txt file. Add more hybrid vit variants for testing
3 years ago
de97be9146
Spell out diff between my small and deit small vit models.
3 years ago
f0ffdf89b3
Add numerous experimental ViT Hybrid models w/ ResNetV2 base. Update the ViT naming for hybrids. Fix #426 for pretrained vit resizing.
3 years ago
5a8e1e643e
Initial Normalizer-Free Reg/ResNet impl. A bit of related layer refactoring.
3 years ago
bb50ac4708
Add DeiT distilled weights and distilled model def. Remove some redudant ViT model args.
3 years ago
c16e965037
Add some ViT comments and fix a few minor issues.
3 years ago
55f7dfa9ea
Refactor vision_transformer entrpy fns, add pos embedding resize support for fine tuning, add some deit models for testing
3 years ago
855d6cc217
More dataset work including factories and a tensorflow datasets (TFDS) wrapper
...
* Add parser/dataset factory methods for more flexible dataset & parser creation
* Add dataset parser that wraps TFDS image classification datasets
* Tweak num_classes handling bug for 21k models
* Add initial deit models so they can be benchmarked in next csv results runs
4 years ago
ce69de70d3
Add 21k weight urls to vision_transformer. Cleanup feature_info for preact ResNetV2 (BiT) models
4 years ago
231d04e91a
ResNetV2 pre-act and non-preact model, w/ BiT pretrained weights and support for ViT R50 model. Tweaks for in21k num_classes passing. More to do... tests failing.
4 years ago
b401952caf
Add newly added vision transformer large/base 224x224 weights ported from JAX official repo
4 years ago
61200db0ab
in_chans=1 working w/ pretrained weights for vision_transformer
4 years ago
f591e90b0d
Make sure num_features attr is present in vit models as with others
4 years ago
f944242cb0
Fix #262 , num_classes arg mixup. Make vision_transformers a bit closer to other models wrt get/reset classfier/forward_features. Fix torchscript for ViT.
4 years ago
736f209e7d
Update vision transformers to be compatible with official code. Port official ViT weights from jax impl.
4 years ago
27a93e9de7
Improve test crop for ViT models. Small now 77.85, added base weights at 79.35 top-1.
4 years ago
d4db9e7977
Add small vision transformer weights. 77.42 top-1.
4 years ago
f31933cb37
Initial Vision Transformer impl w/ patch and hybrid variants. Refactor tuple helpers.
4 years ago
Ross Wightman