fe457c1996Update SwinTransformerV2Cr post-merge, update with grad checkpointing / grad matcher * weight compat break, activate norm3 for final block of final stage (equivalent to pre-head norm, but while still in BLC shape) * remove fold/unfold for TPU compat, add commented out roll code for TPU * add option for end of stage norm in all stages * allow weight_init to be selected between pytorch default inits and xavier / moco style vit variant
Ross Wightman
2022-03-21 14:50:28 -0700
b049a5c5c6Merge remote-tracking branch 'origin/master' into norm_norm_norm
Ross Wightman
2022-03-21 13:41:43 -0700
7cdd164d77Fix#1184, scheduler noise bug during merge madness
Ross Wightman
2022-03-21 13:35:45 -0700
9440a50c95Merge branch 'mrT23-master'
Ross Wightman
2022-03-21 12:30:02 -0700
d98aa47d12Revert ml-decoder changes to model factory and train script
Ross Wightman
2022-03-21 12:29:02 -0700
Merge pull request #1112 from ayasyrev/sched_noise_dup_code
Ross Wightman
2022-03-21 11:52:37 -0700
61d3493f87Fix hf-hub handling when hf-hub is config source
Ross Wightman
2022-03-21 11:12:55 -0700
5f47518f27Fix pit implementation to be clsoer to deit/levit re distillation head handling
Ross Wightman
2022-03-21 11:12:14 -0700
0862e6ebaeFix correctness of some group matching regex (no impact on result), some formatting, missed forward_head for resnet
Ross Wightman
2022-03-19 14:58:54 -0700
94bcdebd73Add latest weights trained on TPU-v3 VM instances
Ross Wightman
2022-03-18 21:31:13 -0700
95739b45d7Fix partially removed alt_lable impl from TFDS variant of ImageNet22/12k
Ross Wightman
2022-03-17 17:57:05 -0700
5e1be34a60Add ImageNet-22k/12k TFDS dataset defs
Ross Wightman
2022-03-17 17:02:39 -0700
59ffab537cFix mistake in wds sample slicing
Ross Wightman
2022-03-17 14:42:40 -0700
ef57561d51Fix some TPU (XLA) issues with swin transformer v2
Ross Wightman
2022-03-16 14:55:36 -0700
ab16a358bbAdd log and continue handler for WDS errors, fix args.num_gpu for validation script fallback
Ross Wightman
2022-03-16 11:44:29 -0700
7eeaf521a0use gopen in wds to open info file in case it's at a url/gs location
Ross Wightman
2022-03-11 20:42:39 -0800
229ac6b8d8Fix alternate label handling in WDS parser to skip invalid alt labels
Ross Wightman
2022-03-11 19:16:04 -0800
a444d4b891Add alternative label support to WDS for imagenet22k/12k split, add 21k/22k/12k indices filters to results/
Ross Wightman
2022-03-09 10:17:12 -0800
da2796ae82Add webdataset (WDS) support, update TFDS to make some naming in parsers more similar. Fix workers=0 compatibility. Add ImageNet22k/12k synset defs.
Ross Wightman
2022-03-07 21:13:20 -0800
3fce010ca8Merge remote-tracking branch 'origin/norm_norm_norm' into bits_and_tpu
Ross Wightman
2022-02-28 17:07:56 -0800
0557c8257dFix bug introduced in non layer_decay weight_decay application. Remove debug print, fix arg desc.
Ross Wightman
2022-02-28 17:06:32 -0800
15cc9eae3eFix Swin v2 tuple type hint
Ross Wightman
2022-02-28 16:44:24 -0800
bb85b09d2aswin v2 fixup for latest changes on norm_norm_norm / bits_and_tpu branch
Ross Wightman
2022-02-28 16:39:16 -0800
10fa42b143Merge branch 'ChristophReich1996-master' into bits_and_tpu
Ross Wightman
2022-02-28 16:33:52 -0800
c639a86c67Change TFDS default to full re-shuffle (init) each epoch (for now)
Ross Wightman
2022-02-28 16:28:42 -0800
a16ea1e355Merge remote-tracking branch 'origin/norm_norm_norm' into bits_and_tpu
Ross Wightman
2022-02-28 16:28:27 -0800
372ad5fa0dSignificant model refactor and additions: * All models updated with revised foward_features / forward_head interface * Vision transformer and MLP based models consistently output sequence from forward_features (pooling or token selection considered part of 'head') * WIP param grouping interface to allow consistent grouping of parameters for layer-wise decay across all model types * Add gradient checkpointing support to a significant % of models, especially popular architectures * Formatting and interface consistency improvements across models * layer-wise LR decay impl part of optimizer factory w/ scale support in scheduler * Poolformer and Volo architectures added
Ross Wightman
2022-02-28 13:56:23 -0800
1420c118dfMissed comitting outstanding changes to default_cfg keys and test exclusions for swin v2
#1150
Ross Wightman
2022-02-23 19:50:26 -0800
c6e4b7895aSwin V2 CR impl refactor. * reformat and change some naming so closer to existing timm vision transformers * remove typing that wasn't adding clarity (or causing torchscript issues) * support non-square windows * auto window size adjust from image size * post-norm + main-branch no
Ross Wightman
2022-02-23 17:28:52 -0800
67d140446bFix bug in classification head
Christoph Reich
2022-02-20 22:28:05 +0100
29add820acRefactor (back to relative imports)
Christoph Reich
2022-02-20 00:46:48 +0100
74a04e0016Add parameter to change normalization type
Christoph Reich
2022-02-20 00:46:00 +0100
2a4f6c13ddCreate model functions
Christoph Reich
2022-02-20 00:40:22 +0100
87b4d7a29aAdd get and reset classifier method
Christoph Reich
2022-02-19 22:47:02 +0100
ff5f6bcd6cCheck input resolution
Christoph Reich
2022-02-19 22:42:02 +0100
81bf0b4033Change parameter names to match Swin V1
Christoph Reich
2022-02-19 22:37:22 +0100
f227b88831Add initials (CR) to model and file
Christoph Reich
2022-02-19 22:14:38 +0100
fafece230bAllow changing base lr batch size from 256 via arg
Ross Wightman
2022-01-27 17:35:50 -0800
7148039f9fTweak base lr log
Ross Wightman
2022-01-27 17:29:49 -0800
f82fb6b608Add base lr w/ linear and sqrt scaling to train script
Ross Wightman
2022-01-27 17:24:47 -0800
066e490605Merge branch 'norm_norm_norm' into bits_and_tpu
Ross Wightman
2022-01-27 16:47:45 -0800
5f81d4de23Move DeiT to own file, vit getting crowded. Working towards fixing #1029, make pooling interface for transformers and mlp closer to convnets. Still working through some details...
Ross Wightman
2022-01-26 22:53:57 -0800
629a0c1b8afix typy hint noise_range_t
#1113
ayasyrev
2022-01-26 16:08:33 +0300
6734cf56edcosine_lr docstring, type hints
ayasyrev
2022-01-26 15:36:31 +0300
07379c6d5dAdd vit_base2_patch32_256 for a model between base_patch16 and patch32 with a slightly larger img size and width
v0.1-mvit-weights
Ross Wightman
2022-01-24 14:45:59 -0800
cf4334391eUpdate benchmark and validate scripts to output results in JSON with a fixed delimiter for use in multi-process launcher
Ross Wightman
2022-01-24 11:13:58 -0800
1331c145a3Add train benchmark results, adjust name scheme for inference and train benchmark files.
Ross Wightman
2022-01-23 14:07:40 -0800
447677616fversion 0.5.5
Ross Wightman
2022-01-20 21:18:30 -0800
499c4749d7Add update NCHW and NHWC inference benchmark numbers for current models. Flip name of 'sam' vit models in results files
Ross Wightman
2022-01-20 10:40:04 -0800
83b40c5a58Last batch of small model weights (for now). mobilenetv3_small 050/075/100 and updated mnasnet_small with lambc/lamb optimizer.
Ross Wightman
2022-01-19 10:01:18 -0800
2d4b7e7080Update results csvs for latest release
Ross Wightman
2022-01-18 22:55:40 -0800
1aa617cb3bAdd AvgPool2d anti-aliasing support to ResNet arch (as per OpenAI CLIP models), add a few blur aa models as well
Ross Wightman
2022-01-18 21:57:24 -0800
f0f9eccda8Add --fuser arg to train/validate/benchmark scripts to select jit fuser type
Ross Wightman
2022-01-17 13:54:25 -0800
010b486590Add Dino pretrained weights (no head) for vit models. Add support to tests and helpers for models w/ no classifier (num_classes=0 in pretrained cfg)
Ross Wightman
2022-01-17 12:20:02 -0800
34fb4a9fedAdding a qk_scale argument to Attention block
#1091
S.Cao
2022-01-17 13:16:40 -0600
738a9cd635unbiased=False for torch.var_mean path of ConvNeXt LN. Fix#1090
Ross Wightman
2022-01-17 09:25:06 -0800
e0c4eec4b6Default conv_mlp to False across the board for ConvNeXt, causing issues on more setups than it's improving right now...
v0.5.4
Ross Wightman
2022-01-16 14:20:08 -0800
b669f4a588Add ConvNeXt 22k->1k fine-tuned and 384 22k-1k fine-tuned weights after testing
Ross Wightman
2022-01-15 15:43:52 -0800
Update README.md
Ross Wightman
2022-01-14 20:11:45 -0800
a8d103e18bGiant/gigantic vits snuck through in a test a broke GitHub test runner, add filter
Ross Wightman
2022-01-14 17:23:35 -0800
ef72ad4177Extra vit_huge model likely to cause test issue (non in21k variant), adding to filters
Ross Wightman
2022-01-14 15:56:59 -0800
e967c72875Update REAMDE.md. Sneak in g/G (giant / gigantic?) ViT defs from scaling paper
Ross Wightman
2022-01-14 15:39:31 -0800
9ca3437178Add some more small model weights lcnet, mnas, mnv2
Ross Wightman
2022-01-14 13:13:10 -0800
fa6463c936Version 0.5.4
Ross Wightman
2022-01-14 12:23:50 -0800
fa81164378Fix stem width for really small mobilenetv3 arch defs
Ross Wightman
2022-01-13 22:42:02 -0800
edd3d73695Add missing dropout for head reset in ConvNeXt default head
Ross Wightman
2022-01-13 22:41:23 -0800
b093dcb46dSome convnext cleanup, remove in place mul_ for gamma, breaking symbolic trace, cleanup head a bit...
Ross Wightman
2022-01-13 21:10:32 -0800
18934debc5Add initial ConvNeXt impl (mods of official code)
Ross Wightman
2022-01-12 23:05:41 -0800
656757d26bFix MobileNetV2 head conv size for multiplier < 1.0. Add some missing modification copyrights, fix starting date of some old ones.
Ross Wightman
2022-01-12 16:18:18 -0800