1186fc9c73
Merge remote-tracking branch 'origin/master' into bits_and_tpu
3 years ago
7c7ecd2492
Add --use-train-size flag to force use of train input_size (over test input size) for validation. Default test-time pooling to use train input size (fixes issues).
3 years ago
eca09b8642
Add MobileVitV2 support. Fix #1332 . Move GroupNorm1 to common layers (used in poolformer + mobilevitv2). Keep ol custom ConvNeXt LayerNorm2d impl as LayerNormExp2d for reference.
3 years ago
7a9c6811c9
Add eps arg to LayerNorm2d, add 'tf' (tensorflow) variant of trunc_normal_ that applies scale/shift after sampling (instead of needing to move a/b)
3 years ago
82c311d082
Add more experimental darknet and 'cs2' darknet variants (different cross stage setup, closer to newer YOLO backbones) for train trials.
3 years ago
07d0c4ae96
Improve repr for DropPath module
3 years ago
e27c16b8a0
Remove unecessary code for synbn guard
3 years ago
0da3c9ebbf
Remove SiLU layer in default args that breaks import on old old PyTorch
3 years ago
879df47c0a
Support BatchNormAct2d for sync-bn use. Fix #1254
3 years ago
dff33730b3
Merge remote-tracking branch 'origin/master' into bits_and_tpu
3 years ago
4b30bae67b
Add updated vit_relpos weights, and impl w/ support for official swin-v2 differences for relpos. Add bias control support for MLP layers
3 years ago
f88c606fcf
fixing channels_last on cond_conv2d; update nvfuser debug env variable
3 years ago
1ba0ec4c18
Merge remote-tracking branch 'origin/master' into bits_and_tpu
3 years ago
f670d98cb8
Make a few more layers symbolically traceable (remove from FX leaf modules)
...
* remove dtype kwarg from .to() calls in EvoNorm as it messed up script + trace combo
* BatchNormAct2d always uses custom forward (cut & paste from original) instead of super().forward. Fixes #1176
* BlurPool groups==channels, no need to use input.dim[1]
3 years ago
749856cf25
Merge remote-tracking branch 'origin/norm_norm_norm' into bits_and_tpu
3 years ago
b049a5c5c6
Merge remote-tracking branch 'origin/master' into norm_norm_norm
3 years ago
9440a50c95
Merge branch 'mrT23-master'
3 years ago
a16ea1e355
Merge remote-tracking branch 'origin/norm_norm_norm' into bits_and_tpu
3 years ago
372ad5fa0d
Significant model refactor and additions:
...
* All models updated with revised foward_features / forward_head interface
* Vision transformer and MLP based models consistently output sequence from forward_features (pooling or token selection considered part of 'head')
* WIP param grouping interface to allow consistent grouping of parameters for layer-wise decay across all model types
* Add gradient checkpointing support to a significant % of models, especially popular architectures
* Formatting and interface consistency improvements across models
* layer-wise LR decay impl part of optimizer factory w/ scale support in scheduler
* Poolformer and Volo architectures added
3 years ago
066e490605
Merge branch 'norm_norm_norm' into bits_and_tpu
3 years ago
95cfc9b3e8
Merge remote-tracking branch 'origin/master' into norm_norm_norm
3 years ago
656757d26b
Fix MobileNetV2 head conv size for multiplier < 1.0. Add some missing modification copyrights, fix starting date of some old ones.
4 years ago
b27c21b09a
Update drop_path and drop_block (fast impl) to be symbolically traceable, slightly faster
4 years ago
214c84a235
Disable use of timm nn.Linear wrapper since AMP autocast + torchscript use appears fixed
4 years ago
7eb7e73216
File will not stay deleted
4 years ago
a52a614475
Remove layer experiment which should not have been added
4 years ago
d829858550
Significant norm update
...
* ConvBnAct layer renamed -> ConvNormAct and ConvNormActAa for anti-aliased
* Significant update to EfficientNet and MobileNetV3 arch to support NormAct layers and grouped conv (as alternative to depthwise)
* Update RegNet to add Z variant
* Add Pre variant of XceptionAligned that works with NormAct layers
* EvoNorm matches bits_and_tpu branch for merge
4 years ago
ab49d275de
Significant norm update
...
* ConvBnAct layer renamed -> ConvNormAct and ConvNormActAa for anti-aliased
* Significant update to EfficientNet and MobileNetV3 arch to support NormAct layers and grouped conv (as alternative to depthwise)
* Update RegNet to add Z variant
* Add Pre variant of XceptionAligned that works with NormAct layers
* EvoNorm matches bits_and_tpu branch for merge
4 years ago
4d7a5544f7
Remove inplace sigmoid for consistency with other impl
4 years ago
88a5b54802
A few small evonorm tweaks for convergence comparisons
4 years ago
66daee4f31
Last change wasn't complete, missed adding full evo_norm changeset
4 years ago
7bbbd5ef1b
EvoNorm and GroupNormAct options for debugging TPU / XLA concerns
4 years ago
d04f2f1377
Update drop_path and drop_block (fast impl) to be symbolically traceable, slightly faster
4 years ago
834a9ec721
Disable use of timm nn.Linear wrapper since AMP autocast + torchscript use appears fixed
4 years ago
78912b6375
Updated EvoNorm implementations with some experimentation. Add FilterResponseNorm. Updated RegnetZ and ResNetV2 model defs for trials.
4 years ago
c11f4c3218
support CNNs
4 years ago
d6701d8a81
Merge branch 'rwightman:master' into master
4 years ago
480c676ffa
Fix FX breaking assert in evonorm
4 years ago
41559247e9
use_ml_decoder_head
4 years ago
93cc08fdc5
Make evonorm variables 1d to match other PyTorch norm layers, will break weight compat for any existing use (likely minimal, easy to fix).
4 years ago
af607b75cc
Prep a set of ResNetV2 models with GroupNorm, EvoNormB0, EvoNormS0 for BN free model experiments on TPU and IPU
4 years ago
c976a410d9
Add ResNet-50 w/ GN (resnet50_gn) and SEBotNet-33-TS (sebotnet33ts_256) model defs and weights. Update halonet50ts weights w/ slightly better variant in1k val, more robust to test sets.
4 years ago
b25ff96768
wip - pre-rebase
4 years ago
e051dce354
Make all models FX traceable
4 years ago
0149ec30d7
wip - attempting to rebase
4 years ago
bc3d4eb403
wip -rebase
4 years ago
2ddef942b9
Better fix for #954 that doesn't break torchscript, pull torch._assert into timm namespace when it exists
4 years ago
4f0f9cb348
Fix #954 by bringing traceable _assert into timm to allow compat w/ PyTorch < 1.8
4 years ago
b745d30a3e
Fix formatting of last commit
4 years ago
3478f1d7f1
Traceability fix for vit models for some experiments
4 years ago
Ross Wightman