Ross Wightman
eb7653614f
Monster commit, activation refactor, VoVNet, norm_act improvements, more
...
* refactor activations into basic PyTorch, jit scripted, and memory efficient custom auto
* implement hard-mish, better grad for hard-swish
* add initial VovNet V1/V2 impl, fix #151
* VovNet and DenseNet first models to use NormAct layers (support BatchNormAct2d, EvoNorm, InplaceIABN)
* Wrap IABN for any models that use it
* make more models torchscript compatible (DPN, PNasNet, Res2Net, SelecSLS) and add tests
4 years ago
Ross Wightman
0ea53cecc3
Merge branch 'master' into densenet_update_and_more
4 years ago
Ross Wightman
6441e9cc1b
Fix memory_efficient mode for DenseNets. Add AntiAliasing (Blur) support for DenseNets and create one test model. Add lr cycle/mul params to train args.
4 years ago
Ross Wightman
afb6bd0669
Add backward and default_cfg tests and fix a few issues found. Fix #153
4 years ago
Ross Wightman
50658b9a67
Add RegNet models and weights
4 years ago
Ross Wightman
7df83258c9
Merge branch 'master' into densenet_update_and_more
4 years ago
Ross Wightman
1904ed8fec
Improve dropblock impl, add fast variant, and better AMP speed, inplace, batchwise... few ResNeSt cleanups
4 years ago
Ross Wightman
17270c69b9
Remove annoying InceptionV3 dependency on scipy and insanely slow trunc_norm init. Bring InceptionV3 code into this codebase and use upcoming torch trunch_norm_ init.
4 years ago
Ross Wightman
c4ca016656
Merge pull request #145 from rwightman/resnest
...
ResNeSt
4 years ago
Ross Wightman
208e7912f7
Missed one of the abalation model entrypoints, update README
4 years ago
Ross Wightman
9cc289f18c
Exclude EfficientNet-L2 models from test
4 years ago
Ross Wightman
2f884a0ce5
Add resnest14, resnest26, and two of the abalation grouped resnest50 models
4 years ago
Ross Wightman
f4cdc2ac31
Add ResNeSt models
4 years ago
Ross Wightman
780860d140
Add norm_act factory method, move JIT of norm layers to factory
4 years ago
Ross Wightman
14edacdf9a
DenseNet converted to support ABN (norm + act) modules. Experimenting with EvoNorm, IABN
4 years ago
Ross Wightman
022ed001f3
Update DenseNet to latest in Torchvision (torchscript compat, checkpointing, proper init). Start adding ehanced configurability, stem options...
4 years ago
Vyacheslav Shults
a7ebe09029
Replace all None by nn.Identity() in all models reset_classifier when False-values num_classes is given.
...
Make small code refactoring
4 years ago
Vyacheslav Shults
f0eb021620
Replace all None by nn.Identity() in HRNet modules
4 years ago
Ross Wightman
8d8677e03b
Fix #139 . Broken SKResNets after BlurPool addition, as a plus, SKResNets support AA now too.
4 years ago
AFLALO, Jonathan Isaac
9c15d57505
added efficientnet pruned weights
4 years ago
Ross Wightman
ea300709f0
Fix model create fn not passing num_classes through. Fix #135
4 years ago
Ross Wightman
2c438c4e0d
Merge branch 'master' of github.com:rwightman/pytorch-models
4 years ago
Ross Wightman
2681a8d618
Final blurpool2d cleanup and add resnetblur50 weights, match tresnet Downsample arg order to BlurPool2d for interop
4 years ago
Ross Wightman
9590f301a9
Merge branch 'blur' of https://github.com/VRandme/pytorch-image-models into VRandme-blur
4 years ago
Ross Wightman
a5220ad3aa
Merge pull request #125 from Separius/patch-1
...
fix typo in eca
4 years ago
Ross Wightman
0834fbc01c
Move pruned model adapt strings to separate txt files. A few minor formatting alignment tweaks.
4 years ago
Ross Wightman
ba793f5c1c
Merge branch 'adding_ECA_resnet' of https://github.com/yoniaflalo/pytorch-image-models into yoniaflalo-adding_ECA_resnet
4 years ago
AFLALO, Jonathan Isaac
07f19dd699
added eca resnet
4 years ago
Ross Wightman
be7c784d21
Small TResNet simplification, just use SelectAdaptivePool, don't notice any perf difference
4 years ago
Ross Wightman
e3a98171b2
Remove some CRLF that made it into TResNet merge
4 years ago
Ross Wightman
0004f37d25
Some TResNet cleanup.
...
* allow use of global pool arg, test-time-pooling
* clean checkpoints to just contain state dict, add 448 res checkpoints
* support DataParallel via lazy filter creation for JIT Downsample
* some minor formatting (mostly alignment) preferences
4 years ago
Sepehr Sameni
20290b5b56
fix typo in eca
4 years ago
Ross Wightman
ebf82b84ac
Merge pull request #122 from mrT23/master
...
TResNet models
4 years ago
Alexey Chernov
bdb165a8a4
Merge changes in feature extraction interface to MobileNetV3
...
Experimental feature extraction interface seems to be changed
a little bit with the most up to date version apparently found
in EfficientNet class. Here these changes are added to
MobileNetV3 class to make it support it and work again, too.
4 years ago
talrid
8a63c1add8
finalizing
4 years ago
talrid
6209146738
TResNet models
4 years ago
Ross Wightman
1a8f5900ab
Update EfficientNet feature extraction for EfficientDet. Add needed MaxPoolSame as well.
4 years ago
Chris Ha
06a50a94a8
Fix minor typos in create_attn.py and resnet.py
...
'eca'->'ceca'
and
doest not-> does not
4 years ago
Ross Wightman
1a9ab07307
One too many changes at a time, fix missing C
4 years ago
Ross Wightman
f17b42bc33
Blur filter no longer a buffer
4 years ago
Ross Wightman
6cdeca24a3
Some cleanup and fixes for initial BlurPool impl. Still some testing and tweaks to go...
4 years ago
Chris Ha
acd1b6cccd
Implement Functional Blur on resnet.py
...
1. add ResNet argument blur=''
2. implement blur for maxpool and strided convs in downsampling blocks
4 years ago
Chris Ha
ce3d82b58b
Update blurpool.py
...
clean up code for PR
4 years ago
Chris Ha
3a287a6e76
Create blurpool.py
...
Initial implementation of blur layer.
currently tests as correct against Downsample of original github
4 years ago
Ross Wightman
c99a5abed4
Merge pull request #115 from rwightman/mobilenetv2-experiment
...
MobileNet-V2 experiments
4 years ago
Ross Wightman
e34074b4da
Add final weights for MobileNet-V2 experiments
4 years ago
Ross Wightman
a6a5565de7
Fiddling...
5 years ago
Ross Wightman
5a16c533ff
Add better resnext50_32x4d weights trained by andravin
5 years ago
Ross Wightman
bc998cad91
Experimenting with some MobileNetV2 variations to compare against EfficientNet-Lite
5 years ago
Ross Wightman
3406e582cf
Add EfficientNet-Lite results, update README
5 years ago
Ross Wightman
bd05258f7b
EfficientNet-Lite model added w/ converted checkpoints, validation in progress...
5 years ago
Ross Wightman
56e2ac3a6d
Merge pull request #94 from rwightman/lr_noise
...
Learning rate noise, MobileNetV3 weights, and activate MobileNetV3/EfficientNet weight init change
5 years ago
Ross Wightman
c16f25ced2
Add MobileNetV3 Large weights, results, update README and sotabench for merge
5 years ago
Ross Wightman
c60069c1eb
Annotate types on drop fns to avoid torchscript error
5 years ago
Ross Wightman
9fee316752
Enable fixed fanout calc in EfficientNet/MobileNetV3 weight init by default. Fix #84
5 years ago
Ross Wightman
43225d110c
Unify drop connect vs drop path under 'drop path' name, switch all EfficientNet/MobilenetV3 refs to 'drop_path'. Update factory to handle new drop args.
5 years ago
Ross Wightman
f1d5f8a6c4
Update comments for Selective Kernel and DropBlock/Path impl, add skresnet34 weights
5 years ago
Ross Wightman
569419b38d
Tweak some comments, add SKNet models with weights to sotabench, remove an unused branch
5 years ago
Ross Wightman
53c47479c4
Batch validation batch size adjustment, tweak L2 crop pct
5 years ago
Ross Wightman
08553e16b3
Merge branch 'master' into attention
5 years ago
Ross Wightman
fa38f24967
Update SK network configs, add weights for skresnet8 and skresnext50
5 years ago
Ross Wightman
ba15ca47e8
Add ported EfficientNet-L2, B0-B7 NoisyStudent weights from TF TPU
5 years ago
Ross Wightman
5e6dbbaf30
Add CBAM for experimentation
5 years ago
Ross Wightman
d725991870
Remove debug print from ECA module
5 years ago
Ross Wightman
2a7d256fd5
Re-enable mem-efficient/jit activations after torchscript tests
5 years ago
Ross Wightman
f902bcd54c
Layer refactoring continues, ResNet downsample rewrite for proper dilation in 3x3 and avg_pool cases
...
* select_conv2d -> create_conv2d
* added create_attn to create attention module from string/bool/module
* factor padding helpers into own file, use in both conv2d_same and avg_pool2d_same
* add some more test eca resnet variants
* minor tweaks, naming, comments, consistency
5 years ago
Ross Wightman
a99ec4e7d1
A bunch more layer reorg, splitting many layers into own files. Improve torchscript compatibility.
5 years ago
Ross Wightman
13746a33fc
Big move, layer modules and fn to timm/models/layers
5 years ago
Ross Wightman
f54612f648
Merge branch 'select_kernel' into attention
5 years ago
Ross Wightman
4defbbbaa8
Fix module name mistake, start layers sub-package
5 years ago
Ross Wightman
7011cd0902
A little bit of ECA cleanup
5 years ago
Ross Wightman
46471df7b2
Merge pull request #82 from VRandme/eca
...
ECA-Net Efficient Channel Attention
5 years ago
Ross Wightman
d0eb59ef46
Remove unused default_init for EfficientNets, experimenting with fanout calc for #84
5 years ago
Chris Ha
e6a762346a
Implement Adaptive Kernel selection
...
When channel size is given,
calculate adaptive kernel size according to original paper.
Otherwise use the given kernel size(k_size), which defaults to 3
5 years ago
Ross Wightman
13e8da2b46
SelectKernel split_input works best when input channels split like grouped conv, but output is full width. Disable zero_init for SK nets, seems a bad combo.
5 years ago
Chris Ha
6db087a1ff
Merge remote-tracking branch 'upstream/master' into eca
5 years ago
Chris Ha
904c618040
Update EcaModule.py
...
Make pylint happy
(commas, unused imports, missed imports)
5 years ago
Chris Ha
db91ba053b
EcaModule(CamelCase)
...
CamelCased EcaModule.
Renamed all instances of ecalayer to EcaModule.
eca_module.py->EcaModule.py
5 years ago
Ross Wightman
5c4991a088
Add PyTorch trained EfficientNet-ES weights from Andrew Lavin
5 years ago
Chris Ha
d63ae121d5
Clean up eca_module code
...
functionally similar
adjusted rwightman's version of reshaping and viewing.
Use F.pad for circular eca version for cleaner code
5 years ago
Chris Ha
f87fcd7e88
Implement Eca modules
...
implement ECA module by
1. adopting original eca_module.py into models folder
2. adding use_eca layer besides every instance of SE layer
5 years ago
Ross Wightman
7d07ebb660
Adding some configs to sknet, incl ResNet50 variants from 'Compounding ... Assembled Techniques' paper and original SKNet50
5 years ago
Ross Wightman
a9d2424fd1
Add separate zero_init_last_bn function to support more block variety without a mess
5 years ago
Ross Wightman
355aa152d5
Just leave it float for now, will look at fp16 later. Remove unused reference code.
5 years ago
Ross Wightman
ef457555d3
BlockDrop working on GPU
5 years ago
Ross Wightman
3ff19079f9
Missed nn_ops.py from last commit
5 years ago
Ross Wightman
9f11b4e8a2
Add ConvBnAct layer to parallel integrated SelectKernelConv, add support for DropPath and DropBlock to ResNet base and SK blocks
5 years ago
Ross Wightman
cefc9b7761
Move SelectKernelConv to conv2d_layers and more
...
* always apply attention in SelectKernelConv, leave MixedConv for no attention alternative
* make MixedConv torchscript compatible
* refactor first/previous dilation name to make more sense in ResNet* networks
5 years ago
Ross Wightman
9abe610931
Used wrong channel var for split
5 years ago
Ross Wightman
58e28dc7e7
Move Selective Kernel blocks/convs to their own sknet.py file
5 years ago
Ross Wightman
a93bae6dc5
A SelectiveKernelBasicBlock for more experiments
5 years ago
Ross Wightman
ad087b4b17
Missed bias=False in selection conv
5 years ago
Ross Wightman
c8b3d6b81a
Initial impl of Selective Kernel Networks. Very much a WIP.
5 years ago
Ross Wightman
91534522f9
Add newly added TF ported EfficientNet-B8 weights (RandAugment)
5 years ago
Ross Wightman
12dbc74742
New ResNet50 JSD + RandAugment weights
5 years ago
Ross Wightman
2f41905ba5
Update ResNet50 weights to AuxMix trained 78.994 top-1. A few commentes re 'tiered_narrow' tn variant.
5 years ago
Ross Wightman
d9a6a9d0af
Merge pull request #74 from rwightman/augmix-jsd
...
AugMix, JSD loss, SplitBatchNorm (Auxiliary BN), and more
5 years ago
Ross Wightman
3eb4a96eda
Update AugMix, JSD, etc comments and references
5 years ago
Ross Wightman
a28117ea46
Add tiered narrow ResNet (tn) and weights for seresnext26tn_32x4d
5 years ago
Ross Wightman
833066b540
A few minor things in SplitBN
5 years ago
Ross Wightman
7547119891
Add SplitBatchNorm. AugMix, Rand/AutoAugment, Split (Aux) BatchNorm, Jensen-Shannon Divergence, RandomErasing all working together
5 years ago
Ross Wightman
ec0dd4053a
Add updated RandAugment trained EfficientNet-B0 trained weights from @michaelklachko
5 years ago
Ross Wightman
40fea63ebe
Add checkpoint averaging script. Add headers, shebangs, exec perms to all scripts
5 years ago
Ross Wightman
53001dd292
ResNet / Res2Net additions:
...
* ResNet torchscript compat
* output_stride arg supported to limit network stride via dilations (support for dilation added to Res2Net)
* allow activation layer to be changed via act_layer arg
5 years ago
Ross Wightman
f96b3e5e92
InceptionResNetV2 torchscript compatible
5 years ago
Ross Wightman
19d93fe454
Add selecsls60 weights
5 years ago
Ross Wightman
0062c15fb0
Update checkpoint url with modelzoo compatible ones.
5 years ago
Ross Wightman
b5315e66b5
Streamline SelecSLS model without breaking checkpoint compat. Move cfg handling out of model class. Update feature/pooling behaviour to match current.
5 years ago
Ross Wightman
d59a756c16
Run PyCharm autoformat on selecsls and change mix cap variables and model names to all lower
5 years ago
Ross Wightman
fb3a0f4bb8
Merge pull request #65 from mehtadushy/selecsls
...
Incorporate SelecSLS Models
5 years ago
Ross Wightman
19fc205a4d
Update comments on the new SE-ResNeXt26 models
5 years ago
Ross Wightman
acc3ed2b8c
Add EfficientNet-B3 weights, trained from scratch with RA.
5 years ago
Dushyant Mehta
2404361f62
correct asset paths
5 years ago
Dushyant Mehta
31939311f6
Added SelecSLS models
5 years ago
rwightman
1f4498f217
Add ResNet deep tiered stem and model weights for seresnext26t_32x4d and seresnext26d_32x4d
5 years ago
Dushyant Mehta
32012a44fd
Added SelecSLS model
5 years ago
Ross Wightman
73b78459dc
Add update RandAugment MixNet-XL weights
5 years ago
Ross Wightman
ff421e5e09
New PyTorch trained EfficientNet-B2 weights with my RandAugment impl
5 years ago
Ross Wightman
3bef524f9c
Finish with HRNet, weights and models updated. Improve consistency in model classifier/global pool treatment.
5 years ago
Ross Wightman
6ca0828166
Update EfficientNet comments, MobileNetV3 non-TF create fns, fix factory arg checks, bump PyTorch version req to 1.2
5 years ago
Ross Wightman
eccbadca74
Update EfficientNet comments
5 years ago
Ross Wightman
902d32fb16
Renamed gen_efficientnet.py -> efficientnet.py
5 years ago
Ross Wightman
5a0a8de7e3
ResNet updates:
...
* remove redundant GluonResNet model/blocks and use the code in ResNet for Gluon weights
* change SEModules back to using AdaptiveAvgPool instead of mean, PyTorch issue long fixed
5 years ago
Ross Wightman
a39cc43374
Bring EfficientNet and MobileNetV3 up to date with my gen-efficientnet repo
...
* Split MobileNetV3 and EfficientNet model files and put builder and blocks in own files (getting too large)
* Finalize CondConv EfficientNet variant
* Add the AdvProp weights files and B8 EfficientNet model
* Refine the feature extraction module for EfficientNet and MobileNetV3
5 years ago
Ross Wightman
ad93347548
Initial HRNet classification model commit
5 years ago
Ross Wightman
2393708650
Missed stashing of out_indices in model
5 years ago
Ross Wightman
35e8f0c5e7
Fixup a few comments, add PyTorch version aware Flatten and finish as_sequential for GenEfficientNet
5 years ago
Ross Wightman
7ac6db4543
Missed activations.py
5 years ago
Ross Wightman
506df0e3d0
Add CondConv support for EfficientNet into WIP for GenEfficientNet Feature extraction setup
5 years ago
Ross Wightman
576d360f20
Bring in JIT version of optimized swish activation from gen_efficientnet as default (while working on feature extraction functionality here).
5 years ago
Ross Wightman
7b83e67f77
Pass drop connect arg through to EfficientNet models
5 years ago
Ross Wightman
0d58c50fb1
Add TF RandAug weights for B5/B7 EfficientNet models.
5 years ago
Ross Wightman
c099374771
Map pretrained checkpoint to cpu to avoid issue with some pretrained checkpoints still having CUDA tensors. Fixes #42
5 years ago
Ross Wightman
b93fcf0708
Add Facebook Research Semi-Supervised and Semi-Weakly Supervised ResNet model weights.
5 years ago
Ross Wightman
a9eb484835
Add memory efficient Swish impl
5 years ago
rwightman
d3ba34ee7e
Fix Mobilenet V3 model name for sotabench. Minor res2net cleanup.
5 years ago
Ross Wightman
2680ad14bb
Add Res2Net and DLA to README
5 years ago
rwightman
adbf770f16
Add Res2Net and DLA models w/ pretrained weights. Update sotabench.
5 years ago
Ross Wightman
aff194f42c
Merge pull request #32 from rwightman/opt
...
More optimizer work
5 years ago
Ross Wightman
3d9c8a6489
Add support for new AMP checkpointing support w/ amp.state_dict
5 years ago
Ross Wightman
e9d2ec4d8e
Merge pull request #31 from rwightman/opt
...
Optimizers and more
5 years ago
Ross Wightman
fac58f609a
Add RAdam, NovoGrad, Lookahead, and AdamW optimizers, a few ResNet tweaks and scheduler factory tweak.
...
* Add some of the trendy new optimizers. Decent results but not clearly better than the standards.
* Can create a None scheduler for constant LR
* ResNet defaults to zero_init of last BN in residual
* add resnet50d config
5 years ago
Ross Wightman
81875d52a6
Update sotabench model list, add Mean-Max pooling DPN variants, disable download progress
5 years ago
Ross Wightman
73fbd97ed4
Add weights for my MixNet-XL creation, include README updates for EdgeTPU models
5 years ago
Ross Wightman
51a2375b0c
Experimenting with a custom MixNet-XL and MixNet-XXL definition
5 years ago
Ross Wightman
9ec6824bab
Finally got around to adding EdgeTPU EfficientNet variant
5 years ago
Ross Wightman
aeaaad7304
Merge pull request #24 from rwightman/gluon_xception
...
Port Gluon Aligned Xception models
5 years ago
Ross Wightman
3b4868f6dc
A few more additions to Gluon Xception models to match interface of others.
5 years ago
Ross Wightman
4d505e0785
Add working Gluon Xception-65 model. Some cleanup still needed.
5 years ago
Minqin Chen
4e7a854dd0
Update helpers.py
...
Fixing out of memory error by loading the checkpoint onto the CPU.
5 years ago