e2b8d44ff0
Halo, bottleneck attn, lambda layer additions and cleanup along w/ experimental model defs
...
* align interfaces of halo, bottleneck attn and lambda layer
* add qk_ratio to all of above, control q/k dim relative to output dim
* add experimental haloregnetz, and trionet (lambda + halo + bottle) models
3 years ago
0ca687f224
Make 'regnetz' model experiments closer to actual RegNetZ, bottleneck expansion, expand from in_chs, no shortcut on stride 2, tweak model sizes
3 years ago
6478bcd02c
Fix regnetz_d conv layer name, use inception mean/std
3 years ago
8e11da0ce3
Add experimental RegNetZ(ish) models for training / perf trials.
3 years ago
4027412757
Add resnet33ts weights, update resnext26ts baseline weights
3 years ago
5bd04714e4
Cleanup weight init for byob/byoanet and related
3 years ago
484e61648d
Adding the attn series weights, tweaking model names, comments...
3 years ago
8449ba210c
Improve performance of HaloAttn, change default dim calc. Some cleanup / fixes for byoanet. Rename resnet26ts to tfs to distinguish (extra fc).
3 years ago
a8b65695f1
Add resnet26ts and resnext26ts models for non-attn baselines
3 years ago
a5a542f17d
Fix typo
3 years ago
925e102982
Update attention / self-attn based models from a series of experiments:
...
* remove dud attention, involution + my swin attention adaptation don't seem worth keeping
* add or update several new 26/50 layer ResNe(X)t variants that were used in experiments
* remove models associated with dead-end or uninteresting experiment results
* weights coming soon...
3 years ago
392368e210
Add efficientnetv2_rw_t defs w/ weights, and gc variant, as well as gcresnet26ts for experiments. Version 0.4.13
3 years ago
d17b374f0f
Minimum input_size needed to be higher
3 years ago
b3b90d944d
Add min_input_size to bat_resnext to prevent test breakage.
3 years ago
10d8fa4620
Add gc and bat attention resnext26ts variants to byob for test.
3 years ago
307a935b79
Add non-local and BAT attention. Merge attn and self-attn factories into one. Add attention references to README. Add mlp 'mode' to ECA.
4 years ago
9611458e19
Throw in some FBNetV3 code I had lying around, some refactoring of SE reduction channel calcs for all EffNet archs.
4 years ago
02f9d4bc34
Add weights for resnet51q model, add 61q def.
4 years ago
f615474be3
Fix broken test, repvgg block doesn't have attn_last attr.
4 years ago
742c2d5247
Add Gather-Excite and Global Context attn modules. Refactor existing SE-like attn for consistency and refactor byob/byoanet for less redundancy.
4 years ago
9a3ae97311
Another set of byoanet models w/ ECA channel + SA + groups
4 years ago
b3d7580df1
Update ByoaNet comments. Fix first Steam feat chs for ByobNet.
4 years ago
7c97e66f7c
Remove commented code, add more consistent seed fn
4 years ago
ce62f96d4d
ByoaNet with bottleneck transformer, lambda resnet, and halo net experiments
4 years ago
6853b07bbd
Improve RegVGG block identity/vs non for clariy and fix attn usage. Add comments.
4 years ago
dc85e5a237
Add ByobNet w/ GPU-EfficientNets and RepVGG. Also add classic vgg models.
4 years ago
Ross Wightman