da2796ae82
Add webdataset (WDS) support, update TFDS to make some naming in parsers more similar. Fix workers=0 compatibility. Add ImageNet22k/12k synset defs.
3 years ago
a16ea1e355
Merge remote-tracking branch 'origin/norm_norm_norm' into bits_and_tpu
3 years ago
372ad5fa0d
Significant model refactor and additions:
...
* All models updated with revised foward_features / forward_head interface
* Vision transformer and MLP based models consistently output sequence from forward_features (pooling or token selection considered part of 'head')
* WIP param grouping interface to allow consistent grouping of parameters for layer-wise decay across all model types
* Add gradient checkpointing support to a significant % of models, especially popular architectures
* Formatting and interface consistency improvements across models
* layer-wise LR decay impl part of optimizer factory w/ scale support in scheduler
* Poolformer and Volo architectures added
3 years ago
fafece230b
Allow changing base lr batch size from 256 via arg
3 years ago
7148039f9f
Tweak base lr log
3 years ago
f82fb6b608
Add base lr w/ linear and sqrt scaling to train script
3 years ago
95cfc9b3e8
Merge remote-tracking branch 'origin/master' into norm_norm_norm
3 years ago
abc9ba2544
Transitioning default_cfg -> pretrained_cfg. Improving handling of pretrained_cfg source (HF-Hub, files, timm config, etc). Checkpoint handling tweaks.
3 years ago
f0f9eccda8
Add --fuser arg to train/validate/benchmark scripts to select jit fuser type
3 years ago
5ccf682a8f
Remove deprecated bn-tf train arg and create_model handler. Add evos/evob models back into fx test filter until norm_norm_norm branch merged.
3 years ago
4c8bb295ab
Remove bn-tf arg
3 years ago
4f338556d8
Fixes and improvements for metrics, tfds parser, loader / transform handling
...
* add back ability to create transform with loader
* change 'samples' -> 'examples' for tfds wrapper to match tfds naming
* add support for specifying feature names for input and target in tfds wrapper
* add class_to_idx for image classification datasets in tfds wrapper
* add accumulate_type to avg meters and metrics to allow float32 or float64 accumulation control with lower prec data
* minor cleanup, log output rate prev and avg
3 years ago
80ca078aed
Fix a few bugs and formatting/naming issues
...
* Pass optimizer resume flag through to checkpoint / updater restore. Related to #961 but not clear how relates to crash.
* Rename monitor step args, cleanup handling of step_end_idx vs num_steps for consistent log output in either case
* Resume from proper epoch (ie next epoch relative to checkpoint)
3 years ago
406c486ba2
Merge remote-tracking branch 'origin/more_datasets' into bits_and_tpu
3 years ago
ba65dfe2c6
Dataset work
...
* support some torchvision datasets
* improvements to TFDS wrapper for subsplit handling (fix #942 ), shuffle seed
* add class-map support to train (fix #957 )
3 years ago
cd638d50a5
Merge pull request #880 from rwightman/fixes_bce_regnet
...
A collection of fixes, model experiments, etc
3 years ago
d9abfa48df
Make broadcast_buffers disable its own flag for now (needs more testing on interaction with dist_bn)
3 years ago
80075b0b8a
Add worker_seeding arg to allow selecting old vs updated data loader worker seed for (old) experiment repeatability
3 years ago
908563d060
fix `use_amp`
...
Fix https://github.com/rwightman/pytorch-image-models/issues/881
3 years ago
25d52ea71d
Merge remote-tracking branch 'origin/fixes_bce_regnet' into bits_and_tpu
3 years ago
0387e6057e
Update binary cross ent impl to use thresholding as an option (convert soft targets from mixup/cutmix to 0, 1)
3 years ago
3581affb77
Update train.py with some flags related to scheduler tweaks, fix best checkpoint bug.
3 years ago
0639d9a591
Fix updated validation_batch_size fallback
3 years ago
5db057dca0
Fix misnamed arg, tweak other train script args for better defaults.
3 years ago
fb94350896
Update training script and loader factory to allow use of scheduler updates, repeat augment, and bce loss
3 years ago
f2e14685a8
Add force-cpu flag for train/validate, fix CPU fallback for device init, remove old force cpu flag for EMA model weights
3 years ago
0d82876132
Add comment for reference re PyTorch XLA 'race' issue
3 years ago
40457e5691
Transforms, augmentation work for bits, add RandomErasing support for XLA (pushing into transforms), revamp of transform/preproc config, etc ongoing...
3 years ago
7c19c35d9f
Global instead of local rank.
3 years ago
c3db5f5801
Worker hack for TFDS eval, add TPU env var setting.
3 years ago
f411724de4
Fix checkpoint delete issue. Add README about bits and initial Pytorch XLA usage on TPU-VM. Add some FIXMEs and fold train_cfg into train_state by default.
3 years ago
91ab0b6ce5
Add proper TrainState checkpoint save/load. Some reorg/refactoring and other cleanup. More to go...
4 years ago
5b9c69e80a
Add basic training resume based on legacy code
4 years ago
cbd4ee737f
Fix model init for XLA, remove some prints.
4 years ago
6d90fcf282
Fix distribute_bn and model_ema
4 years ago
aa92d7b1c5
Major timm.bits update. Updater and DeviceEnv now dataclasses, after_step closure used, metrics base impl w/ distributed reduce, many tweaks/fixes.
4 years ago
76de984a5f
Fix some bugs with XLA support, logger, add hacky xla dist launch script since torch.dist.launch doesn't work
4 years ago
12d9a6d4d2
First timm.bits commit, add initial abstractions, WIP updates to train, val... some of it working
4 years ago
e15e68d881
Fix #566 , summary.csv writing to pwd on local_rank != 0. Tweak benchmark mem handling to see if it reduces likelihood of 'bad' exceptions on OOM.
4 years ago
e685618f45
Merge pull request #550 from amaarora/wandb
...
Wandb Support
4 years ago
7c97e66f7c
Remove commented code, add more consistent seed fn
4 years ago
5772c55c57
Make wandb optional
4 years ago
f54897cc0b
make wandb not required but rather optional as huggingface_hub
4 years ago
f13f7508a9
Keep changes to minimal and use args.experiment as wandb project name if it exists
4 years ago
f8bb13f640
Default project name to None
4 years ago
3f028ebc0f
import wandb in summary.py
4 years ago
a9e5d9e5ad
log loss as before
4 years ago
624c9b6949
log to wandb only if using using wandb
4 years ago
00c8e0b8bd
Make use of wandb configurable
4 years ago
8e6fb861e4
Add wandb support
4 years ago
Ross Wightman