Commit Graph

36 Commits (a25bf974a9a2c67c72fbb780c056c8638c966a83)

Author SHA1 Message Date
Ross Wightman 1186fc9c73 Merge remote-tracking branch 'origin/master' into bits_and_tpu
2 years ago
Ross Wightman bfc0dccb0e Improve image extension handling, add methods to modify / get defaults. Fix #1335 fix #1274.
2 years ago
Ross Wightman 9c321be330
Merge pull request #1239 from dedeswim/parser-fix
3 years ago
Edoardo Debenedetti c76d772670 Add support for different TFDS `BuilderConfig`s
3 years ago
Ross Wightman 95739b45d7 Fix partially removed alt_lable impl from TFDS variant of ImageNet22/12k
3 years ago
Ross Wightman 5e1be34a60 Add ImageNet-22k/12k TFDS dataset defs
3 years ago
Ross Wightman 59ffab537c Fix mistake in wds sample slicing
3 years ago
Ross Wightman ab16a358bb Add log and continue handler for WDS errors, fix args.num_gpu for validation script fallback
3 years ago
Ross Wightman 7eeaf521a0 use gopen in wds to open info file in case it's at a url/gs location
3 years ago
Ross Wightman 229ac6b8d8 Fix alternate label handling in WDS parser to skip invalid alt labels
3 years ago
Ross Wightman a444d4b891 Add alternative label support to WDS for imagenet22k/12k split, add 21k/22k/12k indices filters to results/
3 years ago
Ross Wightman da2796ae82 Add webdataset (WDS) support, update TFDS to make some naming in parsers more similar. Fix workers=0 compatibility. Add ImageNet22k/12k synset defs.
3 years ago
Ross Wightman c639a86c67 Change TFDS default to full re-shuffle (init) each epoch (for now)
3 years ago
Ross Wightman ff0f709c20 Testing TFDS shuffle across epochs
3 years ago
Ross Wightman cad170e494 Merge remote-tracking branch 'origin/norm_norm_norm' into bits_and_tpu
3 years ago
Ross Wightman 55adfbeb8d Add commented code to increase open file limit via Python (for TFDS dataset building)
3 years ago
Ross Wightman 809c7bb1ec Merge remote-tracking branch 'origin/master' into bits_and_tpu
3 years ago
Ross Wightman cfa414cad2 Matching two bits_and_tpu changes for TFDs wrapper
3 years ago
Ross Wightman 4f338556d8 Fixes and improvements for metrics, tfds parser, loader / transform handling
3 years ago
Ross Wightman 406c486ba2 Merge remote-tracking branch 'origin/more_datasets' into bits_and_tpu
3 years ago
Ross Wightman 9ec3210c2d More TFDS parser cleanup, support improved TFDS even_split impl (on tfds-nightly only currently).
3 years ago
Ross Wightman ba65dfe2c6 Dataset work
3 years ago
Ross Wightman 25d52ea71d Merge remote-tracking branch 'origin/fixes_bce_regnet' into bits_and_tpu
3 years ago
Ross Wightman b57a03bd0d Merge branch 'master' into bits_and_tpu
4 years ago
Ross Wightman 91ab0b6ce5 Add proper TrainState checkpoint save/load. Some reorg/refactoring and other cleanup. More to go...
4 years ago
Ross Wightman aa92d7b1c5 Major timm.bits update. Updater and DeviceEnv now dataclasses, after_step closure used, metrics base impl w/ distributed reduce, many tweaks/fixes.
4 years ago
Ross Wightman d53e91218e Fix tf.data options setting for newer TF versions
4 years ago
Ross Wightman 938716c753 Fix import issue, use devenv for dist info in parser_tfds
4 years ago
Ross Wightman f42f1df26c Improve evenness of per-worker split for validation set with TFDS
4 years ago
Ross Wightman 2db2d87ff7 Add epoch-repeats arg to multiply the number of dataset passes per epoch. Currently for iterable datasets (read TFDS wrapper) only.
4 years ago
Ross Wightman 22748f1a2d Convert samples/targets in ParserImageInTar to numpy arrays, slightly less mem usage for massive datasets. Add a few more se/eca model defs to resnet.py
4 years ago
Ross Wightman 5d4c3d0af3 Add enhanced ParserImageInTar that can read images from tars within tars, folders with multiple tars, etc. Additional comment cleanup.
4 years ago
Ross Wightman 855d6cc217 More dataset work including factories and a tensorflow datasets (TFDS) wrapper
4 years ago
Ross Wightman 59ec7e6a53 Merge branch 'master' into imagenet21k_datasets_more
4 years ago
Ross Wightman e35e9760a6 More work on dataset / parser split and imagenet21k (tar) support
4 years ago
Ross Wightman de6046e213 Initial commit for dataset / parser reorg to support additional datasets / types
4 years ago