Ross Wightman
bfc0dccb0e
Improve image extension handling, add methods to modify / get defaults. Fix #1335 fix #1274 .
3 years ago
Ross Wightman
55adfbeb8d
Add commented code to increase open file limit via Python (for TFDS dataset building)
3 years ago
Ross Wightman
cfa414cad2
Matching two bits_and_tpu changes for TFDs wrapper
...
* change 'samples' -> 'examples' for tfds wrapper to match tfds naming
* add class_to_idx for image classification datasets in tfds wrapper
3 years ago
Ross Wightman
9ec3210c2d
More TFDS parser cleanup, support improved TFDS even_split impl (on tfds-nightly only currently).
3 years ago
Ross Wightman
ba65dfe2c6
Dataset work
...
* support some torchvision datasets
* improvements to TFDS wrapper for subsplit handling (fix #942 ), shuffle seed
* add class-map support to train (fix #957 )
3 years ago
Ross Wightman
d53e91218e
Fix tf.data options setting for newer TF versions
4 years ago
Ross Wightman
f42f1df26c
Improve evenness of per-worker split for validation set with TFDS
4 years ago
Ross Wightman
2db2d87ff7
Add epoch-repeats arg to multiply the number of dataset passes per epoch. Currently for iterable datasets (read TFDS wrapper) only.
4 years ago
Ross Wightman
22748f1a2d
Convert samples/targets in ParserImageInTar to numpy arrays, slightly less mem usage for massive datasets. Add a few more se/eca model defs to resnet.py
4 years ago
Ross Wightman
5d4c3d0af3
Add enhanced ParserImageInTar that can read images from tars within tars, folders with multiple tars, etc. Additional comment cleanup.
4 years ago
Ross Wightman
855d6cc217
More dataset work including factories and a tensorflow datasets (TFDS) wrapper
...
* Add parser/dataset factory methods for more flexible dataset & parser creation
* Add dataset parser that wraps TFDS image classification datasets
* Tweak num_classes handling bug for 21k models
* Add initial deit models so they can be benchmarked in next csv results runs
4 years ago
Ross Wightman
59ec7e6a53
Merge branch 'master' into imagenet21k_datasets_more
4 years ago
Ross Wightman
e35e9760a6
More work on dataset / parser split and imagenet21k (tar) support
4 years ago
Ross Wightman
de6046e213
Initial commit for dataset / parser reorg to support additional datasets / types
4 years ago