18 Commits (285771972eed07068c3beccce11d9e3439132257)

Author	SHA1	Message	Date
Ross Wightman	4f18d6dc5f	Fix logs in WDS parser	2 years ago
Ross Wightman	b8c8550841	Data improvements. Improve train support for in_chans != 3. Add wds dataset support from bits_and_tpu branch w/ fixes and tweaks. TFDS tweaks.	2 years ago
Ross Wightman	87939e6fab	Refactor device handling in scripts, distributed init to be less 'cuda' centric. More device args passed through where needed.	2 years ago
Ross Wightman	c88947ad3d	Add initial Hugging Face Datasets parser impl.	2 years ago
Ross Wightman	bfc0dccb0e	Improve image extension handling, add methods to modify / get defaults. Fix #1335 fix #1274.	2 years ago
Ross Wightman	55adfbeb8d	Add commented code to increase open file limit via Python (for TFDS dataset building)	3 years ago
Ross Wightman	cfa414cad2	Matching two bits_and_tpu changes for TFDs wrapper ... * change 'samples' -> 'examples' for tfds wrapper to match tfds naming * add class_to_idx for image classification datasets in tfds wrapper	3 years ago
Ross Wightman	9ec3210c2d	More TFDS parser cleanup, support improved TFDS even_split impl (on tfds-nightly only currently).	3 years ago
Ross Wightman	ba65dfe2c6	Dataset work ... * support some torchvision datasets * improvements to TFDS wrapper for subsplit handling (fix #942), shuffle seed * add class-map support to train (fix #957)	3 years ago
Ross Wightman	d53e91218e	Fix tf.data options setting for newer TF versions	4 years ago
Ross Wightman	f42f1df26c	Improve evenness of per-worker split for validation set with TFDS	4 years ago
Ross Wightman	2db2d87ff7	Add epoch-repeats arg to multiply the number of dataset passes per epoch. Currently for iterable datasets (read TFDS wrapper) only.	4 years ago
Ross Wightman	22748f1a2d	Convert samples/targets in ParserImageInTar to numpy arrays, slightly less mem usage for massive datasets. Add a few more se/eca model defs to resnet.py	4 years ago
Ross Wightman	5d4c3d0af3	Add enhanced ParserImageInTar that can read images from tars within tars, folders with multiple tars, etc. Additional comment cleanup.	4 years ago
Ross Wightman	855d6cc217	More dataset work including factories and a tensorflow datasets (TFDS) wrapper ... * Add parser/dataset factory methods for more flexible dataset & parser creation * Add dataset parser that wraps TFDS image classification datasets * Tweak num_classes handling bug for 21k models * Add initial deit models so they can be benchmarked in next csv results runs	4 years ago
Ross Wightman	59ec7e6a53	Merge branch 'master' into imagenet21k_datasets_more	4 years ago
Ross Wightman	e35e9760a6	More work on dataset / parser split and imagenet21k (tar) support	4 years ago
Ross Wightman	de6046e213	Initial commit for dataset / parser reorg to support additional datasets / types	4 years ago