Commit Graph

11 Commits (b1b024dfed5870cb1238c8c7c326d6ea8ef67300)

Author SHA1 Message Date
Ross Wightman b8c8550841 Data improvements. Improve train support for in_chans != 3. Add wds dataset support from bits_and_tpu branch w/ fixes and tweaks. TFDS tweaks.
2 years ago
Ross Wightman 87939e6fab Refactor device handling in scripts, distributed init to be less 'cuda' centric. More device args passed through where needed.
2 years ago
Ross Wightman 55adfbeb8d Add commented code to increase open file limit via Python (for TFDS dataset building)
3 years ago
Ross Wightman cfa414cad2 Matching two bits_and_tpu changes for TFDs wrapper
3 years ago
Ross Wightman 9ec3210c2d More TFDS parser cleanup, support improved TFDS even_split impl (on tfds-nightly only currently).
3 years ago
Ross Wightman ba65dfe2c6 Dataset work
3 years ago
Ross Wightman d53e91218e Fix tf.data options setting for newer TF versions
4 years ago
Ross Wightman f42f1df26c Improve evenness of per-worker split for validation set with TFDS
4 years ago
Ross Wightman 2db2d87ff7 Add epoch-repeats arg to multiply the number of dataset passes per epoch. Currently for iterable datasets (read TFDS wrapper) only.
4 years ago
Ross Wightman 5d4c3d0af3 Add enhanced ParserImageInTar that can read images from tars within tars, folders with multiple tars, etc. Additional comment cleanup.
4 years ago
Ross Wightman 855d6cc217 More dataset work including factories and a tensorflow datasets (TFDS) wrapper
4 years ago