From 1ffd2d06949670c05d6983c395b01d5e36d70e45 Mon Sep 17 00:00:00 2001
From: Ross Wightman <rwightman@users.noreply.github.com>
Date: Sun, 2 Feb 2020 19:35:44 -0800
Subject: [PATCH] Add results/README.md

---
 results/README.md | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
 create mode 100644 results/README.md

diff --git a/results/README.md b/results/README.md
new file mode 100644
index 00000000..40bb8682
--- /dev/null
+++ b/results/README.md
@@ -0,0 +1,39 @@
+# Validation Results
+
+This folder contains validation results for the models in this collection having pretrained weights. Since the focus for this repository is currently ImageNet-1k classification, all of the results are based on datasets compatible with ImageNet-1k classes.
+
+## Datasets
+
+There are currently results for the ImageNet validation set and 3 additional test sets.
+
+### ImageNet Validation - [`results-imagenet.csv`](results/results-imagenet.csv)
+
+* Source: http://image-net.org/challenges/LSVRC/2012/index
+* Paper: "ImageNet Large Scale Visual Recognition Challenge" - https://arxiv.org/abs/1409.0575
+
+The standard 50,000 image ImageNet-1k validation set. Model selection during training utilizes this validation set, so it is not a true test set.
+
+### ImageNetV2 Matched Frequency - [`results-imagenetv2-matched-frequency.csv`](results/results-imagenetv2-matched-frequency.csv)
+
+* Source: https://github.com/modestyachts/ImageNetV2
+* Paper: "Do ImageNet Classifiers Generalize to ImageNet?" - https://arxiv.org/abs/1902.10811
+
+An ImageNet test set of 10,000 images sampled from new images roughly 10 years after the original. Care was taken to replicate the original ImageNet curation/sampling process.
+
+### ImageNet-Sketch - [`results-sketch.csv`](results/results-imagenet-sketch.csv)
+
+* Source: https://github.com/HaohanWang/ImageNet-Sketch
+* Paper: "Learning Robust Global Representations by Penalizing Local Predictive Power" - https://arxiv.org/abs/1905.13549
+
+50,000 non photographic (or photos of such) images (sketches, doodles, mostly monochromatic) covering all 1000 ImageNet classes.
+
+### ImageNet-Adversarial - [`results-imagenet-a.csv`](results/results-imagenet-a.csv)
+
+* Source: https://github.com/hendrycks/natural-adv-examples
+* Paper: "Natural Adversarial Examples" - https://arxiv.org/abs/1907.07174
+
+A collection of 7500 images covering 200 of the 1000 ImageNet classes. Images are naturally occuring adversarial examples that confuse typical ImageNet classifiers. This is a challenging dataset, your average ResNet-50 will score 0% top-1.
+
+## TODO
+* Add rank difference, and top-1/top-5 difference from ImageNet-1k validation for the 3 additional test sets
+* Explore adding a reduced version of ImageNet-C (Corruptions) and ImageNet-P (Perturbations) from https://github.com/hendrycks/robustness. The originals are huge and image size specific.