From 1a934d2a5e630329d650803d9196bf8045ef6c18 Mon Sep 17 00:00:00 2001
From: Benjamin Bossan <benjamin.bossan@gmail.com>
Date: Thu, 23 Feb 2023 14:11:03 +0100
Subject: [PATCH] Add pytest-cov, requirements-dev, pyproject.toml

When tests finish, a report should be printed that shows the code
coverage of timm. This should give us a better idea where we should work
on test coverage.

I have tested this locally (on a subset of tests) and it worked.

Since the number of test dependencies was getting quite high, I created
a requirements-dev.txt and moved them there. GH action and
CONTRIBUTING.md are adjusted accordingly.

Furthermore, instead of extending the pytest invocation, I created a
pyproject.toml and added the coverage options there. For completeness, I
also added the black settings that come closest to the style of timm.
LMK if this is not desired.

For now, the coverage is only reported but not enforced. I.e. when a PR
is created that adds uncovered lines, CI will still succeed. We could
think about adding codecov or something like that, but it can be
annoying sometimes and the service was flaky for me in the past.
---
 .github/workflows/tests.yml |  2 +-
 CONTRIBUTING.md             |  9 ++-------
 pyproject.toml              | 12 ++++++++++++
 requirements-dev.txt        |  6 ++++++
 4 files changed, 21 insertions(+), 8 deletions(-)
 create mode 100644 pyproject.toml
 create mode 100644 requirements-dev.txt

diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
index 70352d0a..297b75bd 100644
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -30,7 +30,7 @@ jobs:
     - name: Install testing dependencies
       run: |
         python -m pip install --upgrade pip
-        pip install pytest pytest-timeout pytest-xdist pytest-forked expecttest
+        pip install -r requirements-dev.txt
     - name: Install torch on mac
       if: startsWith(matrix.os, 'macOS')
       run: pip install --no-cache-dir torch==${{ matrix.torch }} torchvision==${{ matrix.torchvision }}
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 91c13ffe..aa720c98 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -73,18 +73,13 @@ There are a LOT of gaps in current documentation relative to the functionality i
 
 # Installation
 
-Create a Python virtual environment using Python 3.10. Inside the environment, install the following test dependencies:
-
-```
-python -m pip install pytest pytest-timeout pytest-xdist pytest-forked expecttest
-```
-
-Install `torch` and `torchvision` using the instructions matching your system as listed on the [PyTorch website](https://pytorch.org/).
+Create a Python virtual environment using Python 3.10. Inside the environment, install torch` and `torchvision` using the instructions matching your system as listed on the [PyTorch website](https://pytorch.org/).
 
 Then install the remaining dependencies:
 
 ```
 python -m pip install -r requirements.txt
+python -m pip install -r requirements-dev.txt  # for testing
 python -m pip install --no-cache-dir git+https://github.com/mapillary/inplace_abn.git
 python -m pip install -e .
 ```
diff --git a/pyproject.toml b/pyproject.toml
new file mode 100644
index 00000000..05a5082b
--- /dev/null
+++ b/pyproject.toml
@@ -0,0 +1,12 @@
+[tool.pytest.ini_options]
+addopts = "--cov=timm --cov-report=term-missing"
+
+[tool.coverage.run]
+omit = [
+    "tests/test_*.py",
+]
+
+[tool.black]
+line-length = 120
+target-version = ['py37', 'py38', 'py39', 'py310', 'py311']
+skip-string-normalization = true
diff --git a/requirements-dev.txt b/requirements-dev.txt
new file mode 100644
index 00000000..59574f32
--- /dev/null
+++ b/requirements-dev.txt
@@ -0,0 +1,6 @@
+pytest
+pytest-timeout
+pytest-xdist
+pytest-forked
+expecttest
+pytest-cov