From 137292644b8a959add48d4d63663d2f9fefbab21 Mon Sep 17 00:00:00 2001 From: Ross Wightman Date: Fri, 13 May 2022 15:28:04 -0700 Subject: [PATCH] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f90c6abd..cd9b6071 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,7 @@ I'm fortunate to be able to dedicate significant time and money of my own suppor ### May 13, 2022 * Official Swin-V2 models and weights added from (https://github.com/microsoft/Swin-Transformer). Cleaned up to support torchscript. * Some refactoring for existing `timm` Swin-V2-CR impl, will likely do a bit more to bring parts closer to official and decide whether to merge some aspects. -* More Vision Transformer relative position / residual post-norm experiments w/ 512 dim +* More Vision Transformer relative position / residual post-norm experiments w/ 512 dim (all trained on TPU thanks to TRC program) * `vit_relpos_small_patch16_224` - 81.5 @ 224, 82.5 @ 320 -- rel pos, layer scale, no class token, avg pool * `vit_relpos_medium_patch16_rpn_224` - 82.3 @ 224, 83.1 @ 320 -- rel pos + res-post-norm, no class token, avg pool * `vit_relpos_medium_patch16_224` - 82.5 @ 224, 83.3 @ 320 -- rel pos, layer scale, no class token, avg pool