12 Commits (multi-weight)

Author	SHA1	Message	Date
Ross Wightman	24720abe3b	Merge branch 'master' into attn_update	3 years ago
Ross Wightman	8642401e88	Swap botnet 26/50 weights/models after realizing a mistake in arch def, now figuring out why they were so low...	3 years ago
Ross Wightman	5f12de4875	Add initial AttentionPool2d that's being trialed. Fix comment and still trying to improve reliability of sgd test.	3 years ago
Ross Wightman	54e90e82a5	Another attempt at sgd momentum test passing...	3 years ago
Ross Wightman	fc894c375c	Another attempt at sgd momentum test passing...	3 years ago
Ross Wightman	708d87a813	Fix ViT SAM weight compat as weights at URL changed to not use repr layer. Fix #825. Tweak optim test.	3 years ago
Ross Wightman	c207e02782	MOAR optimizer changes. Woo!	3 years ago
Ross Wightman	42c1f0cf6c	Fix lars tests	3 years ago
Ross Wightman	a426511c95	More optimizer cleanup. Change all to no longer use .data. Improve (b)float16 use with adabelief. Add XLA compatible Lars.	3 years ago
Ross Wightman	a6af48be64	add madgradw optimizer	3 years ago
Ross Wightman	55fb5eedf6	Remove experiment from lamb impl	3 years ago
Ross Wightman	959eaff121	Add optimizer tests and update testing to pytorch 1.9	3 years ago