3 Commits (3599c7e6a4b781cb6147f0cbdceb2e455c36fe03)

Author	SHA1	Message	Date
Ross Wightman	43aa84e861	Add 'fast' layer norm that doesn't cast to float32, support APEX LN impl for slight speed gain, update norm and act factories, tweak SE for ability to disable bias (needed by GCVit)	2 years ago
Ross Wightman	307a935b79	Add non-local and BAT attention. Merge attn and self-attn factories into one. Add attention references to README. Add mlp 'mode' to ECA.	3 years ago
Ross Wightman	742c2d5247	Add Gather-Excite and Global Context attn modules. Refactor existing SE-like attn for consistency and refactor byob/byoanet for less redundancy.	3 years ago