9 Commits (0262a0e8e16c31a4e8157a44dafbe5f6b8c21495)

Author	SHA1	Message	Date
Alexander Soare	b25ff96768	wip - pre-rebase	3 years ago
Alexander Soare	bc3d4eb403	wip -rebase	3 years ago
Ross Wightman	e2b8d44ff0	Halo, bottleneck attn, lambda layer additions and cleanup along w/ experimental model defs ... * align interfaces of halo, bottleneck attn and lambda layer * add qk_ratio to all of above, control q/k dim relative to output dim * add experimental haloregnetz, and trionet (lambda + halo + bottle) models	3 years ago
Ross Wightman	b1c2e3eb92	Match rel_pos_indices attr rename in conv branch	3 years ago
Ross Wightman	b49630a138	Add relative pos embed option to LambdaLayer, fix last transpose/reshape.	3 years ago
Ross Wightman	5bd04714e4	Cleanup weight init for byob/byoanet and related	3 years ago
Ross Wightman	0721559511	Improved (hopefully) init for SA/SA-like layers used in ByoaNets	3 years ago
Ross Wightman	e15c3886ba	Defaul lambda r=7. Define '26t' stage 4/5 256x256 variants for all of bot/halo/lambda nets for experiment. Add resnet50t for exp. Fix a few comments.	3 years ago
Ross Wightman	ce62f96d4d	ByoaNet with bottleneck transformer, lambda resnet, and halo net experiments	4 years ago