Post by Matt Clifford: Great thread on an approach to scaling token windows to >1m tokens. New important ideas emerging ever... - Upcarta

Post

Matt Clifford @matthewclifford · Apr 24, 2023

From Twitter

Great thread on an approach to scaling token windows to >1m tokens. New important ideas emerging every day…

Tweet Apr 24, 2023

Can a new transformer architecture with 2 million 🚀 tokens fix the token window problem? Or are there no free lunches? What do we gain, what do we give up? New paper from @MikhailBurtsev, Aydar Bulatov and @yurakuratov Let's break it down👇 arxiv

by Hrishi Olickel

Recommended by 1 person

1 mention

Replies

No replies yet