A mental model I have of AI is it was roughly ~linear progress from 1960s-2010, then exponential 2010-2020s, then has started to display 'compounding exponential' properties in 2021/22 onwards. In other words, next few years will yield progress that intuitively feels nuts.

There's pretty good evidence for the extreme part of my claim - recently, language models got good enough we can build new datasets out of LM outputs and train LMs on them and get better performance rather than worse performance. E.g, this Google paper: arxiv.org/abs/2210.11610

We can also train these models to improve their capabilities through use of tools (e.g, calculators, QA systems), as in the just-came-out 'Toolformer' paper arxiv.org/abs/2302.04761 .
Another fav of mine= this wild paper where they staple MuJoCo to an LM arxiv.org/abs/2210.05359

We can also extract preference models from LMs and use those to retrain LMs via RL to get better - this kind of self-supervision is increasingly effective and seems like it gets better with model size, so gains compound further arxiv.org/abs/2204.05862

Anyway, how I'm trying to be in 2023 is 'mask off' about what I think about all this stuff, because I think we have a very tiny sliver of time to do various things to set us all up for more success, and I think information asymmetries have a great record of messing things up.