Lesswrong

So funny, right?

Until you have the awkward realization that - while simpler - it may, in the relevant senses, ultimately not be so different from how nature/evo has instilled in our genes/memes the same "I'm alive" (my 2022 story here...

tl;dr I continue work to apply Goodfire lab's manifold steering methods to the Gemma-2-2b model's natural state, both in raw residual space and in the space produced by Anthropic's pre-trained CLT features. I find that the model's natural residual state can be directly projected onto a pre-computed manifold and gradually steered to produce day-of-the-week tokens in the exp...

The Machine Stops (E M Forster, 1909) is an enormously prescient sci-fi dystopia. Everyone lives in isolation, with the titular Machine responsible providing for all their needs, and communication done via video calls. It’s incredibly accurate for predic...

Tl;dr: The frontier AI companies’ implicit alignment plans center on rushing towards RSI with something like current models. This plan is (at best) contentious, and many believe it will have catastrophic consequences. Nonetheless, it is the track we are on. It is therefore important to collect evidence which helps us anticipate how this will go.

—

Note: ...

LLMs are amazing at tasks where they have tight and legible feedback loops. Domains where experiments are quick, deterministic and cheap. I will call these domains "cheap-loop" domains.

Presumably, this is mostly a result of the dynamics of LLM training. It's easier to train models in domains where the AI lab can cheaply and quickly assign reward to them. The maj...