-
Language model harnesses are compositional generalizers
Harnesses can lead to compositional generalization: we observe a property in training RLMs, in which similarly structured tasks are viewed as isomorphic and all individual LM calls in the harness become in-distribution.
-
A Mini Exercise on the Mismanaged Geniuses Hypothesis (RLMs on LongCoT)
We study an example of the Mismanaged Geniuses Hypothesis at play on the LongCoT benchmark
-
The Mismanaged Geniuses Hypothesis
We propose the mismanaged geniuses hypothesis, which posits that existing frontier language models are severely underutilized due to sub-optimal use of individual language model calls.
-
Language Models will be Scaffolds
The language models we interact with in the near future will be what we call scaffolds today.
-
Recursive Language Models
We propose Recursive Language Models (RLMs), an inference strategy where language models can decompose and recursively interact with input context of unbounded length through REPL environments.