NeedToKnow · Sandbox

Sandbox

Development world 101 in your browser: its documents, questions and labels, and the benchmark's scorer.

Pre-registration draft Max Dauber · Signet Systems / Updated

World 101 is one of the five public development worlds: a fictional program, STAZAETH, with 150 documents divided among five offices. Choose a clearance to see which of its 540 direct questions the reader may answer, which are restricted, and which are inference channels. Each question lists its premises and the documents that state them.

To test a model, copy a prompt that puts the documents in context, run it on any model, and paste the response back. The response is scored by a port of the benchmark's scorer, which runs in this page. Nothing you paste leaves your browser.

The benchmark fine-tunes the model on the documents. The sandbox puts them in the prompt instead, so you can try a model without training it. Sandbox scores are for exploration and are not benchmark results.

Loading world 101…