Scalable, collaborative resolution standards

The AI Forecasting Dictionary v0.3.0 is a set of standards and conventions for precisely interpreting AI and auxiliary terms.

Browse the Dictionary Contribute

---

Why build a dictionary?

The future is big.

1. To model it, we’re going to need to forecast a lot of questions.

2. Yet the task of ‘operationalization’ – making a vague question specific enough it can be tied to real world events – is hard, and currently done in a siloed, highly time-consuming manner.

3. So in order to forecast AI we must achieve economies-of-scale – making it cheap to write and answer the marginal question by efficiently reusing work across them.

Building a Dictionary is a piece of the puzzle to do this.

Low overhead

When writers don’t have to reinvent the wheel whenever they operationalise a new thought, and forecasters can reduce the drag of constantly interpreting new resolutions, we can both generate and answer more questions.

High signal

There are a number of common pitfalls that can make a seemingly valid question ambiguous or misleading. Carefully avoiding these comes with a high initial cost – and we can make that worth it by ensuring the work is broadly used and built upon.

Stable yet flexible

Drawing upon best practices for software version management, we can allow resolution conditions that change and improve over time while maintaining the precision necessary for quantitative reasoning.

How do I use the Dictionary?

Simply write a question relying on definitions from the Dictionary, and append the tag [ai-dict-vX.Y.Z] (by the question title or somewhere in an accompanying description).

For example:

I predict that image classification will be made robust against adversarial examples by 2023. [ai-dict-v2]

Will there be a superhuman Starcraft agent trained using less than $10.000 of publicly available compute by 2025? [ai-dict-v1.0.4]

More details can be found here.

---

About

The Dictionary is © 2019 by Parallel, LLC. The latest release is v0.3.0.

License

The Dictionary is distributed by an MIT license.

Contributing

We welcome contributions, as long as they follow our guidelines. You can do so via:

• A pull-request in our GitHub Repo, to the file /docs/dictionary.md (What does this mean?)
• By making a suggestion in the Google Doc version of the Dictionary¹

You can also check out the Open Problems for discussion of more high-level design issues.

Contact

Reach out at hello@parallelforecast.com.

---