The Forecasting Revolution

Just as Philip Tetlock discovered dramatic differences in forecasting ability that transformed political prediction, we believe exceptional research forecasters exist in machine learning. These researchers can predict not just whether a hypothesized solution will work, but how well it will perform—potentially revolutionizing how we conduct research by filtering promising ideas before costly implementation.

10x Potential Time Savings
20-30 Recent Papers Tested
Continuous Prediction Values

Core Research Questions

Expert Identification

Are there researchers in machine learning whose skill in predicting experiment effectiveness and generating solutions enables a fundamentally different research process?

Idea Filtration

Can high-quality forecasters dramatically reduce time to progress by predicting which experiment ideas are likely to perform well before implementation?

Risk Assessment

How do we balance filtering out predictably weak ideas while preserving breakthrough discoveries that conflict with expert intuitions?

Forecasting Methodology

Our approach builds on Tetlock's superforecasting principles, adapted for research prediction:

{ "prediction_target": "Continuous performance improvement on established benchmarks", "baseline_approach": "Reference class comparison with outside view first", "decomposition": "Break problems into tractable causal sub-components", "calibration": "Balance under/overconfidence through nuanced uncertainty", "evaluation": "Degrees of error measurement, not binary success/failure" }

Training Protocol

Duration: 1-2 hour forecasting course
Method: Randomized control trial with trained vs. untrained researchers

Prediction Targets

Benchmark progress, specific experiment outcomes, and research trend forecasting using recent published results with unknown outcomes.

Implementation Roadmap

Forecasting Tournament

Kaggle-style competition with leaderboards tracking the most effective research predictors, with potential financial incentives and market mechanisms.

Expert Collaboration

Partnership opportunities with Philip Tetlock for forecasting methodology and Ray Kurzweil for long-term AI prediction expertise.

Market Applications

Infrastructure for selling predictions to VCs and feeding technological forecasts into stock trading algorithms for breakthrough technologies.

Join the Research

Take the Prediction Quiz

Test your current forecasting abilities by predicting research outcomes on recent ML papers with established benchmarks.

Complete Forecasting Training

Participate in our 1-2 hour superforecasting course covering reference class comparison, decomposition, and calibration techniques.

Join Prediction Tournament

Compete in ongoing forecasting challenges to identify the most skilled research predictors and validate forecasting effectiveness.

Apply to Real Research

High-performing forecasters can begin making predictions on unimplemented experiment ideas to accelerate research progress.

The Vision

Just as medicine transformed from ignorance and confidence in the 1800s to evidence-based practice, research prediction can evolve from intuition to systematic forecasting. We're building the infrastructure to identify exceptional research forecasters and revolutionize how scientific progress is made.