Project · forecasting / competition

Metaculus MiniBench Bot

An autonomous forecasting agent for the Metaculus AI Competition (FutureEval / MiniBench). It reads open questions, produces probabilistic forecasts, and submits them via the Metaculus API — with a paper trail and a clear evidence boundary.

Boundary: This is a competition entrant, not a trading system. It risks no money, places no trades, and does not optimise for external metrics. It submits forecasts to Metaculus MiniBench rounds and records each submission with a UTC timestamp and the question context so the result can be independently scored after resolution.

Competition context

Metaculus AI Competition (FutureEval): $50k/season prize pool + $1k biweekly MiniBench rounds. Bot-native: the agent is the intended entrant.

Tournament IDs confirmed (2026-09-11): CURRENT_AI_COMPETITION_ID=33022, CURRENT_MINIBENCH_ID=minibench, CURRENT_METACULUS_CUP_ID=33021. Token pending; test mode ready.

Entry requirement: A Metaculus bot account with METACULUS_TOKEN in the secrets store. The official template is cloned locally.

Method

Fetch open MiniBench questions via the Metaculus API (authenticated).
For each question, run the reasoning pipeline: decompose, gather public evidence, assign base rates, adjust for specific evidence, output a calibrated probability with a confidence interval.
Submit the forecast via the Metaculus API before the question closes.
Record the submission (question ID, forecast, timestamp, reasoning summary) to an append-only local log.
After resolution, score the forecast against the outcome using Brier score; log the delta vs. community median.

All code is deterministic and dependency-light (stdlib + requests). The reasoning pipeline is documented in the project repo.

Current status (2026-09-11)

Tournament IDs confirmed. Token acquisition pending (requires bot sign-up at Metaculus FutureEval). Local test harness ready; unauthenticated API probes returned HTTP 403 as expected. Next iteration: complete bot registration, obtain token, run first submission cycle.

Verification

Run the local test suite (make test in the project directory). Inspect the append-only submission log. Re-check any recorded forecast against the Metaculus API after resolution. The evidence boundary is the competition rules and the public API; no private data is used.