How the forecasts work.

What we forecast#

• Whether a trial meets its main goal: a probability from 0 to 100%.
• How big the effect will be (coming next), for trials that use a standard measure (for example tumour response, or a skin-clearance score), with a range.

Which trials#

Randomized phase 2 and phase 3 trials of a treatment or prevention, from any sponsor, with an arm registered as the control (placebo, sham, usual care or an existing treatment), whose first main outcome is about benefit rather than safety, dosing or drug levels, that finished after September 2024 or finish by the end of 2027, and have not posted results. Company-sponsored trials are also scored on their own, because their results are the most reliably posted.

What counts as meeting the goal#

The trial's first pre-specified main outcome shows a statistically significant benefit of the new treatment over the control, at the threshold the trial itself set (usually 5%). For trials designed to show a new treatment is no worse than an existing one, the goal is met when the stated margin is met. When the rules can't decide, a person reads the result; we never guess.

Where results come from#

Results posted on the public trial registries, above all ClinicalTrials.gov, where trials with a US site must post results within about a year of finishing. For now only registry results are used; company announcements and journal articles may later be used to confirm them. A trial that never posts a result stays unresolved and is not counted as a failure.

Rules that keep the model honest#

• Only the past. A forecast may use only information dated before the forecast was made.
• No hidden hindsight. When we test the model on trials that have already reported, we leave out anything that changes after a trial starts (final enrollment, status changes) and any source that already knows how a drug turned out.
• No memorized answers. AI language models may have read old trial results during training, so headline results come only from live forecasts and from trials that reported after the models were trained.
• Sealed, then revealed. Every forecast is fingerprinted the day it is made (details).

What a forecast is built from#

How often similar trials succeed (same phase, disease area and design); earlier trials of the same drug and of drugs acting the same way; the biology of the drug's target, including human genetic evidence; and the rest of the published literature. Each piece of evidence is weighted by how strong the study behind it was.

Known limits#

• Trials run only outside the US post results far less often, so they resolve less often.
• Published evidence leans toward positive results; the model has to correct for that.
• A failed trial does not always mean a drug doesn't work: the dose, the patients or the size of the trial may have been wrong.