Metaculus and medians
or, Scope-sensitive snafus in summing speculations
previously: Metaculus has some issues
(1)
Should I expect monkeypox to be a big deal for the world? Well, fortunately, Metaculus has a pair of questions that ask users to predict how many infections and deaths there will be in 2022:
203 users(!) made 817 predictions of infections, and Metaculus helpfully aggregates those into a "community prediction" of ~248k infections. 77 users made 180 predictions of deaths, with a community prediction of 541.
The y-axis is on a log scale (as are the predictors' distributions). This is a good choice! Whatever you expect the most-likely case to be, there's definitely a chance with things like this that one a misestimation or shift in one factor can make it bigger or smaller by a multiple, not just an additive amount.
What's not a good choice is to report the median outcome of the aggregate position as the "community prediction". This causes a headline reported value that is way too low. Like, four to seven times too low (at least for my intended purposes).
Because the predictors gave (and are scored on) probability distributions, Metaculus will happily give you an aggregate distribution, of which the 248k "community prediction" is the median scenario (the middle of the three dashed lines):
However, on the same plot, the aggregate distribution predicts a 10% chance of at least 4,950k infections. If it's 10% likely to be 5 million infections, then that's already lot more concerning than the 250k in the community prediction! And when I say I'm interested in how much monkeypox to expect, I