Social media predictions

Prediction market criticism

Unfortunately, statistics—and the scientific process more generally—often seems to be used more as a way of laundering uncertainty, processing data until researchers and consumers of research can feel safe acting as if various scientific hypotheses are unquestionably true.

They consider prediction markets as a solution, but largely reject them for reasons both bad and not so bad. I’ll respond here to their article in unusual detail. First the bad:

Would prediction markets (or something like them) help? It’s hard to imagine them working out in practice. Indeed, the housing crisis was magnified by rampant speculation in derivatives that led to a multiplier effect.

Yes, speculative market estimates were mistaken there, as were most other sources, and mistaken estimates caused bad decisions. But speculative markets were the first credible source to correct the mistake, and no other stable source had consistently more accurate estimates. Why should the most accurate source should be blamed for mistakes made by all sources?

Allowing people to bet on the failure of other people’s experiments just invites corruption, and the last thing social psychologists want to worry about is a point-shaving scandal.

What about letting researchers who compete for grants, jobs, and publications write critical referee reports and publish criticism, doesn’t that invite corruption too? If you are going to forbid all conflicts of interest because they invite corruption, you won’t have much left you will allow. Surely you need to argue that bet incentives are more corrupting that other incentives.

And there are already serious ways to bet on some areas of science. Hedge funds, for instance, can short the stock of biotech companies moving into phase II and phase III trials if they suspect earlier results were overstated and the next stages of research are thus underpowered.

So by your previous argument, don’t you want to forbid such things because they invite corruption? You can’t have it both ways; either bets are good so you want more, or bets are bad so you want less, or you must distinguish the good from the bad somehow.

More importantly, though, we believe that what many researchers in social science in particular are more likely to defend is a general research hypothesis, rather than the specific empirical findings. On one hand, researchers are already betting—not just money (in the form of research funding) but also their scientific reputations—on the validity of their research.

No, the whole problem here that we’d like to solve is that scientific reputations are not tied very strongly to research validity. Folks often gain enviable reputations from publishing lots of misleading research.

On the other hand, published claims are vague enough that all sorts of things can be considered as valid confirmations of a theory (just as it was said of Freudian psychology and Marxian economics that they can predict nothing but explain everything).

Now we have a not-so-bad reason to avoid prediction markets: people are often unclear about what they mean, and they often don’t really want to be clear. And honestly, many of their patrons don’t want them to be clear either. We might create a prediction market on if what they meant will ever be clear. But they won’t want to pay for it, and others paying for it might just be mean.

Self-Regulatory Organizations; ICE Clear Europe Limited; Order Granting ..  — Insurance News Net
ICE Clear Europe states that the changes to the Risk Policy amend the calculation of CDS initial margin requirements to comply with margin requirements under EMIR Article 41 and Article 24 of the implementing Regulatory Technical Standards. /5/ ICE ..

Popular Q&A

avatar
How critical is it for a startup to be first to market?

Being first to market can be an extremely bold way to make your mark. However, as the first to do anything, you have to deal with all the various unknowns, which can be extremely difficult.  All decisions about your product will me made with little to no basis, unless there's an extensive internal effort to prototype and test.  And even then, there's no telling how something will perform in the real world.

Related Posts