Why do fact-based debate platforms keep failing?
Why do fact-based debate platforms keep failing?
I’ve been experimenting with a small MVP that tries to structure online debates differently.
The idea is simple:
one claim/theory
users add individual supporting or opposing facts (with sources)
each fact is discussed and voted on independently
no final verdicts, no “truth score”, no authority layer
The goal is not to determine truth, but to observe how collective belief and disagreement form when arguments are forced to be atomic.
After sharing this experiment with skeptic-oriented communities, I ran into a set of strong critiques that seem to recur whenever projects like this appear:
Voting is inherently argumentum ad populum, even if applied to individual facts
There’s a strong asymmetry of effort: real evidence is costly, bad evidence is cheap
Coordinated actors, cranks, or propagandists are more motivated than average users
Non-experts struggle to distinguish relevance, quality, and weight of evidence
“Fact overload” and gish gallop can drown out meaningful signal
Moderation only works with subject-matter experts, which doesn’t scale
Similar platforms have failed when public voting elevated weak or misleading evidence over rigorous research
Many commenters argued that this model inevitably legitimizes misinformation rather than containing it.
Before taking this experiment any further, I’d really like input from people here who’ve seen similar systems succeed or fail.
My questions:
Is this kind of structure fundamentally doomed outside of peer review or expert-only contexts?
Are there known constraints or design patterns that prevent collapse into noise or popularity contests?
Does this only work in narrow, technical domains (e.g. software, math, engineering)?
Or is the failure mode intrinsic to letting non-experts evaluate evidence at all?
If it helps to see the concrete implementation, the MVP is here (no signup required): https://fact2check.com
I’m less interested in defending the project than in understanding where - structurally - this approach breaks.