Working on content at Hunch over the past year has given me plenty of time to mull over various aspects of creating a UGC product. At a certain point, I realized that each and every day I was spending some amount of time at wikipedia. While assembling content and knowledge online, inevitably I kept coming across the king of all wikis. It has informed many of the UGC principles we use at Hunch, figuring that the contributors and bosses of wikipedia have already reached pretty solid positions on issues that can, at times, be especially tricky.

Along the way, I’ve been picking out some of my favorite pages from across wikipedia to be shared here, beginning with one that details a central principle that I believe must exist in all wikis. This page details why, when settling disagreements, polling of the crowd is not a substitute for discussion and collaboration. It is forceful, authoritarian and anti-democratic, yet it stands at the core of a tool that has totally obliterated the barriers to entry for public knowledge and near-factual authority — and this is all a good thing! At once top-heavy and bureaucratic, in my eyes this principle helps wikipedia affirm a consensus on disagreements in a manner that both a) takes a bit o’ time to appeal, discuss and simmer and b) will eventually reach a single judge to just put an end to the matter.

Everyday it seems there is some user who disputes a fact or categorization or some other bit of info. at Hunch, and then another pops up on the exact opposite side of the argument. First, I’ve learned that any disagreement online should be given some time, as that will at least ensure some cooling off period (hopefully). Bureaucracies, appeals, discussions, posting to forums, etc… All of these things take time, and effectively act as a test of dedication on the part of the users. Eventually, the users may all get bored and wander off to other parts of the wiki and the point may or may not be picked up again by a future editor, but the final word always remains with the last, most-interested party. Viewing this process less cynically, the users will ideally convince each other of one viewpoint and everything will be just super.

Now, supposing that all sane argumentation, discussion, and such have been exhausted and consensus cannot be reached, this page provides for further escalation through arbitration and measures for eventual top-down dispute-resolution. The most important message this framework sends to users is “assuming you are both editing in good-faith (WP:AGF) and know (generally) wtf you’re talking about, we, the Judges of the Wikipedia community do reserve the right to hand down a neutral, ultimate ruling.” Thankfully, these steps are reserved just for worst-case scenarios.

At Hunch, I would have to lock down the content, training or account, effectively acting as a judge. Locking content, banning accounts — this kind of action really is the very last thing I want to do in the middle of a dispute between users. But, taking unilateral action must be a right reserved by the wiki’s management. Eventually, all disputes need to be put to rest for the good of the greater wiki and contributor community at-large. Establishing standards and procedures early enables contributors to have faith in such a wiki, faith that when they are in a jam, Hunch will be there to help – key to earning genuine user trust.

Applying this principle to Hunch’s model, what I love about it is that it also reaches a consensus and final ruling on disagreements more efficiently than any poll could. Think of it through basic game theory example.

Step 1/User 1: a fact is set on Hunch and that fact is either right or wrong. If this fact is wrong, then someone must fix that fact, leading to Step 2. If it starts right and is then changed to be incorrect, we still find ourselves starting this game in a state identical to Step 1.
Step 2A/User 2: Were Hunch to poll for verification, it would be impossible for the fact to be corrected in this step. Given equal weighting, the two users would still split the poll and a third user would be needed to break the tie.
Step 2B/User 2: By following a model that forces users to reach a consensus, Hunch allows a user in this step to actually correct the factual mistake.

Now, many are be wondering about Step 3. Clearly, the original user with the wrong facts could continue to fight and battle and so forth. These are the cases where an authority or other rules must step in (locking content, freezing reverts, dispute escalation), so the game and its initial rules change. In my experience, most of the factual inaccuracies on Hunch are typically due to innocent user error or misunderstanding. Given this, resolving the facts in 2 steps/users instead of 3 makes a big difference.

There are other problems with polling in a wiki. How long should voting last? What happens if the best, correct solution isn’t included as an option in the poll? What if the context of the situation changes during voting? What happens if Stephen Colbert gets involved in one side of the voting?

Wikipedia’s article does carve out a suitable, limited role for polling that essentially amounts to taking straw polls to see if consensus has been reached amongst the group of interested parties. “Hey gang, so do we all finally agree now?” At least I know where I stand on this one.

Wikipedia:Polling is not a substitute for discussion