Discourse Plugin Upgrade: tracking wiki updates

[burrrata]

"Cred Historians" and Curators

Adding wiki edit tracking would be a feature for the Discourse plugin. I haven’t looked into the APIs for getting that data, but I suspect it would be the kind of feature that @Beanow or I could add with a couple days work. Feel free to write an initiative for it.

Creating an Initiative wish list item to brainstorm what this might look like.

Since Cred Historians and Curators spend a lot of time updating Wikis, we should track Wiki updates. This is useful for Initiatives, documentation and user guides, and really any information that is created and maintained by the community.

The tricky part here is how do you value wiki updates? Often the first draft of wikis is just a sketch and what makes them valuable is the iterative updates that occur over time. With those updates, however, fixing a typo is very different from updating documentation or adding a reference to an initiative. Since a wiki is a set of data and modifying a wiki updates that data. This is somewhat similar to a pull request where you contribute additional data or a change to a dataset. Discourse tracks the history of a wiki over time, and GitHub tracks the history of a repo over time. Perhaps the mechanics of tracking wiki updates might be similar to tracking the value of PRs? This way editing a typo would be weighted very differently than adding an important reference or something.

[decentralion]

Yeah, in principle we have the same problem with wiki edits as we do with GitHub PRs; a PR could be a typo fix or could be a super important refactor.

My solution with the PRs is (mostly) that we should put in some manual information by linking to important PRs from initiatives; the typo fix either won’t be linked or will be linked with very low weight, while the important refactor will have stronger connections to the supernodes.

This won’t work as well for wiki edits because individual wiki edits are harder to reference and categorize. For now we could go for a three step process:

1. Initially, every wiki edit gets an edge from the content in question. This will incentivize making many tiny changes and be pretty easy to game; we’ll need to rely on something like the honor system to regulate it.
2. Afterwards, we can add a heuristic that the strength of the edge is based on # of lines changed in the wiki, or such. This would be harder to game since it’s more obvious if you make huge nonsense changes to a wiki, although it will still create bad incentives to churn with needless re-writes.
3. Afterwards, we can come up with a cleverer system for evaluating wiki edits that involves more review by humans-in-the-loop.

Note that for the special case of initiatives and artifacts and such, I expect that a lot of the work will move out of the Discourse and into a dedicated UI within 2020.

[burrrata]

@decentralion

For now we could go for a three step process:

1. Initially, every wiki edit gets an edge from the content in question. This will incentivize making many tiny changes and be pretty easy to game; we’ll need to rely on something like the honor system to regulate it.
2. Afterwards, we can add a heuristic that the strength of the edge is based on # of lines changed in the wiki, or such. This would be harder to game since it’s more obvious if you make huge nonsense changes to a wiki, although it will still create bad incentives to churn with needless re-writes.
3. Afterwards, we can come up with a cleverer system for evaluating wiki edits that involves more review by humans-in-the-loop.

We’re still in Trust Level 1 so that works. In the future we could set it up so that Historians/Curators with high trust levels review wiki edits and flag or boost them as seems necessary.