Preliminary CredSperiment Cred

[Beanow]

@decentralion

A better approach may be to have the value of the PR be independent of the activity, but the cred from the PR flows out based on the activity. So a PR that merges without any fuss gives most of the cred to the author; a PR that involves a long conversation prior to merge splits the cred between the participants. Of course this would introduce its own issues (e.g. people incentivized to bikeshed so they can take cred from a PR) but for now I think it would be a better heuristic.

I would like to suggest one bit of nuance to this. I feel like the value of a PR is:

• The value of the commits (accepted or not).
• Having gone through the collaborative process to improve it’s quality and support base within the community.

Hence I feel like a “no fuss” PR being valued the most does not incentivize the collaboration side of things. PR reviews in particular I feel are valuable, but I do think there is a point where there is a diminishing returns effect. 10 reviews does not add much over 3 reviews and 100 comments doesn’t add much over 10 succinct ones.

So I wouldn’t encourage a system of: each PR is 5 cred, share between all interaction. I would suggest the first 3 reviews and 10 comments are additive. From there on it’s split between interaction or some kind of diminishing returns curve for their value. Incentives wise I feel like this creates space for dedicated / regular reviewing roles. Of course, ideally this would be something you can give weight to suit your community.

---

@decentralion

I think the best approach will be to come up with a simple system for categorizing PRs by kind (“feature-work”, “bugfix”, “refactor”, “documentation”, “build”, “misc”) and perhaps by difficulty (“trivial”, “easy”, “medium”, “hard”). Then we can categorize the PRs manually, and assign weights to each of these types.

I get where this is coming from, but foresee a number of issues with relying on this too much to get to “pretty good” scores. One is the obvious politics / governance of having to make these judgement calls what is easy or hard. Launchpad for example has a “bug heat” score to compliment a traditional category assigned by someone with triage permissions to make sure not all of this falls to people with authority. Another is that it’s hard to let this be an exhaustive list and thus is not great for unexpected contributions. Finally it comes back down to having to judge the value of these things either ahead of time or around time of merging.

But I do like some aspects of this. A minimal set of “coloring” nodes so they can be weighted differently. I mentioned security before, but this is difficulty to quantify both with metrics and judgement calls as it requires expertise. A minimal coloring of a PR as security let’s you boost it if you like. For most contributions though I would rather look for a “reevaluating” metric instead of classifying them. Such as number of commits the code remains unchanged, or usage statistics of the features it touches on. This coloring doesn’t need to be just manually on github though. It could also mark portions of the code as being “core” and automatically color PRs and it’s interactions from that. For example to boost reviews for PRs against the core.

Bug heat

Launchpad helps you to appraise a bug by giving you a calculated measure — called bug heat — of its likely significance. You can see bug heat in bug listings, and also on individual bug pages, as a number next to a flame icon.

Here’s how Launchpad calculates the bug heat score:

Attribute	Calculation
Private	Adds 150 points
Security issue	Adds 250 points
Duplicates	Adds 6 points per duplicate bug
Affected users	Adds 4 points per affected user
Subscribers (incl. subscribers to duplicates)	Adds 2 points per subscriber