Weekly Cred Analysis -- Week ending Jan 12

As a part of rolling out review culture, I’d like to start a regular practice of reviewing our weekly Cred scores. A key takeaway from our recent weight change discussion was that we should focus our attention not on the people earning cred, but on how we’re valuing contributions themselves.

To that end, I put together a cred analysis notebook which shows which lists every contribution that received at least 1 cred in the past week. I figure we can use this as a starting point to assess whether the Cred scores actually align with our values. You can open that notebook here.

I think the scores are pretty interesting. Some of it makes sense–for example the “Changing the CredSperiment Weights” thread shows up quite a bit, which makes sense, as it was an important discussion. Also, it identifies that this post was among the most valuable on the “GitHub PR labels” thread, which I agree with.

It was interesting that this old PR got the most cred. I can sort of see why – it was referenced by two recent PRs, #1534 and #1535. But those two pulls only have ~12 cred in total. So how did they flow over 23 cred to the old PR?

Personally, I think one of my most valuable contributions last week was kicking off the docs repo with the review culture guide but that only got about 6 cred–about a third of the cred of this github comment. I think this shows the need for boosting and/or the initiatives plugin. However, since the guide landed near the end of the week, most of the activity (reviews, comments, etc) didn’t come until after the week ended. So we’ll have to see how much cred it gets next week.

Finally, many important contributions (e.g. @LB booking the airbnbs for CredCon) simply don’t show up in the graph at all.

In future reviews, I’d like us to collaboratively improve the cred, e.g. by collectively adding initiatives and manually recorded contributions. We don’t quite have those tools ready yet. So for this week, please just share what your thoughts are, and what kinds of improvements you’d like to see us focus on.


Firstly I had a go at tweaking the notebook:

While this adds a grouped-by-type view, I think the mixed view is also really fascinating.

For example, the cred of individual posts and comments is sometimes surprisingly high. Like #1512(comment) and #1512(comment) with 16.3 cred each feels really high.

The comments were about fixing an issue that was actually fixed by:

Though I think, we may need some type of tree-view to make sense of this. Because those PRs don’t only get cred for the PR node, but also from reviews and comments.

When I add a filter to select this weeks’ nodes related to these 3 PRs a very different image appears:

Following the same thought, I tried looking into the review culture guide as well.

A whopping 60! nodes when removing the >1 cred threshold show up. Almost all of them 0 :open_mouth: (even when removing the rounding).

Could that be a bug?

Turns out! It has to do with the timestamp. This week of cred stops at Jan 12th (UTC). The only contributions in that time frame are the PR itself and the first comment.

The 58 other contributions are not included in this week :sweat_smile: So I’m expecting that will take a big leap next week.

1 Like

Nice notebook, @Beanow! Appreciate the added features.

I think a ‘tree view’ makes a lot of sense. We could have the following hierarchy:

  • Discourse Server
    • Discourse Topic
      • Discourse Posts
  • GitHub Repo
    • GitHub Issue
      • GitHub Issue Comment
    • GitHub Pull Request
      • GitHub Pull Request Comment
      • GitHub Review
        • GitHub Review Comment

I think exploring this tree view will be a lot more manageable, and will let us try to answer two questions separately: one, how much cred at a high level is being created around different clusters of activity, and secondly, how does that cred get distributed within the cluster.

Want to take a stab at it? :slight_smile:

Liking this idea as well. But for now Initiatives is stalled on my time so I will focus on that some more. This is a component I could also imagine in the explorer.

Suppose you’re on a profile page, and it lists a particular Topic. From there, being able to look at this topic in this tree view would be a natural ‘zooming in’.