Hey. So I ran SourceCred on some Decred repos (crypto project I contribute to). Thought I’d share my first impressions. Below are live instances (hosted on GitHub pages) for five Decred repos I have some experience with. I’ve also created a ‘decred meta’ instance containing all active repos (13 total). Impressions below repos.
(prototype)
https://s-ben.github.io/politeiagui-sourcecred-demo/site/prototype/decred/politeiagui/
(timeline)
https://s-ben.github.io/politeiagui-sourcecred-demo/site/timeline/decred/politeiagui/
(prototype)
https://s-ben.github.io/politeia-sourcecred-demo/site/prototype/decred/politeia/
politeia
(timeline),
https://s-ben.github.io/politeia-sourcecred-demo/site/timeline/decred/politeia/
dcrd:
(prototype)
https://s-ben.github.io/dcrd-sourcecred-demo/site/prototype/decred/dcrd/
(timeline)
not avail, too much history for graph to handle (have reported bug to sourcecred)
(prototype)
https://s-ben.github.io/dcrdocs-sourcecred-demo/site/prototype/decred/dcrdocs/
(timeline)
https://s-ben.github.io/dcrdocs-sourcecred-demo/site/timeline/decred/dcrdocs/
(prototype),
https://s-ben.github.io/dcrdata-sourcecred-demo/site/prototype/decred/dcrdata/
(timeline)
https://s-ben.github.io/dcrdata-sourcecred-demo/site/timeline/decred/dcrdata/
Decred-meta,
(prototype)
https://s-ben.github.io/decred-meta-sourcecred-demo/site/prototype/decred/decred-meta/
Impressions
- It works! For the repo I’m most familiar with, dcrdocs (I do mostly technical writing, transitioning to dev), SourceCred was pretty spot on; SC might work well with documentation as well as code. Timeline view right away had me telling interesting stories to myself about the data (there’s where I started contributing! That must be that person I heard about that left before I started…Yeah, I should have about that much total cred as that person, etc.).
- The meta decred repo doesn’t feel accurate… Perhaps it’s because I don’t have the domain and historical knowledge to rank contributors across the project that well, but SourceCred is clearly rating contributions from some repos higher than others, in a way that doesn’t seem fair. E.g. dcrdata seems overrepresented. Presumably because it’s a sprawling, mostly frontend project (a block explorer, stats charts, etc.), which generates more activity than, say, dcrd, the node daemon, which is mostly extremely complex consensus code that changes less frequently.
- Related to above, cred scores seem most interesting at the repo level. Each repo, even in the same org, tends to have scores that reflect its pace, contributor mix, type of content, process. On a personal level, repos I knew were engaging to interact with. Repos I didn’t weren’t that interesting. I’m wondering if the audience, for the main prototype view anyway, will be mostly of interest to contributors to that repo. More interesting to those outside the repo could be analysis that uses SourceCred tools to answer more specific questions (e.g. what contributors are most helpful to newcomers?). Or, (more interesting from a research perspective) asking questions that span the crypto space (e.g. which projects have the highest gini coefficient among devs (i.e. which dev teams are most centralized)?).
- For the repo I’m a main contributor to, dcrdocs (#2 and closing!), the rankings seemed accurate from my perspective. For the repo I have been a more minor contributor, politeia (Decred’s proposal system), my ranking feels off. It’s too high. I’m above at least two contributors I know have contributed a lot more. Perhaps it’s because I contributed a mix of documentation (inline documentation in source code files) and code. The documentation I created generated a decent number of PRs, with a lot of back and forth in the comments with the current maintainer (#2 in cred). I also went down a rabbit hole chasing a bug that was closed and never replicated–the Issue was closed, but not without 7 comments on it. My one merged code PR has comments from the founder (#1 in cred) questioning if it was needed (it was merged after defense from #2), and complaining that we had to add cludge (cludge cred).
- I presented some of my initial results in Decred’s slack to see if there was any interest in a broader collaboration (working with me to tweak a custom instance). Some found it interesting, but didn’t find any takers. My sense is that this is partly due to a strong aversion by some developers to ‘dev metrics’ in general, which some have had negative experiences with in the past. The perception being that it could be a mechanism to control them and erode their sovereignty in the system. To be fair, they’ve basically have a functioning DAO/co-op for 2-3 years now that pays people (including myself) to work on OSS, working on pretty much whatever they want-- a rarity I can understand people are protective of. I am reminded of a quote from @mzargham’s medium piece Network Formation Games, “…A second order challenge is that challenging the parameters of the Cred release policies too often or too much can undermine the faith the contributors have in the incentive system.”. Upon reflection, I can see how generally, if someone has invested a lot of time (years in some cases) into a system that’s working for them, they could see SourceCred as a threat to that. Perhaps SourceCred will be more welcomed at projects where hierarchies are not as established, and/or large sums of money are not yet at stake (~$250k/mo is spent by the Decred Treasury, mostly to contractors). Or perhaps in projects generating considerable wealth, but have contributors begging for rent money on Twitter and a community actively looking for a solution. SourceCred could also being deployed in “sandbox” situations that aren’t as threatening. E.g. use in pure research, or as a recruiting tool.