Thanks everyone for participating in this discussion. On a meta-level, I’m glad to see this conversation happening, uncomfortable parts and all. The difficulty around deciding what “fair” is for a system like this, and negotiating changes to it in a way that lets people still feel respected, enfranchised, and heard, is one of the biggest challenges for us (and for anyone who uses SourceCred). As @anon60584824 mentioned, the “human bits” of deploying a system like SourceCred are what makes it really challenging.
I really appreciate that in this thread, everyone has made a clear effort to respect and care for “the human bits” of other SourceCred contributors: working to understand others’ perspectives and intentions with good faith, hearing when feelings have been hurt and apologizing, and generally being supportive and honest with each other. Our ability to thrive is going to depend on our ability to support each other in these ways, just as much as it will depend on our ability to implement increasingly clever algorithms.
With that said: I think a lot of the challenges that have been surfaced in this thread reflect the immaturity of the SourceCred algorithm and parameter system. The promise of SourceCred’s graph-based approach is that we can mint cred at the level of things we really care about – goals like deployability (as supported by having a docker container and GitHub actions) or documentation (as supported by having a bevy of Getting Started guides on the Discourse). Once we have a supernode that represents the high level goal, we can tune the cred flow for the high level goal, and that will propagate out to everyone who worked on it.
However, we haven’t finished building the system that lets us do this. @Beanow is hard at work on the initiatives plugin, and afterwards we need to refactor the Graph to give us better APIs for controlling cred minting. It will probably be somewhere between “weeks” and “two months” before the more sophisticated, goal-based cred minting process is online.
While we’re waiting for that better system, we have an extremely crude alpha version, which mints cred solely on the amount and type of activity that SourceCred observes. As such, we mint cred for every Discourse post, for every GitHub pull request, and so forth. We can tune the cred minting (and thus the relative cred weights) by changing those overall weights.
This is a really crude system, kind of like hitting the cred distribution with a cudgel. Any weight we set for Discourse posts will either be too low for super in-depth guides and detailed discussion of the system, or too high for a placeholder post retroactively initializing an initiative. Any weight we set for GitHub pulls will either be too low for an involved refactor which increases Discourse performance, or it will be too high for a tiny PR to remove a deprecated file.
Ideally, we would already have next-gen system online, and I wouldn’t need to make crude and contentious changes to the weights. But, we’re not there yet, and I have an ongoing responsibility to tune the system to try to accomplish a few goals:
- Have the overall cred do a reasonable job of representing the value that all contributors have been providing
- Have the incremental cred create reasonable incentives for future work, so that as people are implicitly guided by the scores, they’re also contributing healthily to the community
When I first turned on the system, we had more than a year of GitHub contribution history, and very little activity on the Discourse. When I gave the system what I considered “reasonable weights”, it responded by giving almost all of the cred to myself and @wchargin, due to our very long history of code contributions. However, I felt it was important to cultivate the nascent Discourse community, and give meaningful cred to people who had started contributing in non-code ways.
So I deliberately chose weights that were unreasonable: a Discourse thread had 8x more weight than a GitHub pull request; a Discourse post had 32x more weight than a GitHub comment. Looked at directly, these weights didn’t really make sense. However, choosing these weights made for the cred distribution I felt was healthiest for the project at the time. I don’t regret setting the weights this way, but I should have anticipated and more clearly communicated that they would need to once Discourse activity levels approached our GitHub activity levels.
Now, several months later, we have a thriving Discourse community that is accomplishing a ton: exploring and explaining how SourceCred works, coming up with norms and patterns for our community, dog-fooding new features like Artifacts and Initiatives. This is great! But it also means that, because I chose unbalanced weights earlier, our overall cred is becoming very unbalanced. We’ve reached the point where the cred earned from a single week focused on Discourse can outpace the cred earned from several months of work on GitHub.
While we are still working with a set of crude levers to tune the cred, I still need to use those levers to do the best I can to keep the overall system in alignment. In this case, this means a change to make the Discourse and GitHub weights more consistent. If we had the tools to do so, I’d prefer to change the weights prospectively, not retroactively, and “stand by” our past weights (perhaps up to the most recent week or two). Unfortunately, we just don’t have that capability yet–and the result is a fairly disruptive change to cred.
The good news is, just as we can retro-actively update the cred now, we will retroactively update the cred with the new tools we develop in the coming months. Therefore, you can think of this current cred distribution as a temporary “working copy” that we’re going to keep improving. And the way Grain distribution works always tries to keep alignment between the most recent Cred and the lifetime Grain flows; therefore, people with too little cred right now will be “made whole” once we launch the improved system.
We’ll also launch a new UI which allows exploring cred at the level of initiatives, artifacts, and other supernodes. So rather than SourceCred explaining cred in terms of raw activity, like so:
It will be able to communicate my cred in terms of specific initiatives or artifacts I’m connected to (like setting up the core graph module and algorithm, or my leadership roles within the project as a whole). I think that will help these discussion a lot, by making it easy to see why other people have Cred, in terms of the specific initiatives and artifacts they supported.
This won’t be a perfect system either, and we’ll keep on having friction as we work on SourceCred. But I hope by improving the algorithm, we we’ll be able to make more intentional changes to the system that won’t be as frustrating or lead to so much churn in the cred.
In the mean time, we’re still very much in CredSperiment mode, so we need to accept there will be some turbulence in deploying our still-experimental and alpha quality system. 
Thanks for being along for this ride, y’all!