This quarter, I’d like to start dogfooding SourceCred on SourceCred itself. That is to say, I want to pay SC contributors based on their cred scores. This is the best way for us to get lived experience on how SourceCred really performs in practice and under pressure. We’re the best equipped to deal with the issues that come up, and we’ll learn so much from the experiment. So let’s do it!
Now that we have timeline cred, we have the technical pre-requisites on the SourceCred side to be able to do this. So there are two questions: funding and implementation. On funding, happily @Protocol Labs is willing to chip in, and I’m also hoping to find other supporters of the project who will help finance the experiment. On implementation, we need a system that:
- makes it easy for people to contribute financially
- makes it easy for people to get paid
- makes it easy for project maintainers to set a cred-based distribution
- makes it easy for project backers and contributors to have transparency and see that the funds were allocated appropriately
Initially, I was planning to use crypto (see discussion here). But the truth is that crypto would actually be a PITA in almost every dimension. (IMO, the breakthrough feature of crypto is the ability to create “incentive focal points” via token generation, using it for actual payments is still super frustrating.) I think we should go with OpenCollective instead; they are interested in working with us, already have the right feature set, are well-aligned with us, and as an added bonus lots of other OS projects are there; if this experiment works well, it will be easy for more people to adopt this approach.
So, as we decide how to run this experiment, we should both view it as a great dogfooding experience for SourceCred, and as a potential template for other collectives to start cred-based payments.
OpenCollective seems to operate on month-long reporting intervals, so I propose the following procedure for our experiment:
- Every month, the collective has revenues
R. The project chooses a payout ratiop, which determines what fraction of revenues will get paid via cred, and what fraction will accumulate in a treasury or is otherwise disbursed through non-cred mechanisms. For the SC experiment, setp=1.0. Then each month a total ofR*pwill be disbursed. - SourceCred calculates cred on a weekly basis, aligned on calendar weeks. At the end of the month, we calculate monthly cred for all contributors as the sum of their weekly score in each of the 4 weeks.
- The project maintainer(s) (in SourceCred’s case, me) review the cred distribution. They may change weights (e.g. increasing/decreasing weights on individual nodes) to subjectively improve the quality of the distribution; if so, they will publish the new weights. This produces the “canoncial cred distribution” for the month.
- If we haven’t yet added good in-band filtering to remove attackers/spammers, they may also exclude users from the cred distribution, by setting their cred to
0in the distribution. Maintainers shouldn’t make arbitrary out-of-band changes to the distribution; that should happen through the weights. Letting maintainers zero out some users’ scores is just a fallback to make sure that egregious attacks won’t be successful. - Then, supposing that
Dis the modified cred distribution, andcrepresents the identity of a particular contributor, we calculate the monthly payout for each contributor:D[c] / ∑D * R * pto each contributor. (I.e. we split the disbursement amount evenly based on cred) - Based on feedback from OpenCollective, we will define a minimum payout amount (e.g. $5) as sending smaller payments is likely to be uneconomical. As a long term solution, we should keep track of contributors unpaid payouts, so they can get paid once they are owned at least the minimum payout amount. For the purposes of the experiment, we may just filter out tiny payments and re-normalize the payments for the month.
- The maintainer (me) will then post a public thread, along with the payout totals, the weights used, and a link to a SC instance showing the cred for those weights. People in the community will have the chance to review the weights, discuss whether certain contributions were under/over-valued, and suggest changes.
- After a week (or when consensus is reached), the payment amounts will be finalized.
- We’ll then send those payments to the contributors via OpenCollective. (I’ll need to check with OpenCollective to find out what APIs are most applicable; I suspect every contributor who wants to get paid will need to make an account.)
The most controversial point here will be that the maintainer has a free hand in choosing the cred weights, and there’s a conflict of interest because they can choose weights so that they get paid more. A knee-jerk response would be to exclude the maintainer from payments, but that would be a mistake: in many cases, paying the maintainers is the most important part of making OS sustainable, so a payment mechanism that excludes maintainers is flawed from the start. Instead, we will need better governance and oversight mechanisms. In the long run, the maintainer shouldn’t be unilaterally changing the parameters; all stakeholders (contributors, funders, maintainers, community) should be involved in that process. But trusting the maintainers is a reasonable approach at the start.
I propose we scope this as a 3-month experiment, and communicate that to potential funders. At the end of the 3 months, we’ll evaluate how well this worked along many dimensions:
- What pain points did we encounter within SourceCred itself? Did we feel that the cred scores were doing a good job of reflecting value creation?
- How did the experiment change the overall level of engagement with the project?(Impossible to know for sure, but we can get a sense.)
- How did the experiment change the behavior of project participants? Which “cred gaming” behaviors were helpful / favorable? Which ones were bad?
- How did trusting the maintainer (i.e. me) work out? Do we have confidence in my decisions? Was there good transparency in those decisions, and would it have been easy to tell if I were choosing the weights in a self-serving way?
- How well did OpenCollective work as a platform for this experiment?
- Do we want to continue funding SC itself in this way?
- Would we recommend that other projects try the same approach?
As a bit of skin in the game, I’m planning to help personally fund this experiment (independent of contributions from PL or others). If people are concerned about my conflict of interest in getting paid from it, I can commit to providing enough funding that my net cash flow from the experiment is less than or equal to zero, although I think it actually makes a better, “realer” experiment if we don’t paper over all the conflict of interest issues. 
As for timeline, I’d love to have August be the first month of the experiment, which means we’d have our first payout at the beginning of September. On the SourceCred side, the main thing I’d want to have before we go live is the Discourse plugin; I want to make sure that community involvement is being rewarded alongside technical work from the start. I’m confident this is achievable before end of month. The other side would be working with OpenCollective to ensure that we can ship it from a technical perspective, and lining up at least some initial funding for the first month (e.g. from Protocol Labs).
I’d love any thoughts and feedback on this proposal!