This is an interesting perspective; thanks for sharing it. I think the conceptual framework of looking at contributions in terms of “supply” and “demand” is really promising. However, I think minting cred based on fixed types, even with a supply/demand model, still has the same structural weaknesses of the current model.
Minting cred based on node type is the real flaw
The way I chose the weights was aiming towards getting an overall distribution that matched my intuitions. In particular, I wanted to give meaningful cred to the participants on this Discourse; to do so, I set very high weights on Discourse activity. However, as you point out very clearly, these weights don’t make economic sense. If we keep these weights, in the medium term the community will likely become unbalanced as you describe.
However, the issue with weights goes deeper than their current paramter choices being wrong; minting a fixed amount of cred per unit of activity is just a bad model. This is because just knowing that something is a PR or is a post tells you approximately nothing about its actual value. Fortunately SourceCred isn’t all this dumb, the cred itself gets awarded based on the graph structure, but minting the cred based on the node type is really naive.
Supply/Demand by node type has the same issues
Overall, I think that thinking about supply and demand at the level of node types suffers some of the same issues as setting fixed weights based on node type. If we have a “demand curve” for pull requests, won’t that incentivize a spate of low-effort minor PRs every week? (Possibly with people “saving up” small PRs to send in on Monday morning.)
Also, since PRs are not interchangeable, a static supply/demand system will always grossly misprice some PRs. E.g. I get 4 PRs fixing minor UI issues, so my “demand curve” says I’m saturated, but then I get a 5th PR adding an important new feature. I’d be a lot more excited about that 5th one than about the 4 earlier ones.
I think there are a few cases where the “demand curve” model may make sense, like code reviews. Maybe I want every PR to get at least code reviews, and so we’re willing to incentivize more to get the first few reviews.
Alternative: Mint cred based on human review
Going back to @kronosapiens’s thoughts on PvE vs PvP dynamics. I think your proposal is basically a more sophisticated “PvE” system, but what if we just switch cred minting to “PvP”? Here’s how it could work:
Weekly Updates
For every week, we write a “Weekly Update” that summarizes what happened that week (interesting forum posts, links to pull requests that merged, new features added, etc). Through some offline process, we choose an amount of cred to mint, and flow it through the weekly update to the contributions mentioned therein. We can use the existing heuristics (and any other heuristics we come up with) as an input to our deliberation about how much cred to issue.
Obviously, this process requires a lot of trust, involves conflicts of interest, and won’t scale; over time we will find better ways to systematize it. (E.g. moving to a prediction-market based curation mechanism.) But in the spirit of Do things that don’t scale: this would give us much better short-term results than any system of heuristics based on node types.
Roadmap bounties
As discussed during office hours: once we do our roadmap prioritization we can set large cred bounties on accomplishing different goals for SourceCred. Once the goal is accomplished, we will “unlock” a large amount of cred that then flows to the nodes relevant to accomplishing that goal.
These approaches will implicitly involve supply and demand, since we (when setting the weekly weight) will have an implicit “demand” function that underlies our judgements about how to score each week.