This post describes an incentive incompatibility in SourceCred today, along with a concrete plan to fix it.
Let’s suppose we’re interested in nodes that have two types of edges: AUTHORS edges and REFERENCES edges.
You’re the author of one of these nodes. Having written it, you know there are 9 relevant citations it could reference.
Let’s suppose that we’re in the current iteration of SourceCred, and AUTHORS and REFERENCES both have 1x
weight. Your options:
- Hide all the references, and only add the AUTHORS edge. You get 100% of the posts cred flows, but you might get called out for putting none of the references.
- Add all references. You’ve been very honest, and now you only get 10% of the cred.
- Add only the 3 best known references. You’ve “cheated” 6 references from their cred, but now you get 25% of the post’s cred.
Clearly, this is a bad incentive, since it encourages people to under-reference. In reverse, there’s a sort of Gresham’s Law dynamic at play, where one can “spam away” cred from certain edges by adding other low-quality edges.
I think we can solve this by adding a new top-level division of cred between edge types, which I’m tentatively calling “edge tranches”. The basic idea is: a node’s cred is split up in advance between the different edge types. And if there are no edges of a given type, all of that cred is recycled to the seed vector. Let’s suppose we decide to split 50% cred to authors and 50% cred to references. Then your options are:
- Hide all the references. You get 50% of the cred, 50% gets recycled to the seed vector.
- Add the most popular 3 references. You get 50% of the cred, 50% is split between those 3 references.
- Add all 9 references. You get 50% of the cred, 50% is split across all the references.
As you can see, there is no longer any incentive to under-report the references. Therefore, I think this will be a substantial improvement over the current system.
(Of course, there is an incentive to preferentially add references that will flow cred back to you, e.g. referencing works that you yourself authored. But that is a problem for another Discourse post. )
If folks agree with taking this approach, we should write an initiative to scope out this change.