Edge Tranches: Fixing an Incentive Misalignment

The CredSperiment Research
#1

This post describes an incentive incompatibility in SourceCred today, along with a concrete plan to fix it.

Let’s suppose we’re interested in nodes that have two types of edges: AUTHORS edges and REFERENCES edges.

You’re the author of one of these nodes. Having written it, you know there are 9 relevant citations it could reference.

Let’s suppose that we’re in the current iteration of SourceCred, and AUTHORS and REFERENCES both have 1x weight. Your options:

  1. Hide all the references, and only add the AUTHORS edge. You get 100% of the posts cred flows, but you might get called out for putting none of the references.
  2. Add all references. You’ve been very honest, and now you only get 10% of the cred.
  3. Add only the 3 best known references. You’ve “cheated” 6 references from their cred, but now you get 25% of the post’s cred.

Clearly, this is a bad incentive, since it encourages people to under-reference. In reverse, there’s a sort of Gresham’s Law dynamic at play, where one can “spam away” cred from certain edges by adding other low-quality edges.

I think we can solve this by adding a new top-level division of cred between edge types, which I’m tentatively calling “edge tranches”. The basic idea is: a node’s cred is split up in advance between the different edge types. And if there are no edges of a given type, all of that cred is recycled to the seed vector. Let’s suppose we decide to split 50% cred to authors and 50% cred to references. Then your options are:

  1. Hide all the references. You get 50% of the cred, 50% gets recycled to the seed vector.
  2. Add the most popular 3 references. You get 50% of the cred, 50% is split between those 3 references.
  3. Add all 9 references. You get 50% of the cred, 50% is split across all the references.

As you can see, there is no longer any incentive to under-report the references. Therefore, I think this will be a substantial improvement over the current system.

(Of course, there is an incentive to preferentially add references that will flow cred back to you, e.g. referencing works that you yourself authored. But that is a problem for another Discourse post. :wink:)

If folks agree with taking this approach, we should write an initiative to scope out this change.

cc @mzargham @wchargin

3 Likes
The Social Norms of Flowing Cred Questions (please answer!):sourcecred:
SourceCred Protocol
How Discourse Cred Flows
Likes: altruism => mini-boosting
Support Cred
#2

Seems like an improvement from our current situation. I’m into it. Thanks for putting this together :slight_smile:

#3

I know I am late to the party in reviewing this but I am working through my SourceCred backlog as my contribution to CredCon today.

Concur strongly with option 3. Sub-partitioning cred allocations by types preserves all necessary properties of the core algorithm.

There remains higher order gaming in the sense that you might self-cite to draw some of the citation allocation back to yourself. However, i think for the mean time this becomes a normative checkpoint. If you are genuinely building on your own past work, this should be fine, but to cite only your own work to capture the citation cred would generally be construed as an attack.

Also, here is an old “note” I wrote with David Sisson a round a year ago, before discourse was in use. It pertained to the MultiClass variation of pagerank and how it might be considered in sourcecred.

Happy CredCon! -Z