Cred represents contributions to a project. Projects are constantly evolving, and as they do so does Cred.
Source: the source code (data) of a project that is represented as a graph.
Cred: scores that nodes on the graph receive based on contributions and engagement.
SourceCred creates a graph over every project with nodes represent actors and items. SourceCred then gives those nodes scores according to a PageRank style algorithm. As people contribute to a project the graph will change and Cred scores will be updated accordingly. Cred is a living representation of contributions to a project.
We want open-source contributors to earn a living, and a share in the value created on top of their contributions. Tracking contributions in a living graph that is updated over time allows people to be rewarded for the value their work provides, today and into the future.
Weâre designing the SourceCred protocol around the following principles:
Transparency
It should be easy to see why cred is attributed as it is, and link a personâs cred directly to contributions theyâve made.
Extensibility
SourceCred is designed around a plugin architecture, so you can add support for new data sources, new algorithms, or even entirely new kinds of work.
Decentralization
Each community has the final say on that communityâs cred. When the algorithm and the community disagree, the community wins. Projects own their own data, and control their own cred. The SourceCred project provides tools, but has no control.
So many⌠Iâm honestly not even sure where to start. Would the entire SourceCred repo qualify here? Or is that a dependency?
The SourceCred prototype allows you to check your Cred scores.
opened 12:58AM - 05 Dec 18 UTC
closed 05:33AM - 06 Aug 20 UTC
We should have mission plan for SourceCred.
I've put a preliminary outline b⌠elow. I'll try to convert this into a longform document (likely checked into Git in this repository) soon.
Thanks to @meiqimichelle for prompting this.
cc/ @flyingzumwalt @sternhenri @wchargin @momack2
---
# 1. Vision/Setup
- Credit Attribution: The problem SourceCred is trying to solve
- Why this matters / What it enables (paying open source, better-resourced communities)
- Vision for how SourceCred will look + operate in the long run
# 2. How SourceCred Works
- PageRank algorithm and the graph based approach
- Customizability + flexibility through...
- Plugins which add new nodes (w/ example)
- Plugins which add new kinds of edges (w/ example)
- Heuristics on the values of nodes and edge (w/ example)
# 3. Current State of the project
- "Late Alpha" type software that proves the general approach (link to prototype)
- Strong technical foundation (thoughtfully architected Graph class, great test coverage, static typing)
- Uses data mostly from GitHub (issues, pulls, comments, etc), few plugins
- Decent job with identifiying top contributors, but struggles to accurately assess value of specific contributions
- e.g. it can't tell which commits are important
# 4. Key Risks / Issues
- SourceCred might be easy to game
- Discussion of how social norms constrain gameability, hence focus on transparency, but it needs to be proven out
- SourceCred might fail to appreciate non-technical contributors
- SourceCred might produce weird social norms / disrupt the social ecology of open-source projects
- The PageRank/Cred Graph approach might not be flexible enough to accurately value contributions
- GitHub could restrict access to their APIs
# 5. Key Initiatives
- Grow the project
- Increase # of people working on / contributing to SourceCred
- Increase # of people using SourceCred, i.e. increase adoption
- Improve credit attribution on open-source projects
- Add features like time-based cred and code analysis so that SourceCred is dramatically better at credit assignment on a median open-source project
- Improve credit attribution on SourceCred itself
- Use SourceCred as a testing ground for new techniques that require engagement from the community but could be a lot better overall.
- Try to prove that SourceCred can reward all types of work, not just coding, by proving it on SourceCred itself
https://github.com/sourcecred/pm/blob/master/overview.md
I feel like almost everything related to SourceCred and the CredSperiment could be in this section lol
https://sourcecred.io/cred/
I want to really understand how SourceCred works. While thereâs a lot of high level overviews in various places, there isnât a protocol specification or whitepaper. Starting this thread so that we can aggregate reference material useful to help people understand SourceCred and also draft a protocol specification that explains how SourceCred actually works
Tonight, at a dinner that @jbenet organized, he pointed out the importance of using SourceCred to showcase best practices around fairly and freely sharing cred, and suggested that we should start developing this practice soon, while cred is still young. In the ensuing conversation, I realized we already have all the tools we need to start documenting it, and created the Spotlights category on this forum. Thanks Juan!
This thread is intended for defining Reference Cases for Credit Attribution.
The Original reference cases discussion was for a line graph or chain of dependencies from an early discussion with @decentralion and @miyazono from @Protocol Labs. This reference case and others are briefly explored here using some preliminary research infrastucture. See GitHub issues in SourceCred Research for how we are testing and expanding our infrastructure.
There is not necessarily one right answer for how creâŚ
At dinner with @3blue1brown tonight, I described the weight composition problem to him.
He suggested viewing the weights / heuristics as unit vectors (in node-space), and viewing adding heuristics as weighted interpolation between the current vector, and the vector defined by the new heuristic. If the new heuristic isnât defined for every node, then you just interpolate on the dimensions the new vector is defined for, while staying on the unit simplex in the whole space.
I havenât fully grokkeâŚ
Placeholder text that is at least 20 characters.
SourceCred contributor payouts are calculated via Cred scores.
As per this thread, creating a thread to highlight properties of Cred and Grain. The idea is to aggregate conversations and data from this forum into threads that explore a theme.
If you want to add any relevant data points and/or link to relevant content from the forum, please do! Every post is a wiki so you can edit freely.
This is an experiment, so set your expectations accordingly.
This is a really interesting thought experiment!
Iâve spent a lot of time thinking about how to do token distributions based on cred. The systems Iâve had in mind have involved creating new project-specific cryptoassets called Grain and doing distributions at the project level, with large rewards, large upfront commitments, and requiring solid buy in from the project maintainers. One thing I really like about your approach is itâs something we could launch soon (e.g. this year), with very smallâŚ
Disclaimer: This content is outdated.
Please see SourceCred in 5 minutes for the latest overview of the CredSperiment.
This post describes the initial design of the CredSperiment, as envisioned in August 2019. Since then, the system has changed materially. Please view this document as a piece of history in SourceCredâs evolution, but not an up-to-date view of how SourceCred operates.
The CredSperiment creates a game in which people are rewarded for contributing to SourceCred, based on SourceâŚ
Is SourceCred looking to measure all-time cred (i.e. an objective measurement of all contributors/contributions), or the current credibility of active members? It seems that implementing time-based cred (the most commonly asked for feature), would be more measuring the latter.
In the coming weeks, Iâm planning to write a roadmap for SourceCred, which basically documents all of the initiatives currently in flight, and lays out the different priorities we can focus on. For example, here are some of the priorities we could focus on:
Documenting SourceCred
Improving SourceCred instance management (make it easy for anyone to run SourceCred)
Improving cred transparency and UI (make it easier to tell why someone has cred, or see cred for a specific week, etc)
Build the nexâŚ
Implement Cred Weighted Prioritization
Status: proposal
Champion:
Initiative Description:
As discussed here , weâd like to use cred-weighted voting to prioritize SourceCred initiatives. We can then give the prioritized initiatives a nice cred bounty .
As a practical matter, I propose that we use cred-weighted, dependency-aware conviction voting.
Cred-Weighted
Voting power will be based not on number of votes, but based on the amount of cred behind each vote. We can experiment with different flaâŚ
So, based on a couple of recent discussions in yesterdaysâ office hours, and forum posts on:
What is the absolute ($) value of cred?
Are our current weights right ?
Diminishing returns curves for certain contributions .
Creating new cred for specific project goals / areas of improvement.
Our cred âmintingâ (node weight == new cred) system being pretty simplistic compared to the sophisticated cred âflowingâ (pagerank).
Putting all those thoughts together, I came up with the idea that, maybe ouâŚ
Greetings, credizens! I am very pleased to share a rough draft of SourceCredâs own cred, for the CredSperiment.
In contrast to the instances I shared in the CredSperiment Progress Report , we now have a combined instance which properly resolves identities across GitHub and Discourse.
You can check out the instance here . For posterity: If the url is down, you can also start any http server in the docs subdirectory of this commit .
My analysis of the scores
Here are the scores from the new instanâŚ
In the Preliminary CredSperiment Cred mega topic, we had some interesting discussion around valuing code reviews.
@Beanow makes a really good point here. SourceCred should actively support and encourage the collaborative norms of open-source. However, the approach I described originally could have the opposite effect. If each PR is only âworthâ a certain amount of cred, and it gets split between the author and reviewers, then authors may resent reviewers for âtakingâ some of âtheirâ cred. ThiâŚ
Maybe itâs just me, but it seems like there arenât as many likes flowing around as before. If this is true, then this is a problem. Hoarding Cred will lead to low value Cred for all parties involved! The more we give Cred, the more people are likely to engage in order to earn Cred. The more people engage with the community and earn Cred, the more Cred they have to give away. This creates a virtuous feedback cycle.
Currently, as far as I know, nodes that donât flow Cred out kind of just accumulaâŚ
Moving from a discord chat .
Currently it looks like you can farm cred by spamming.
I tried reading up on the timeline cred algorithm and from what I understand.
Nodes like issues, comments, PRs and PR reviews have a base value equal to the weight.
That value then flows to other nodes it has a relationship to, such as the repository and the comment author.
In small volumes of spam, this would mean so long as you donât âfeed the trollsâ by avoiding interacting with these threads the cred gainedâŚ
The key challenge for SourceCred is keeping the cred graph accurate and up-to-date. So far, weâve depended on a mixture of automatic processes (edges between posts and replies) and well-intentioned user behavior (adding references / citations to relevant work). These approaches have gotten us started, but they wonât scale.
For example, consider the SourceCred poster art . Since the poster uses the logo design, it should have an edge to the logo explorations . It doesnât, because @LB didnât do theâŚ
Raw PageRank outputs a probability distribution. When used as scores for human display, they wind up being unintelligible, because a characteristic score would be 0.000007, which is hard to read and discouraging to the user.
To fix this, we started normalizing scores so that usersâ cred sums to 1000. This has the advantage that usersâ scores are usually readable numbers between 1 and 1000, and that scores are comparable across contexts. However, it also has some drawbacks. In very large repos (âŚ
https://discourse.sourcecred.io/t/about-the-cred-feedback-category/16
https://discourse.sourcecred.io/t/flow-cred-feedback/44
Cred Analysis Notebooks
Status: help wanted
Occasionally new types of Notebooks are created, but thereâs room for a lot more.
Additionally itâs a great starting point for new contributors.
Please give a shoutout if youâd like to help.
Champion:
None atm.
It would be very helpful if a Champion could oversee the quality, organization and prioritized wishlist
of Notebooks.
Initiative Description:
Observable Notebooks have proven extremely useful tools for building new interfaces, experiences, andâŚ
https://discourse.sourcecred.io/t/ropensci-cred-feedback/24
https://discourse.sourcecred.io/t/libp2p-cred-feedback/21
A thing that came to mind talking about the dynamic between cred and the second quantity.
Cred flow has had this situation where being someone with higher cred can create circular reasoning. This issue is worth more cred, because a high cred person engaged with it, earning the high cred person more cred. Iâm sure this has been discussed a lot before.
Adding on top of this a second quantity, I wonder if this will amplify or dilute this effect. Would it sidestep the clout of high cred people becâŚ
Suppose that the SourceCred team goes to a hackathon, and you help me contribute there.
Suppose that you arrange my flights, so that I can actually make it to the hackathon.
Clearly, you deserve some cred.
How can we actually represent that within the SourceCred graph?
To start, we should make nodes in the graph that represent your contributions:
âarranging Dandelionâs hackathon logisticsâ. But how do we connect those nodes,
so that you receive an appropriate amount of cred?
I imagine that thâŚ
Weâve had a few chats here about whether it makes sense to think of cred as a âcurrencyâ. Iâve generally said that cred isnât a currency, because currencies are fungible and exchangeable, and cred is not, because cred is intrinsically tied to a specific identity and specific contributions. Thatâs why Iâve thought we should eventually create Grain as the currency that is derived from cred.
However, I just found an interesting post (Avoid Blunders in Designing Reputation Systems ) which distinguisâŚ
Currently, we have a problem in SourceCred: we measure the value of activity,
but not the associated cost.
This means that SourceCred will always reward doing moreâmore pull requests,
more comments, more notifications, more activity, even if the cost of that
activity exceeds its value.
Hereâs an example that @wchargin and I have discussed. Consider two pull
requests suggesting equally valuable changes to the codebase. One pull request
has been carefully prepared, and merges smoothly, with a siâŚ
Been doing lots of research on DAOs over the years and decided to put it all in one place (a Discourse forum) so that people could engage with the material.
The way the forum is setup each DAO and/or platform has itâs own category. The pinned thread in each category is for the main DAO or platform, and then threads in that category are for forks and/or stuff built on that platform. In every thread the top level comment is a wiki so that the community can update it with relevant info, and thenâŚ
After much late night hacking⌠I now have a live timeline cred prototype to share!
Hereâs what the UI looks like:
[image]
The basic approach is that we slice the history of the project by week. For each week, we run PageRank and assign scores, with some time-based twists:
we set the edge weights based on time: edges that donât yet exist have 0 weight, edges that were just created have full weight, and older edges have their weight decayed exponentially
we run PageRank with a seed vector poiâŚ
What is SourceCred?
SourceCred is a reputation protocol for open collaboration. At a basic level, SourceCred tracks the contributions that are made to a community project. It does this by assigning each contribution and contributor a score, called âCredâ, based on the contributionsâ value to the project.
You can think of SourceCred as turning community collaboration into a game. To earn high Cred scores you need to make contributions that the community values. SourceCred itself is dogfooding SâŚ
? :
initiative system . The basic idea is that the initiative will have a bounty amount, and a defined âsuccess conditionâ. Once the success condition is realized, we mint new cred (equal to the bounty amount), and flow it to the initiative node. From the initiative node, it will flow to the contributions, to the initiative proposer, and to the references.
The bounty getting paid out doesnât necesâŚ
To Do
Figure out what qualifies as a reference and what actually contributed to the design and development of âCred.â
Update the description.
Create a Cred overview post.
Thinking about this more, and âCredâ is really hard to separate from âSourceCred.â This is because Cred is a force of the system, so really itâs a part of the system. Cred canât exist on itâs own without the SourceCred protocol, and the SourceCred protocol couldnât exist if it didnât measure contributions via Cred.
I would say in my mind the current distinction is that âCredâ is really a shorthand for the âSourceCred protocol,â whereas âSourceCredâ refers to the entire SourceCred project?
https://sourcecred.io/cred/
SourceCred, as the name might suggest, is all about attributing cred.
Cred is a metric that describes every contribution and contributor in
a project, giving a sense of how important they were.
For example, forum posts (like this one) can earn cred. If this post earns 5
cred, and another post earns 10 cred, then that other post is considered
twice as important as this one. (Note: SourceCred doesnât load
posts from this forum yet.)
The contributions are arranged in a graph , where contributions âŚ
An Overview of Grain : an initial overview of Grain. While this Grain design is an open design space, this sets the foundation for many of the resulting conversations around Cred/Grain dynamics
Disclaimer: This content is outdated.
Please see SourceCred in 5 minutes for the latest overview of the CredSperiment.
This post describes the initial design of the CredSperiment, as envisioned in August 2019. Since then, the system has changed materially. Please view this document as a piece of history in SourceCredâs evolution, but not an up-to-date view of how SourceCred operates.
The CredSperiment creates a game in which people are rewarded for contributing to SourceCred, based on SourceâŚ
What is SourceCred?
SourceCred is a reputation protocol for open collaboration. At a basic level, SourceCred tracks the contributions that are made to a community project. It does this by assigning each contribution and contributor a score, called âCredâ, based on the contributionsâ value to the project.
You can think of SourceCred as turning community collaboration into a game. To earn high Cred scores you need to make contributions that the community values. SourceCred itself is dogfooding SâŚ
https://github.com/sourcecred/pm/blob/master/overview.md
burrrata:
Thinking about this more, and âCredâ is really hard to separate from âSourceCred.â This is because Cred is a force of the system, so really itâs a part of the system. Cred canât exist on itâs own without the SourceCred protocol, and the SourceCred protocol couldnât exist if it didnât measure contributions via Cred.
I agree, I think that âCredâ is a somewhat awkward artifact because itâs so integrally a part of SourceCred.
Maybe a clearer artifact would be something like âCred algorithmâ or âSourceCred system designâ. (Or maybe âcred algorithmâ is a sub-artifact within âSourceCred system designâ.) Then it would be a bit clearer that weâd be pointing to dependencies like the PageRank paper, the initiative for prototyping credv1, the initiative for adding timeline cred, and so forth.
What if we thought of the âSourceCred protocolâ as the graph and the way to attribute Cred scores to nodes on that graph?
Then everything else thatâs a derivative off of that graph and itâs scores (like Grain) could be part of the âSourceCred gameâ (or âsystemâ if the word âgameâ is unpopular lol).
This would allow the system to change as we design new features and plugins, but the heart of SourceCred would always be the core protocol that creates a graph and assigns Cred scores based on nodes on that graph.
Related tread that explains more how Cred will flow through the SourceCred protocol
This post describes an incentive incompatibility in SourceCred today, along with a concrete plan to fix it.
Letâs suppose weâre interested in nodes that have two types of edges: AUTHORS edges and REFERENCES edges.
Youâre the author of one of these nodes. Having written it, you know there are 9 relevant citations it could reference.
Letâs suppose that weâre in the current iteration of SourceCred, and AUTHORS and REFERENCES both have 1x weight. Your options:
Hide all the references, and only âŚ