Cred represents contributions to a project. Projects are constantly evolving, and as they do so does Cred.
Source: the source code (data) of a project that is represented as a graph.
Cred: scores that nodes on the graph receive based on contributions and engagement.
SourceCred creates a graph over every project with nodes represent actors and items. SourceCred then gives those nodes scores according to a PageRank style algorithm. As people contribute to a project the graph will change and Cred scores will be updated accordingly. Cred is a living representation of contributions to a project.
We want open-source contributors to earn a living, and a share in the value created on top of their contributions. Tracking contributions in a living graph that is updated over time allows people to be rewarded for the value their work provides, today and into the future.
Weāre designing the SourceCred protocol around the following principles:
Transparency
It should be easy to see why cred is attributed as it is, and link a personās cred directly to contributions theyāve made.
Extensibility
SourceCred is designed around a plugin architecture, so you can add support for new data sources, new algorithms, or even entirely new kinds of work.
Decentralization
Each community has the final say on that communityās cred. When the algorithm and the community disagree, the community wins. Projects own their own data, and control their own cred. The SourceCred project provides tools, but has no control.
So manyā¦ Iām honestly not even sure where to start. Would the entire SourceCred repo qualify here? Or is that a dependency?
The SourceCred prototype allows you to check your Cred scores.
opened 12:58AM - 05 Dec 18 UTC
closed 05:33AM - 06 Aug 20 UTC
We should have mission plan for SourceCred.
I've put a preliminary outline bā¦ elow. I'll try to convert this into a longform document (likely checked into Git in this repository) soon.
Thanks to @meiqimichelle for prompting this.
cc/ @flyingzumwalt @sternhenri @wchargin @momack2
---
# 1. Vision/Setup
- Credit Attribution: The problem SourceCred is trying to solve
- Why this matters / What it enables (paying open source, better-resourced communities)
- Vision for how SourceCred will look + operate in the long run
# 2. How SourceCred Works
- PageRank algorithm and the graph based approach
- Customizability + flexibility through...
- Plugins which add new nodes (w/ example)
- Plugins which add new kinds of edges (w/ example)
- Heuristics on the values of nodes and edge (w/ example)
# 3. Current State of the project
- "Late Alpha" type software that proves the general approach (link to prototype)
- Strong technical foundation (thoughtfully architected Graph class, great test coverage, static typing)
- Uses data mostly from GitHub (issues, pulls, comments, etc), few plugins
- Decent job with identifiying top contributors, but struggles to accurately assess value of specific contributions
- e.g. it can't tell which commits are important
# 4. Key Risks / Issues
- SourceCred might be easy to game
- Discussion of how social norms constrain gameability, hence focus on transparency, but it needs to be proven out
- SourceCred might fail to appreciate non-technical contributors
- SourceCred might produce weird social norms / disrupt the social ecology of open-source projects
- The PageRank/Cred Graph approach might not be flexible enough to accurately value contributions
- GitHub could restrict access to their APIs
# 5. Key Initiatives
- Grow the project
- Increase # of people working on / contributing to SourceCred
- Increase # of people using SourceCred, i.e. increase adoption
- Improve credit attribution on open-source projects
- Add features like time-based cred and code analysis so that SourceCred is dramatically better at credit assignment on a median open-source project
- Improve credit attribution on SourceCred itself
- Use SourceCred as a testing ground for new techniques that require engagement from the community but could be a lot better overall.
- Try to prove that SourceCred can reward all types of work, not just coding, by proving it on SourceCred itself
https://github.com/sourcecred/pm/blob/master/overview.md
I feel like almost everything related to SourceCred and the CredSperiment could be in this section lol
https://sourcecred.io/cred/
I want to really understand how SourceCred works. While thereās a lot of high level overviews in various places, there isnāt a protocol specification or whitepaper. Starting this thread so that we can aggregate reference material useful to help people understand SourceCred and also draft a protocol specification that explains how SourceCred actually works
Tonight, at a dinner that @jbenet organized, he pointed out the importance of using SourceCred to showcase best practices around fairly and freely sharing cred, and suggested that we should start developing this practice soon, while cred is still young. In the ensuing conversation, I realized we already have all the tools we need to start documenting it, and created the Spotlights category on this forum. Thanks Juan!
This thread is intended for defining Reference Cases for Credit Attribution.
The Original reference cases discussion was for a line graph or chain of dependencies from an early discussion with @decentralion and @miyazono from @Protocol Labs. This reference case and others are briefly explored here using some preliminary research infrastucture. See GitHub issues in SourceCred Research for how we are testing and expanding our infrastructure.
There is not necessarily one right answer for how creā¦
At dinner with @3blue1brown tonight, I described the weight composition problem to him.
He suggested viewing the weights / heuristics as unit vectors (in node-space), and viewing adding heuristics as weighted interpolation between the current vector, and the vector defined by the new heuristic. If the new heuristic isnāt defined for every node, then you just interpolate on the dimensions the new vector is defined for, while staying on the unit simplex in the whole space.
I havenāt fully grokkeā¦
Placeholder text that is at least 20 characters.
SourceCred contributor payouts are calculated via Cred scores.
As per this thread, creating a thread to highlight properties of Cred and Grain. The idea is to aggregate conversations and data from this forum into threads that explore a theme.
If you want to add any relevant data points and/or link to relevant content from the forum, please do! Every post is a wiki so you can edit freely.
This is an experiment, so set your expectations accordingly.
This is a really interesting thought experiment!
Iāve spent a lot of time thinking about how to do token distributions based on cred. The systems Iāve had in mind have involved creating new project-specific cryptoassets called Grain and doing distributions at the project level, with large rewards, large upfront commitments, and requiring solid buy in from the project maintainers. One thing I really like about your approach is itās something we could launch soon (e.g. this year), with very smallā¦
Disclaimer: This content is outdated.
Please see SourceCred in 5 minutes for the latest overview of the CredSperiment.
This post describes the initial design of the CredSperiment, as envisioned in August 2019. Since then, the system has changed materially. Please view this document as a piece of history in SourceCredās evolution, but not an up-to-date view of how SourceCred operates.
The CredSperiment creates a game in which people are rewarded for contributing to SourceCred, based on Sourceā¦
Is SourceCred looking to measure all-time cred (i.e. an objective measurement of all contributors/contributions), or the current credibility of active members? It seems that implementing time-based cred (the most commonly asked for feature), would be more measuring the latter.
In the coming weeks, Iām planning to write a roadmap for SourceCred, which basically documents all of the initiatives currently in flight, and lays out the different priorities we can focus on. For example, here are some of the priorities we could focus on:
Documenting SourceCred
Improving SourceCred instance management (make it easy for anyone to run SourceCred)
Improving cred transparency and UI (make it easier to tell why someone has cred, or see cred for a specific week, etc)
Build the nexā¦
Implement Cred Weighted Prioritization
Status: proposal
Champion:
Initiative Description:
As discussed here , weād like to use cred-weighted voting to prioritize SourceCred initiatives. We can then give the prioritized initiatives a nice cred bounty .
As a practical matter, I propose that we use cred-weighted, dependency-aware conviction voting.
Cred-Weighted
Voting power will be based not on number of votes, but based on the amount of cred behind each vote. We can experiment with different flaā¦
So, based on a couple of recent discussions in yesterdaysā office hours, and forum posts on:
What is the absolute ($) value of cred?
Are our current weights right ?
Diminishing returns curves for certain contributions .
Creating new cred for specific project goals / areas of improvement.
Our cred āmintingā (node weight == new cred) system being pretty simplistic compared to the sophisticated cred āflowingā (pagerank).
Putting all those thoughts together, I came up with the idea that, maybe ouā¦
Greetings, credizens! I am very pleased to share a rough draft of SourceCredās own cred, for the CredSperiment.
In contrast to the instances I shared in the CredSperiment Progress Report , we now have a combined instance which properly resolves identities across GitHub and Discourse.
You can check out the instance here . For posterity: If the url is down, you can also start any http server in the docs subdirectory of this commit .
My analysis of the scores
Here are the scores from the new instanā¦
In the Preliminary CredSperiment Cred mega topic, we had some interesting discussion around valuing code reviews.
@Beanow makes a really good point here. SourceCred should actively support and encourage the collaborative norms of open-source. However, the approach I described originally could have the opposite effect. If each PR is only āworthā a certain amount of cred, and it gets split between the author and reviewers, then authors may resent reviewers for ātakingā some of ātheirā cred. Thiā¦
Maybe itās just me, but it seems like there arenāt as many likes flowing around as before. If this is true, then this is a problem. Hoarding Cred will lead to low value Cred for all parties involved! The more we give Cred, the more people are likely to engage in order to earn Cred. The more people engage with the community and earn Cred, the more Cred they have to give away. This creates a virtuous feedback cycle.
Currently, as far as I know, nodes that donāt flow Cred out kind of just accumulaā¦
Moving from a discord chat .
Currently it looks like you can farm cred by spamming.
I tried reading up on the timeline cred algorithm and from what I understand.
Nodes like issues, comments, PRs and PR reviews have a base value equal to the weight.
That value then flows to other nodes it has a relationship to, such as the repository and the comment author.
In small volumes of spam, this would mean so long as you donāt āfeed the trollsā by avoiding interacting with these threads the cred gainedā¦
The key challenge for SourceCred is keeping the cred graph accurate and up-to-date. So far, weāve depended on a mixture of automatic processes (edges between posts and replies) and well-intentioned user behavior (adding references / citations to relevant work). These approaches have gotten us started, but they wonāt scale.
For example, consider the SourceCred poster art . Since the poster uses the logo design, it should have an edge to the logo explorations . It doesnāt, because @LB didnāt do theā¦
Raw PageRank outputs a probability distribution. When used as scores for human display, they wind up being unintelligible, because a characteristic score would be 0.000007, which is hard to read and discouraging to the user.
To fix this, we started normalizing scores so that usersā cred sums to 1000. This has the advantage that usersā scores are usually readable numbers between 1 and 1000, and that scores are comparable across contexts. However, it also has some drawbacks. In very large repos (ā¦
https://discourse.sourcecred.io/t/about-the-cred-feedback-category/16
https://discourse.sourcecred.io/t/flow-cred-feedback/44
Cred Analysis Notebooks
Status: help wanted
Occasionally new types of Notebooks are created, but thereās room for a lot more.
Additionally itās a great starting point for new contributors.
Please give a shoutout if youād like to help.
Champion:
None atm.
It would be very helpful if a Champion could oversee the quality, organization and prioritized wishlist
of Notebooks.
Initiative Description:
Observable Notebooks have proven extremely useful tools for building new interfaces, experiences, andā¦
https://discourse.sourcecred.io/t/ropensci-cred-feedback/24
https://discourse.sourcecred.io/t/libp2p-cred-feedback/21
A thing that came to mind talking about the dynamic between cred and the second quantity.
Cred flow has had this situation where being someone with higher cred can create circular reasoning. This issue is worth more cred, because a high cred person engaged with it, earning the high cred person more cred. Iām sure this has been discussed a lot before.
Adding on top of this a second quantity, I wonder if this will amplify or dilute this effect. Would it sidestep the clout of high cred people becā¦
Suppose that the SourceCred team goes to a hackathon, and you help me contribute there.
Suppose that you arrange my flights, so that I can actually make it to the hackathon.
Clearly, you deserve some cred.
How can we actually represent that within the SourceCred graph?
To start, we should make nodes in the graph that represent your contributions:
āarranging Dandelionās hackathon logisticsā. But how do we connect those nodes,
so that you receive an appropriate amount of cred?
I imagine that thā¦
Weāve had a few chats here about whether it makes sense to think of cred as a ācurrencyā. Iāve generally said that cred isnāt a currency, because currencies are fungible and exchangeable, and cred is not, because cred is intrinsically tied to a specific identity and specific contributions. Thatās why Iāve thought we should eventually create Grain as the currency that is derived from cred.
However, I just found an interesting post (Avoid Blunders in Designing Reputation Systems ) which distinguisā¦
Currently, we have a problem in SourceCred: we measure the value of activity,
but not the associated cost.
This means that SourceCred will always reward doing moreāmore pull requests,
more comments, more notifications, more activity, even if the cost of that
activity exceeds its value.
Hereās an example that @wchargin and I have discussed. Consider two pull
requests suggesting equally valuable changes to the codebase. One pull request
has been carefully prepared, and merges smoothly, with a siā¦
Been doing lots of research on DAOs over the years and decided to put it all in one place (a Discourse forum) so that people could engage with the material.
The way the forum is setup each DAO and/or platform has itās own category. The pinned thread in each category is for the main DAO or platform, and then threads in that category are for forks and/or stuff built on that platform. In every thread the top level comment is a wiki so that the community can update it with relevant info, and thenā¦
After much late night hackingā¦ I now have a live timeline cred prototype to share!
Hereās what the UI looks like:
[image]
The basic approach is that we slice the history of the project by week. For each week, we run PageRank and assign scores, with some time-based twists:
we set the edge weights based on time: edges that donāt yet exist have 0 weight, edges that were just created have full weight, and older edges have their weight decayed exponentially
we run PageRank with a seed vector poiā¦
What is SourceCred?
SourceCred is a reputation protocol for open collaboration. At a basic level, SourceCred tracks the contributions that are made to a community project. It does this by assigning each contribution and contributor a score, called āCredā, based on the contributionsā value to the project.
You can think of SourceCred as turning community collaboration into a game. To earn high Cred scores you need to make contributions that the community values. SourceCred itself is dogfooding Sā¦
Cred Boosting
Status:
Proposal
Champion? :
Initiative Description:
Letās add ācred bountiesā to the initiative system . The basic idea is that the initiative will have a bounty amount, and a defined āsuccess conditionā. Once the success condition is realized, we mint new cred (equal to the bounty amount), and flow it to the initiative node. From the initiative node, it will flow to the contributions, to the initiative proposer, and to the references.
The bounty getting paid out doesnāt necessariā¦
To Do
Figure out what qualifies as a reference and what actually contributed to the design and development of āCred.ā
Update the description.
Create a Cred overview post.
Thinking about this more, and āCredā is really hard to separate from āSourceCred.ā This is because Cred is a force of the system, so really itās a part of the system. Cred canāt exist on itās own without the SourceCred protocol, and the SourceCred protocol couldnāt exist if it didnāt measure contributions via Cred.
I would say in my mind the current distinction is that āCredā is really a shorthand for the āSourceCred protocol,ā whereas āSourceCredā refers to the entire SourceCred project?
https://sourcecred.io/cred/
SourceCred, as the name might suggest, is all about attributing cred.
Cred is a metric that describes every contribution and contributor in
a project, giving a sense of how important they were.
For example, forum posts (like this one) can earn cred. If this post earns 5
cred, and another post earns 10 cred, then that other post is considered
twice as important as this one. (Note: SourceCred doesnāt load
posts from this forum yet.)
The contributions are arranged in a graph , where contributions ā¦
An Overview of Grain : an initial overview of Grain. While this Grain design is an open design space, this sets the foundation for many of the resulting conversations around Cred/Grain dynamics
Disclaimer: This content is outdated.
Please see SourceCred in 5 minutes for the latest overview of the CredSperiment.
This post describes the initial design of the CredSperiment, as envisioned in August 2019. Since then, the system has changed materially. Please view this document as a piece of history in SourceCredās evolution, but not an up-to-date view of how SourceCred operates.
The CredSperiment creates a game in which people are rewarded for contributing to SourceCred, based on Sourceā¦
What is SourceCred?
SourceCred is a reputation protocol for open collaboration. At a basic level, SourceCred tracks the contributions that are made to a community project. It does this by assigning each contribution and contributor a score, called āCredā, based on the contributionsā value to the project.
You can think of SourceCred as turning community collaboration into a game. To earn high Cred scores you need to make contributions that the community values. SourceCred itself is dogfooding Sā¦
https://github.com/sourcecred/pm/blob/master/overview.md
burrrata:
Thinking about this more, and āCredā is really hard to separate from āSourceCred.ā This is because Cred is a force of the system, so really itās a part of the system. Cred canāt exist on itās own without the SourceCred protocol, and the SourceCred protocol couldnāt exist if it didnāt measure contributions via Cred.
I agree, I think that āCredā is a somewhat awkward artifact because itās so integrally a part of SourceCred.
Maybe a clearer artifact would be something like āCred algorithmā or āSourceCred system designā. (Or maybe ācred algorithmā is a sub-artifact within āSourceCred system designā.) Then it would be a bit clearer that weād be pointing to dependencies like the PageRank paper, the initiative for prototyping credv1, the initiative for adding timeline cred, and so forth.
What if we thought of the āSourceCred protocolā as the graph and the way to attribute Cred scores to nodes on that graph?
Then everything else thatās a derivative off of that graph and itās scores (like Grain) could be part of the āSourceCred gameā (or āsystemā if the word āgameā is unpopular lol).
This would allow the system to change as we design new features and plugins, but the heart of SourceCred would always be the core protocol that creates a graph and assigns Cred scores based on nodes on that graph.
Related tread that explains more how Cred will flow through the SourceCred protocol
This post describes an incentive incompatibility in SourceCred today, along with a concrete plan to fix it.
Letās suppose weāre interested in nodes that have two types of edges: AUTHORS edges and REFERENCES edges.
Youāre the author of one of these nodes. Having written it, you know there are 9 relevant citations it could reference.
Letās suppose that weāre in the current iteration of SourceCred, and AUTHORS and REFERENCES both have 1x weight. Your options:
Hide all the references, and only ā¦