Description
Cred represents contributions to a project. Projects are constantly evolving, and as they do so does Cred.
Source: the source code (data) of a project that is represented as a graph.
Cred: scores that nodes on the graph receive based on contributions and engagement.
SourceCred creates a graph over every project with nodes represent actors and items. SourceCred then gives those nodes scores according to a PageRank style algorithm. As people contribute to a project the graph will change and Cred scores will be updated accordingly. Cred is a living representation of contributions to a project.
We want open-source contributors to earn a living, and a share in the value created on top of their contributions. Tracking contributions in a living graph that is updated over time allows people to be rewarded for the value their work provides, today and into the future.
We’re designing the SourceCred protocol around the following principles:
Transparency
It should be easy to see why cred is attributed as it is, and link a person’s cred directly to contributions they’ve made.
Extensibility
SourceCred is designed around a plugin architecture, so you can add support for new data sources, new algorithms, or even entirely new kinds of work.
Decentralization
Each community has the final say on that community’s cred. When the algorithm and the community disagree, the community wins. Projects own their own data, and control their own cred. The SourceCred project provides tools, but has no control.
Contributions
So many… I’m honestly not even sure where to start. Would the entire SourceCred repo qualify here? Or is that a dependency?
The SourceCred prototype allows you to check your Cred scores.
opened 12:58AM - 05 Dec 18 UTC
closed 05:33AM - 06 Aug 20 UTC
We should have mission plan for SourceCred.
I've put a preliminary outline b… elow. I'll try to convert this into a longform document (likely checked into Git in this repository) soon.
Thanks to @meiqimichelle for prompting this.
cc/ @flyingzumwalt @sternhenri @wchargin @momack2
---
# 1. Vision/Setup
- Credit Attribution: The problem SourceCred is trying to solve
- Why this matters / What it enables (paying open source, better-resourced communities)
- Vision for how SourceCred will look + operate in the long run
# 2. How SourceCred Works
- PageRank algorithm and the graph based approach
- Customizability + flexibility through...
- Plugins which add new nodes (w/ example)
- Plugins which add new kinds of edges (w/ example)
- Heuristics on the values of nodes and edge (w/ example)
# 3. Current State of the project
- "Late Alpha" type software that proves the general approach (link to prototype)
- Strong technical foundation (thoughtfully architected Graph class, great test coverage, static typing)
- Uses data mostly from GitHub (issues, pulls, comments, etc), few plugins
- Decent job with identifiying top contributors, but struggles to accurately assess value of specific contributions
- e.g. it can't tell which commits are important
# 4. Key Risks / Issues
- SourceCred might be easy to game
- Discussion of how social norms constrain gameability, hence focus on transparency, but it needs to be proven out
- SourceCred might fail to appreciate non-technical contributors
- SourceCred might produce weird social norms / disrupt the social ecology of open-source projects
- The PageRank/Cred Graph approach might not be flexible enough to accurately value contributions
- GitHub could restrict access to their APIs
# 5. Key Initiatives
- Grow the project
- Increase # of people working on / contributing to SourceCred
- Increase # of people using SourceCred, i.e. increase adoption
- Improve credit attribution on open-source projects
- Add features like time-based cred and code analysis so that SourceCred is dramatically better at credit assignment on a median open-source project
- Improve credit attribution on SourceCred itself
- Use SourceCred as a testing ground for new techniques that require engagement from the community but could be a lot better overall.
- Try to prove that SourceCred can reward all types of work, not just coding, by proving it on SourceCred itself
https://github.com/sourcecred/pm/blob/master/overview.md
References
I feel like almost everything related to SourceCred and the CredSperiment could be in this section lol
https://sourcecred.io/cred/
I want to really understand how SourceCred works. While there’s a lot of high level overviews in various places, there isn’t a protocol specification or whitepaper. Starting this thread so that we can aggregate reference material useful to help people understand SourceCred and also draft a protocol specification that explains how SourceCred actually works
Tonight, at a dinner that @jbenet organized, he pointed out the importance of using SourceCred to showcase best practices around fairly and freely sharing cred, and suggested that we should start developing this practice soon, while cred is still young. In the ensuing conversation, I realized we already have all the tools we need to start documenting it, and created the Spotlights category on this forum. Thanks Juan!
This thread is intended for defining Reference Cases for Credit Attribution.
The Original reference cases discussion was for a line graph or chain of dependencies from an early discussion with @decentralion and @miyazono from @Protocol Labs. This reference case and others are briefly explored here using some preliminary research infrastucture. See GitHub issues in SourceCred Research for how we are testing and expanding our infrastructure.
There is not necessarily one right answer for how cre…
At dinner with @3blue1brown tonight, I described the weight composition problem to him.
He suggested viewing the weights / heuristics as unit vectors (in node-space), and viewing adding heuristics as weighted interpolation between the current vector, and the vector defined by the new heuristic. If the new heuristic isn’t defined for every node, then you just interpolate on the dimensions the new vector is defined for, while staying on the unit simplex in the whole space.
I haven’t fully grokke…
Placeholder text that is at least 20 characters.
SourceCred contributor payouts are calculated via Cred scores.
As per this thread, creating a thread to highlight properties of Cred and Grain. The idea is to aggregate conversations and data from this forum into threads that explore a theme.
If you want to add any relevant data points and/or link to relevant content from the forum, please do! Every post is a wiki so you can edit freely.
This is an experiment, so set your expectations accordingly.
This is a really interesting thought experiment!
I’ve spent a lot of time thinking about how to do token distributions based on cred. The systems I’ve had in mind have involved creating new project-specific cryptoassets called Grain and doing distributions at the project level, with large rewards, large upfront commitments, and requiring solid buy in from the project maintainers. One thing I really like about your approach is it’s something we could launch soon (e.g. this year), with very small…
Disclaimer: This content is outdated.
Please see SourceCred in 5 minutes for the latest overview of the CredSperiment.
This post describes the initial design of the CredSperiment, as envisioned in August 2019. Since then, the system has changed materially. Please view this document as a piece of history in SourceCred’s evolution, but not an up-to-date view of how SourceCred operates.
The CredSperiment creates a game in which people are rewarded for contributing to SourceCred, based on Source…
Is SourceCred looking to measure all-time cred (i.e. an objective measurement of all contributors/contributions), or the current credibility of active members? It seems that implementing time-based cred (the most commonly asked for feature), would be more measuring the latter.
In the coming weeks, I’m planning to write a roadmap for SourceCred, which basically documents all of the initiatives currently in flight, and lays out the different priorities we can focus on. For example, here are some of the priorities we could focus on:
Documenting SourceCred
Improving SourceCred instance management (make it easy for anyone to run SourceCred)
Improving cred transparency and UI (make it easier to tell why someone has cred, or see cred for a specific week, etc)
Build the nex…
Implement Cred Weighted Prioritization
Status: proposal
Champion:
Initiative Description:
As discussed here , we’d like to use cred-weighted voting to prioritize SourceCred initiatives. We can then give the prioritized initiatives a nice cred bounty .
As a practical matter, I propose that we use cred-weighted, dependency-aware conviction voting.
Cred-Weighted
Voting power will be based not on number of votes, but based on the amount of cred behind each vote. We can experiment with different fla…
So, based on a couple of recent discussions in yesterdays’ office hours, and forum posts on:
What is the absolute ($) value of cred?
Are our current weights right ?
Diminishing returns curves for certain contributions .
Creating new cred for specific project goals / areas of improvement.
Our cred “minting” (node weight == new cred) system being pretty simplistic compared to the sophisticated cred “flowing” (pagerank).
Putting all those thoughts together, I came up with the idea that, maybe ou…
Greetings, credizens! I am very pleased to share a rough draft of SourceCred’s own cred, for the CredSperiment.
In contrast to the instances I shared in the CredSperiment Progress Report , we now have a combined instance which properly resolves identities across GitHub and Discourse.
You can check out the instance here . For posterity: If the url is down, you can also start any http server in the docs subdirectory of this commit .
My analysis of the scores
Here are the scores from the new instan…
In the Preliminary CredSperiment Cred mega topic, we had some interesting discussion around valuing code reviews.
@Beanow makes a really good point here. SourceCred should actively support and encourage the collaborative norms of open-source. However, the approach I described originally could have the opposite effect. If each PR is only “worth” a certain amount of cred, and it gets split between the author and reviewers, then authors may resent reviewers for “taking” some of “their” cred. Thi…
Maybe it’s just me, but it seems like there aren’t as many likes flowing around as before. If this is true, then this is a problem. Hoarding Cred will lead to low value Cred for all parties involved! The more we give Cred, the more people are likely to engage in order to earn Cred. The more people engage with the community and earn Cred, the more Cred they have to give away. This creates a virtuous feedback cycle.
Currently, as far as I know, nodes that don’t flow Cred out kind of just accumula…
Moving from a discord chat .
Currently it looks like you can farm cred by spamming.
I tried reading up on the timeline cred algorithm and from what I understand.
Nodes like issues, comments, PRs and PR reviews have a base value equal to the weight.
That value then flows to other nodes it has a relationship to, such as the repository and the comment author.
In small volumes of spam, this would mean so long as you don’t “feed the trolls” by avoiding interacting with these threads the cred gained…
The key challenge for SourceCred is keeping the cred graph accurate and up-to-date. So far, we’ve depended on a mixture of automatic processes (edges between posts and replies) and well-intentioned user behavior (adding references / citations to relevant work). These approaches have gotten us started, but they won’t scale.
For example, consider the SourceCred poster art . Since the poster uses the logo design, it should have an edge to the logo explorations . It doesn’t, because @LB didn’t do the…
Raw PageRank outputs a probability distribution. When used as scores for human display, they wind up being unintelligible, because a characteristic score would be 0.000007, which is hard to read and discouraging to the user.
To fix this, we started normalizing scores so that users’ cred sums to 1000. This has the advantage that users’ scores are usually readable numbers between 1 and 1000, and that scores are comparable across contexts. However, it also has some drawbacks. In very large repos (…
https://discourse.sourcecred.io/t/about-the-cred-feedback-category/16
https://discourse.sourcecred.io/t/flow-cred-feedback/44
Cred Analysis Notebooks
Status: help wanted
Occasionally new types of Notebooks are created, but there’s room for a lot more.
Additionally it’s a great starting point for new contributors.
Please give a shoutout if you’d like to help.
Champion:
None atm.
It would be very helpful if a Champion could oversee the quality, organization and prioritized wishlist
of Notebooks.
Initiative Description:
Observable Notebooks have proven extremely useful tools for building new interfaces, experiences, and…
https://discourse.sourcecred.io/t/ropensci-cred-feedback/24
https://discourse.sourcecred.io/t/libp2p-cred-feedback/21
A thing that came to mind talking about the dynamic between cred and the second quantity.
Cred flow has had this situation where being someone with higher cred can create circular reasoning. This issue is worth more cred, because a high cred person engaged with it, earning the high cred person more cred. I’m sure this has been discussed a lot before.
Adding on top of this a second quantity, I wonder if this will amplify or dilute this effect. Would it sidestep the clout of high cred people bec…
Suppose that the SourceCred team goes to a hackathon, and you help me contribute there.
Suppose that you arrange my flights, so that I can actually make it to the hackathon.
Clearly, you deserve some cred.
How can we actually represent that within the SourceCred graph?
To start, we should make nodes in the graph that represent your contributions:
“arranging Dandelion’s hackathon logistics”. But how do we connect those nodes,
so that you receive an appropriate amount of cred?
I imagine that th…
We’ve had a few chats here about whether it makes sense to think of cred as a ‘currency’. I’ve generally said that cred isn’t a currency, because currencies are fungible and exchangeable, and cred is not, because cred is intrinsically tied to a specific identity and specific contributions. That’s why I’ve thought we should eventually create Grain as the currency that is derived from cred.
However, I just found an interesting post (Avoid Blunders in Designing Reputation Systems ) which distinguis…
Currently, we have a problem in SourceCred: we measure the value of activity,
but not the associated cost.
This means that SourceCred will always reward doing more–more pull requests,
more comments, more notifications, more activity, even if the cost of that
activity exceeds its value.
Here’s an example that @wchargin and I have discussed. Consider two pull
requests suggesting equally valuable changes to the codebase. One pull request
has been carefully prepared, and merges smoothly, with a si…
Been doing lots of research on DAOs over the years and decided to put it all in one place (a Discourse forum) so that people could engage with the material.
The way the forum is setup each DAO and/or platform has it’s own category. The pinned thread in each category is for the main DAO or platform, and then threads in that category are for forks and/or stuff built on that platform. In every thread the top level comment is a wiki so that the community can update it with relevant info, and then…
After much late night hacking… I now have a live timeline cred prototype to share!
Here’s what the UI looks like:
[image]
The basic approach is that we slice the history of the project by week. For each week, we run PageRank and assign scores, with some time-based twists:
we set the edge weights based on time: edges that don’t yet exist have 0 weight, edges that were just created have full weight, and older edges have their weight decayed exponentially
we run PageRank with a seed vector poi…
What is SourceCred?
SourceCred is a reputation protocol for open collaboration. At a basic level, SourceCred tracks the contributions that are made to a community project. It does this by assigning each contribution and contributor a score, called “Cred”, based on the contributions’ value to the project.
You can think of SourceCred as turning community collaboration into a game. To earn high Cred scores you need to make contributions that the community values. SourceCred itself is dogfooding S…
Cred Boosting
Status:
Proposal
Champion? :
Initiative Description:
Let’s add “cred bounties” to the initiative system . The basic idea is that the initiative will have a bounty amount, and a defined “success condition”. Once the success condition is realized, we mint new cred (equal to the bounty amount), and flow it to the initiative node. From the initiative node, it will flow to the contributions, to the initiative proposer, and to the references.
The bounty getting paid out doesn’t neces…
To Do
Figure out what qualifies as a reference and what actually contributed to the design and development of “Cred.”
Update the description.
Create a Cred overview post.
Thinking about this more, and “Cred” is really hard to separate from “SourceCred.” This is because Cred is a force of the system, so really it’s a part of the system. Cred can’t exist on it’s own without the SourceCred protocol, and the SourceCred protocol couldn’t exist if it didn’t measure contributions via Cred.
I would say in my mind the current distinction is that “Cred” is really a shorthand for the “SourceCred protocol,” whereas “SourceCred” refers to the entire SourceCred project?
Threads that somewhat answer the question: “What is Cred?”
https://sourcecred.io/cred/
SourceCred, as the name might suggest, is all about attributing cred.
Cred is a metric that describes every contribution and contributor in
a project, giving a sense of how important they were.
For example, forum posts (like this one) can earn cred. If this post earns 5
cred, and another post earns 10 cred, then that other post is considered
twice as important as this one. (Note: SourceCred doesn’t load
posts from this forum yet.)
The contributions are arranged in a graph , where contributions …
An Overview of Grain : an initial overview of Grain. While this Grain design is an open design space, this sets the foundation for many of the resulting conversations around Cred/Grain dynamics
Disclaimer: This content is outdated.
Please see SourceCred in 5 minutes for the latest overview of the CredSperiment.
This post describes the initial design of the CredSperiment, as envisioned in August 2019. Since then, the system has changed materially. Please view this document as a piece of history in SourceCred’s evolution, but not an up-to-date view of how SourceCred operates.
The CredSperiment creates a game in which people are rewarded for contributing to SourceCred, based on Source…
What is SourceCred?
SourceCred is a reputation protocol for open collaboration. At a basic level, SourceCred tracks the contributions that are made to a community project. It does this by assigning each contribution and contributor a score, called “Cred”, based on the contributions’ value to the project.
You can think of SourceCred as turning community collaboration into a game. To earn high Cred scores you need to make contributions that the community values. SourceCred itself is dogfooding S…
https://github.com/sourcecred/pm/blob/master/overview.md
burrrata:
Thinking about this more, and “Cred” is really hard to separate from “SourceCred.” This is because Cred is a force of the system, so really it’s a part of the system. Cred can’t exist on it’s own without the SourceCred protocol, and the SourceCred protocol couldn’t exist if it didn’t measure contributions via Cred.
I agree, I think that “Cred” is a somewhat awkward artifact because it’s so integrally a part of SourceCred.
Maybe a clearer artifact would be something like “Cred algorithm” or “SourceCred system design”. (Or maybe “cred algorithm” is a sub-artifact within “SourceCred system design”.) Then it would be a bit clearer that we’d be pointing to dependencies like the PageRank paper, the initiative for prototyping credv1, the initiative for adding timeline cred, and so forth.
What if we thought of the “SourceCred protocol” as the graph and the way to attribute Cred scores to nodes on that graph?
Then everything else that’s a derivative off of that graph and it’s scores (like Grain) could be part of the “SourceCred game” (or “system” if the word “game” is unpopular lol).
This would allow the system to change as we design new features and plugins, but the heart of SourceCred would always be the core protocol that creates a graph and assigns Cred scores based on nodes on that graph.
Related tread that explains more how Cred will flow through the SourceCred protocol
This post describes an incentive incompatibility in SourceCred today, along with a concrete plan to fix it.
Let’s suppose we’re interested in nodes that have two types of edges: AUTHORS edges and REFERENCES edges.
You’re the author of one of these nodes. Having written it, you know there are 9 relevant citations it could reference.
Let’s suppose that we’re in the current iteration of SourceCred, and AUTHORS and REFERENCES both have 1x weight. Your options:
Hide all the references, and only …