Supernodes: Moving past raw activity

The CredSperiment Research
1

So far, SourceCred has tracked contributions at the level of raw activity. Here are some examples of activities we track:

  • GitHub pull requests
  • GitHub comments
  • GitHub Reactions
  • Discourse posts
  • Discourse likes

The activity data has been a great starting point. Activity data is easy to collect automatically, so we can get a reasonable baseline for cred scores.

However, using activity data alone has serious issues. Valuing any individual piece of activity can be quite hard, and the activity data is so fine-grained that it’s often illegible. Also, valuing everything in terms of raw activity makes it hard for the community to decide to reward specific goals or values. For example, suppose the community wants to prioritize “usability” or “documentation”. It’s hard to express those goals by changing weights on pull requests or Discourse posts.

Supernodes

This post explores how we will use manually-curated “supernodes” to fix this, by creating a new, coarser-grained level of the Cred graph that’s amenable to explicit curation by the community. The first types of supernodes I have in mind are Initiatives and Artifacts.

Initiatives

An Initiative is a specific workstream that creates value for the project. You can think of an initiative as a flow of value.

Examples of initiatives include:

Artifacts

An Artifact is some durable or long-lived piece of a project. You can think of an initiative as a stock of value within the project.

Examples of artifacts include:

  • SourceCred as a whole (the “meta” artifact)
  • The SourceCred Discourse plugin
  • The SourceCred podcast (the whole collection / long running program)
  • The SourceCred whitepaper (if we ever write one)

Supernode Relationships

I expect supernodes to flow cred to each other, and to individual pieces of activity. Here are some examples, focused on specific artifacts.

Podcast Artifact

  • Any particular podcast episode flows cred to the Podcasts artifact
  • The podcasts artifact flows cred to specific contributions that helped set up the podcast
  • The podcasts artifact also flows cred to every individual podcast (cylical cred relationships are OK!)

Artifacts Artifact

  • Any individual artifact flows some cred to the “Artifacts Artifact”
  • The artifacts artifact flows cred to this post (for helping define it)
  • This post flows cred to the Discourse artifact (because it is a Discourse post itself)
  • The Discourse artifact flows cred to Discourse, the open-source project and company
  • The Discourse artifact also flows cred to this initiative for adding an important feature to the Discourse plugin

Both of these examples focus on specific, concrete parts of the project. But we could also use artifacts to identify more abstract kinds of value in the project. Here are examples:

Usability Artifact

  • Pull requests like #1431 which make the project more usable can be directly connected to the Usability Artifact.
  • There will be a bit more judgement / subjectivity in deciding which contributions affect usability, and how strong the weights should be.
  • For now, I’ll act as the arbiter if we have contentious disputes, but we’ll want to come up with a better governance mechanism before long.

Community Artifact

  • We could have a general artifact representing the SourceCred Community.
  • It can be connected to other artifacts that are important to the community–e.g. the artifact representing our Discourse forum, the podcast, the website and documentation, etc.
  • As with the Usability Artifact, deciding what connections should exist may be contentious.

Representing Supernodes

For now, we’re going to represent supdernodes via Discourse Wikis within special categories. See the Initiatives category and the Artifacts category.

Later on, we’ll likely build a dedicated UI for viewing and editing supernodes. However, bootstrapping on Discourse makes the initial prototyping a lot faster and easier.

Modifying Supernode Connections

For now, we’ll manually manage the supernodes’ connections by adding links to the post containing the M-node. For example, you can look at the Initiative template. It has dedicated sections for adding links to references, links to contributions, etc.

For now, anyone can edit the supernodes, adding new links, etc. As the community scales out of Trust Level 1, people may start to add spurious edges. This will be easy to detect and revert; once it’s a consistent issue, we’ll build a better permissioning model for editing those nodes.

Over time, we’ll need to develop clear policies and frameworks for assessing what contributions should be connected to supernodes, and what the strength of those connections should be. For example, both this PR adding Discourse references and this PR capitalizing the Discourse plugin name should both be connected to an artifact representing the Discourse plugin. However, they should have very different weights.

For now, I want to crowd-source the connections and weights, and in the event of controversy/disagreement I’ll step in to make a decision.

Valuing Supernodes

Because supernodes are much higher level, they are easier to explicitly value. For example, we could set an explicit and high Cred score that flows from a “Documentation Artifact”, so that everyone who writes documentation will earn a lot more cred.

Supernodes will also be great candidates for boosting, whereby someone can increase the (Cred) valuation of an supernode by burning some of their Grain.

Of course, like anything else in SourceCred, a lot of the value of supernodes will be derived from their connections. For example, if the “Founding SourceCred” initiative is connected strongly to the “SourceCred meta-Artifact”, then it will earn a lot of Cred, even if no one has explicitly boosted “Founding SourceCred”.

Since Cred is time-specific, there’s an open question of when supernodes should recieve cred. In the case of Initiatives, there may be a clear “completion” date, and we can have the Cred flow from the node as of that date. But for Artifacts, which may have very long lives, I’m not sure in what timestep the cred should emerge.

Retroactive Coverage

One of the best features Cred is that it updates retroactively. That means the community can always go back and do a better job of valuing past contributions.

In the case of SourceCred, as we build out the infrastructure for representing supernodes, I’d like us to go back through SourceCred’s history and create all of the missing initiatives and artifacts. It will be a lot of work, but it will leave us with a Cred instance that is more legible, more fair, and a much better foundation to build from as the community scales. In particular, people who have contributed via ideas, conversation, and advice are currently under-credited, because we don’t see their activity. Via supernodes, we can more directly connect people to the value they’ve contributed.

Creating all of those Initiatives and Artifacts will take some effort. Naturally, that work will itself be recognized and rewarded by SourceCred itself! Each supernode will flow some of its cred to the contributions that upkeep the supernode itself. (E.g. the work of adding new links to the supernode, refactoring supernodes, etc).

Making it Grain

Once we’ve achieved good retroactive coverage via updating past supernodes, we will have a cred distribution that is more robust and more accurate. That will be a good moment to celebrate. Once we achieve this milestone, I think we should “make it Grain” by paying out a very large amount of Grain to everyone based on their updated cred. See here for some related ideas.

5 Likes
2

Super nodes, initiatives and artifacts seem like a fairly easy to understand, robust abstraction layer (which is needed to simplify the UI for most users).

It feels like this new meta structure is maybe enough for now? Best to implement and play before creating more? Perhaps I’m just feeling slightly overwhelmed, as I’ve been busy the last couple weeks and I haven’t had as much time as I’d like to follow developments.

I like the idea working towards a big milestone that unlocks a lot of Grain. $100k seems about the right level of boost…My only feedback there would be that, to be properly motivated to work towards such a big payment, I think I’d need some more clarity on the payment guarantees. I still see only ~$5.7k in the OpenCollective budget, and while a promise on the social layer is good, from years of freelancing, I know there’s often just a lot of variables at play, many outside the control of the parties involved. Want to reduce counter-party risk as much as possible. Now if this were a smart contract with cryptographic guarantees, things get really interesting…but I digress :slight_smile:

1 Like
3

Yeah, I think the new meta structure will be all we need for a while (although polishing it and making it intuitive and easy to use will take some time). After this lands, I’m not planning to add new abstractions for a while. (Though maybe we will prioritize boosting along the way, I think it will be really fun to play with.)

The OpenCollective budget won’t be a good proxy for PL’s funding commitment; putting money through that interface is just way too expensive (10-15% overhead). I could ask PL about making a public post announcing the funding commitment though, possibly just as a post from the CFO on this forum. Would that work for you?

1 Like
4

If PL has already earmarked money for this internally, that is enough for me personally. Just wanted to know the wheels are in motion. If someone from PL wants to post publicly about it, that may increase confidence for others, though afaik I’m the only one asking about this.

5

One thing I would like to watch out for, is an over-reliance on these supernodes vs the “out of the box” accuracy. One of the aspects I find tremendously valuable about SourceCred is to quickly gain insights into projects you’re not familiar with, regardless of whether they have SourceCred currently set up.

Of course that wouldn’t be lost by gaining these capabilities, but I’m wondering if there’s a suitable way to ensure we don’t lose sight of the other use-case quality going forward.

2 Likes
6

Right now I imagine us building a “cred editor” (“creditor?”) tool with a workflow like follows:

  • Run SourceCred on your OS project. It produces initial cred based just on PRs, discourse activity, etc
  • Creditor invites you to define a few supernodes (artifacts) that represent the most important aspects of your project
  • For each supernode, creditor lets you search for individual contributions (e.g. PRs) that should be connected to the supernode
  • Within each supernode, creditor lets you select a group of related contributions and group them into a new supernode (an initiative)
  • For each supernode, the Creditor also lets you add “manual contributions” (catch all for activity off GitHub)
  • Within each supernode, the Creditor lets you see how much cred is flowing, and tweak it

The result would be that you can start with no manual action, and then progressively improve the cred distribution via the editor.

1 Like
7

While that sounds like an easy way to do editing. What I meant is, what can we do to guard the quality of results we get without any editing.

8

This would be a great way to simultaneously improve documentation and draft a light-paper.

Could totally set up a DAO to do this in an afternoon. PL buys DAI and puts it in the DAO’s vault. Cred is calculated off-chain and then manually minted to contributors via the DAO’s token manager app on weekly basis (@decentralion could do this as they already do this without a DAO). Key decisions could be voted on (or signaled) via cred weighted voting. The DAO could use the redemptions or fundraising apps to allow cred to be exchanged for Grain. All this could happen within the DAO. Would take a few hours to setup.

9

Not sure how close SourceCred is to creating a token and DAO, but it’s certainly an exciting possibility. Have spent a couple hours here tinkering and wanted to share some initial impressions and think out loud. Plan on diving deeper next week. Take with large grains of salt, as I’m just getting into the guts of this stuff.

Aragon smart contracts and tooling seem to be maturing… Played around with a couple DAOs, and the UI was surprisingly painless (at least compared to other blockchain UI). Smart contracts have passed extensive security auditing iirc, and MetaMask is better since the last time I used it. Project did just do a bunch of major launches last month (@burrrata must have been busy!). There could still be some kinks to work out, but feels like it’s finally becoming stable enough to start building on…I still have some reservations about putting too much money on a pure PoS chain, but that’s another bikeshed to paint:)

So the r/EthTrader subreddit’s Aragon DAO is quite interesting. It basically has all the basic features we’re talking about here for a SC DAO.

  • Has a non-transferrable reputation token (CONTRIB) based on contributions to the r/EthTrader subreddit.
  • Mints a transferrable ERC-20 token (DONUTS) based on reputation.
  • Just did an airdrop based on past contributions, where people were given a window of time to claim their DONUTS.
  • Has polls/voting
  • Owners of DONUTS can affect change in an off-chain, centralized system (Reddit, which offered some engineering support to the project), namely buying the banner ad space on the subreddit. Users can also tip each other, and will be able to purchase ‘special membership’ soon, presumably giving them more features or permissions.
  • Token is trading on Uniswap DEX, which trivially easy to set up for ERC-20. The token just needs liquidity providers (of which there appear to be none yet for DONUTS?).

Have gone through the flow of claiming my DONUTS (all 95 of them (out of 46M), presumably from a comment I made two years ago and forgot about ¯_(ツ)_/¯)–took less than a minute, and onboarding was surprisingly easy. Poked around the DAO and tipped 1 DONUT to its creator, which was also easy and seamless, cost me $0.03 in gas to do the transaction on the Ethereum mainnet.

This will be a good project to watch, as it is tackling some of the same challenges SourceCred will face if it does something similar.

I am wondering how they integrate CONTRIB/DONUTS with Reddit, as we’ll presumably need to do something similar with boosting.

In general, I think that running the SourceCred algorithm off-chain, and distributing tokens manually in the UI is not a major issue. While a fully autonomous smart contract would be very powerful (and inevitable on a longer timespan), at this stage, I think it’s fine to trust our TBD (Temporary Benevolent Dictator) @decentralion, as we are already with the prototype. There exist several paths to further decentralization from here. The most obvious and easy to implement IMO would be the community voting on any distributions proposed by Dandelion. That way if Dandelion isn’t calculating Cred or minting Grain they way they said they would, or the community wants to re-negotiate, it can veto the transaction. I believe the standard Aragon DAO voting and finance apps already have this capability. Another, perhaps longer-term path would be using oracles. E.g. the smart contract calls a Chainlink (or other) oracle service run by an independent third party that updates the scores.

Risk here could also be mitigated by forking. As SourceCred is still currently centralized in its dev resources and funding (@protocol is the only buyer of Grain currently afaik), if we wanted to we could just blow away the DAO and re-mint the Cred/Grain off-chain, start over. Though if the token is out in the wild and trading, there’s a possibility it continues as a zombie coin, or gets picked up by another party. Seems like a remote possibility for now, but SourceCred does have some buzz right now. We don’t want to attract scammers that try to co-opt that into some pump and dump scheme or something. If there was another legitimate party that wanted to run with it (it is an open source project), then I think you’d just want to be sure “SourceCred Cash” (SCC) is clearly dileniated in the market. Getting a bit ahead of myself now, but if Grain becomes widespread, issues around network effects, exchange support, IP (e.g. ticker), will arise. And I do believe we should design SourceCred to be forkable without undue technical barriers. This ability to exit will hopefully give the community a voice should the token be captured by interests not aligned with the community’s values.

OK, stepping back slowly from the rabbit hole…:slight_smile:

1 Like
10

The DAOnuts experiment is super cool. Reddit integration required coordination with the Reddit devs. If we wanted to do something similar to integrate an Aragon DAO into Discourse that would require some hacking. Would be a very cool project. Since Aragon also uses Discourse this could be a win win all around. Also, we (mostly me) plan to explore integrating SourceCred into the Aragon Discourse as well. If we had both SourceCred and DAO integration into Discourse that would be a dream come true :slight_smile:

11

Timescale matters

Right now I think Initiatives make a lot of sense to me as a supernode. But it doesn’t do well for this need to have “ongoing tasks”.

And TL;DR I think the difficulty to capture it lies in time.

Here’s my thought process.

Imagine this ongoing task: Managing the Twitter account. The way I would go about that is probably make a wiki that gives a description of this task. What we’re trying to achieve, some guidelines about how to do it, and who the current Champion is. Then probably we should have a log of the contributions. In the case of Twitter that would be the list tweets, who authored it, when they… wait a minute! This is just a Twitter plugin, we can add this to the graph! Right? Well yes, a Twitter account would fit very snugly into the graph. Perfect example of what would be a good plugin. What’s the problem then?

Granularity is the problem. We’re back to the situation where if you have a bunch of tweets in the graph. You either need to value individual tweets (too fine). Or the entire twitter account (too coarse). There isn’t a natural node in between that that’s just the right granularity.

Meet Initiatives; just the right granularity. Initiatives are exactly that size that has all the benefits @decentralion outlined. They’re a natural fit to this granularity issue, because they are larger and finite tasks. But even though they’re finite, the retroactive aspect is till important. I’ll get back to that.

Ok so what’s the granularity of a Twitter account? That depends on how long the Twitter account has been going for, in other words time. Obviously to answer the question: how do you rate the all-time value of the Twitter account? Is something you might be able to reason about if the account has seen a month of activity. You can go through all the tweets, get some stats about views and retweets and make a pretty sound argument why you think it should have a certain score. But if it’s 5 years old that is too much. You probably can’t remember 5 years worth of tweets at the same time. And putting a Cred score to that is going to be subject to all sorts of gymnastics our brains are capable of, such as vividly remembering things that never happened.

So here’s a suggestion. Why not group the tweets by time? I think the earlier point about being able to give a decent argument for a score for a month worth of tweets could be applied to any of the following months as well. We could just evaluate every month separately, couldn’t we?

Promised I would get back to needing the retrospect. Of course evaluating a month of tweets makes most sense when that month is over. But imagine this scenario. A few months in there’s a discussion happening and we have some different ideas about what the tweets are for and want to change the guidelines for it. You could imagine someone saying “the kind of tweets we did back in august are a good example of this and were really helpful”. Having a “Tweets in Aug '19” supernode would make it really easy to go back and give extra Cred.

Champions would make sense again too. The same granularity and time issues apply to Champions. I think from the Initiatives and Champions discussions so far, it’s safe to say that having a Champion increases the value of the Initiative beyond the sum of it’s contributions. But saying who our Twitter Champion is for that 5 year old account sounds like it would cause a civil war, while I think agreeing on who the Aug '19 Twitter Champion was should be achievable without bloodshed :smile:.

The best time frame probably depends on what we’re talking about. Maybe weekly is better for high volume things. Maybe quarterly or annually for low volume things?

I think a nice side effect of arbitrary time frames is that they’re logical moments for a Champion to step down if they’re getting overburdened for example or if there’s a new candidate offering to help. For a quarterly sliced task, the end of the quarter is a great moment to decide whether you want to do this for another quarter.

2 Likes
12

Riffing off this time-sliced ongoing task idea a bit more. I think this would make it easier to capture intangible things as well.

Let’s take “Being a friendly and welcoming community” as an example of that. Having a list of all acts of friendliness that will fit in the graph is not likely to happen. However, we could set out to make a quarterly status report on this goal.

It could include polls of how people feel we’ve scored in this regard recently. It could involve someone taking the time to investigate and highlight some cases. We could have some stats that might be an indicator. Such as number of newly signed up and active forum users. And it makes sense to have Champions who commit to promoting friendly and welcoming behavior for a given time period. The periodic report might also identify and attribute Heroes.

(Of course creating polls, investigating and writing reports are tasks that in itself could be goals we incentivize.)

1 Like
13

Circling back to highlight a few important things.

First, when I first read through this it didn’t click. After talking to @decentralion today the idea of Supernodes (literally “super” nodes on the SourceCred graph) makes a lot more sense. We can insert nodes on the graph and flow Cred to them and as more people reference those nodes more Cred flow to them. This can be Initiatives, and lots of Initiatives can combine into Artifacts: a Supernode of Supernodes. Before I was thinking from the perspective of the Discourse UI and the connection didn’t make sense, but thinking about it from the perspective of the SourceCred graph makes sense. Would be really cool to create a Supernode visualizer like @decentralion mentioned so that people could see this right away.

Second, after talking to @decentralion and also @s_ben about how to turn Initiatives into Artifacts it’s becoming clear that the temporal aspect of Artifacts is incredibly important. Reading through this thread the first time I didn’t quite appreciate the contributions @Beanow brought to the conversation. Frankly I still don’t get how we’re going to track contributions through time, but I’m glad we’re exploring how that might work.

14

Thanks for this. Have been confused on this as well. Super, or “meta” nodes make sense…though still not sure what that means in practice. For instance, I’m supposed to turn the podcast into an artifact. But the temporal aspect confuses me. The podcast makes sense as an initiative I champion, but now that I’ve made three, that’s an artifact? Even though it’s an ongoing thing? Where is the distinction exactly and who makes that distinction?