DAO Reputation System Challenges

The CredSperiment Research
1

SourceCred is primarily a reputation protocol. This could be extremely useful for DAOs, which typically allow for pseudonymous participants and need a way to gauge the trustworthiness of various inputs to the system. Below, I attempt to define the general challenges DAOs face when using reputation systems. Goal is to develop this into an article on how SourceCred could work with DAOs, with integration ideas/examples added, perhaps merge these ideas with @decentralion’s SourceCred and DAOs: A Sketch if it makes sense. Any feedback, ideas welcome. This is such a broad topic and I’m having a hard time focusing on what direction to take.

(1/2 finished rough DRAFT) Reputation Protocols for Decentralized Autonomous Organizations (DAOs)

2019 is indeed shaping up to be the year of the Decentralized Autonomous Organization (DAO). The term DAO, once tainted by the disastrous DAO hack of 2016, has regained its luster, with an explosion of new DAOs, DAO platforms and tools to support them. In this post, we explore what a DAO is exactly, the challenges they face and how SourceCred fits into the emerging DAO stack.

What is a DAO?

What constitutes a DAO is a subject of great debate. Fundamentally, a DAO is an organization that is both autonomous and decentralized. Autonomous in the sense that there are hard-coded rules that define how the organization behaves. This can take the form of smart contracts executing pre-defined rules, members voting to enforce or change the rules, or some combination of both. The organization must also be decentralized; i.e. the DAO’s rules should not be defined and enforced by a centralized entity, but by members of the DAO. While decentralization exists on multiple dimensions, and always on a spectrum, it generally means that no centralized party (internal or external) can seize control of the organization. DAOs are typically permissionless, allowing members to be pseudonymous to avoid censorship.

Challenges

While DAOs sidestep many problems with traditional companies, and unlock potentially revolutionary new forms of collaboration and production, they face significant challenges. Many of the proposed solutions rely on reputation systems. Below we explore the challenges these reputation systems face, and how SourceCred could address them.

Coordination

DAO members are often pseudonymous, globally distributed and do not know each other very well. This makes it difficult for participants to build trust by traditional means. They must therefore must build trust by other means. Typically this is done using some form of reputation system.

Reputation systems typically employ one of two strategies: measuring reputation based on past behavior, or measuring reputation based on a trust graph (i.e. the trustworthiness of connections). Both of these approaches are vulnerable to Sibyl attacks, where an attacker creates a large number of pseudonymous identities and uses them to gain a disproportionately large influence.

If DAOs measure reputation based on past behavior, an attacker can fraudulently gain reputation by simulating good behavior, or by working for Sibyl identities that rate their behavior. If measuring reputation based on trust graphs, an attacker can gain reputation by connecting to their Sibyl identities.

Reputation-based networks such as Google, Facebook and Twitter defend against Sibyl attacks (fake users, bots, spam links, etc.) by using proprietary, closed-source data and algorithms trained to identify and kill Sibyl identities. DAOs, by definition, cannot do this without introducing centralization.

Governance

Perhaps the most difficult coordination problem DAOs face is reaching consensus on important decisions without relying on a centralized authority (i.e. decentralized governance). Most DAOs reach consensus via some form of voting mechanism. A wide variety of mechanisms have been proposed. From 1-person-1-vote, to stakeholder voting, to experimental new voting mechanisms such as Quadratic Voting (QV) and Knowledge-extractable Voting (KEV), DAOs are trying it all.

A major challenge here is that most voting mechanisms are identity based, opening them to Sibyl attacks. 1 person 1 vote, for instance, will be highly vulnerable to Sibyl attacks unless the DAO has highly Sibyl-resistant identities (unlikely). Voting mechanisms that do not rely on human identities (e.g. stakeholder voting, or 1 coin 1 vote), risk a “rich get richer” scenario, where large holders can use their voting power to collude and form cartels to exploit smaller holders–precisely what most DAOs are trying to escape.

To improve decision quality, many governance systems aim to verify not only that a voter is human, but that they have some trustworthy characteristic. For instance, in stake-based systems, voters must stake (put at risk) tokens, proving they have “skin-in-the-game”. In coin age voting schemes, older tokens are given more weight, assuming that long-term holders are more trustworthy, etc. While such approaches hold promise, if not designed properly, they can also introduce unnecessary complexity and expose new attack vectors, weakening the system.

Rewarding Participants

One of the most exciting possibilities DAOs present is the ability to more fairly compensate people for the value they create. But how to measure value? In what form is that value rewarded? This is perhaps the most difficult challenge.

DAOs often measure value using some form of “activity metric”, such as time spent performing a role ($/hr), units of work produced (e.g .lines of code) or measures of performance (e.g. usage of lines of code). This can introduce objectivity, but also suffers from a “flattening effect”, whereby value not measurable by metrics is not rewarded. Metrics will also be gamed. The problem is amplified in a DAO, where participants are often pseudonymous (free from IRL consequences), and often engaged in activities that transcend traditional employment roles. Members are free to add value in different domains, at different skill levels, and collaborate in new creative ways; if they are not, the value prop vs traditional companies diminishes. Therefore, DAOs must have a way to value contributions in a way that is not over-reliant on traditional roles or task definitions. DAOs must also be careful not to incentivize behavior that is not in alignment with the organization’s goals or values (#xrpthestandard). Here, reputation systems can be helpful, as they tap into the human capacity to see value beyond what is capturable by metrics.

1 Like
2

I see this as a promising jumping-off point. Assuming that cred works well (i.e. its robust to gaming), then “proof-of-contribution” is a sibyl-resistant way to allocate governance power within a community. But it’s a lot more egalitarian/meritocratic than 1-token-1-vote. I’m also reminded of this other post: SourceCred as Store-of-Community Value (SoCV)

A general question: what’s holding you back from writing the second part of the essay? (The piece that focuses more on SC.) I’d be curious to see a stab at it, even if it’s just a bunch of bullet point exploration of different ideas.

Also, feel free to borrow from / reuse the ideas from my SourceCred + DAOs sketch. It’s not plagarism so long as you leave a cred arrow pointing back. :wink:

1 Like
3

This section on governance feels weak. 1coin/1vote and KYC/identity based voting is the most naive version of DAO governance. There are many token models that create balances of power so that no single party has all the control, but participants can remain anonymous. Not sure covering that is a goal of this post, but if it is I’m happy to contribute. It’s kind of the main thing I’ve been focusing with this “burrrata” experiment: contributing to and building these systems so that anyone anywhere can communicate and coordinate in a trustless permissionless decentralized way :slight_smile:

This is huge. A lot of people who are unfamiliar with the space try to map traditional business models onto DAOs, but they’re missing the point. The unique value prop of using a DAO over traditional methods is all the new things that you can do that were not possible before. More research and awareness around “business model innovation” is essential to create adoption, so I’m glad you bring this up :slight_smile:

This is awesome. I actually missed this the first read through, so thanks for catching this @decentralion! I think that this point should be emphasized: the point is not to map addresses to government issued identities, but to verify that nodes in the network are contributing real value and to align incentives for all parties involved. Then it doesn’t matter if an account is an individual contributor, organization, or bot - anyone/anything can contribute

1 Like
4

Stealing this:)

cred-> @decentralion

OK, here’s a stab at some bullets (so violent…)…

Due to the richness of SourceCred’s graph, and the open, plugin structure that allows filtering and valuation by any heuristic, SourceCred can produce metrics that express nearly any community value. Instead of being limited to Proof-of-Work (PoW) or Proof-of-Stake (PoS), SourceCred can produce Source-of-X (insert community value), with surprising robustness.

  • Proof-of-contribution: we give more votes to someone who has contributed to the project.
  • Proof-of-”work”: we give more votes to someone who has worked more relative to others
  • Proof-of-committment: we give more votes to someone who is a long-term contributor
  • Proof-of-domain-expertise: we give more votes to someone who as contributed more in a particular domain
  • Proof-of-generalist: we give more votes to someone who has contributed across many domains
  • Proof-of-ideation: we give more votes to someone who has originated ideas that are later implemented
  • Proof-of-execution: we give more votes to someone who has coordinated others to execute ideas
  • Proof-of-bridge-builder: we give more votes to someone who brings people into collaboration that are normally separate
  • Proof-of-ally: we give more votes to someone who has cred in ally projects
  • Proof-of-biz-dev: we give more votes to someone who has introduced the project to a connection that led to new business
  • Proof-of-value-generation: we give more votes to someone who generates the most cred (or increases its value in other projects)
  • Proof-of-generosity: we give more votes to someone who gives away more of the cred they generate than those that hoard it (e.g. by only interacting with other higher-cred contributors)
  • Proof-of-onboarder: we give more votes to someone who has had early interactions with new contributors that stick around
  • Proof-of-defender: we give more votes to someone who has identified and killed Sibyl identities (bots, malicious actors, etc.)
2 Likes
5

When I was writing this, I was imagining some DAO people might react this way:) I thought about adding more here, but its such a deep topic I don’t want to derail the piece. Actually, a separate piece just exploring the different voting mechanims/governance structures and how they relate to SC could be interesting. I’m hesitant to commit bandwidth right now, but if there was a magical valuation machine that meant the article would be worth something, even if just a couple dollars…which didn’t require me to negotiate any contracts, and could earn me cred (and potentially $$) in the future if the piece was referenced by other contributors and the project took off…:rocket:

Here is some text I wrote re: governance issues generally, which I scrapped due to length. Perhaps it can be of use:

Newer voting mechanisms such as Quadratic Voting (QV) may be promising, but are largely untested, introducing risk. Many also introduce complexity, in the form of new, unoptimized variables and equations (quorums, dynamic vote durations, quadratic weighting, etc.). This complexity can lead to voter confusion and the introduction of new attack vectors.

Another big challenge is voter apathy. Most governance experiments in the cryptocurrency space, for instance, see very low voter turnout. To be incentivized, voters need skin-in-the-game incentives (they must be affected by the outcome), a voice (their vote must actually affect the outcome), an understanding of the voting process (if the mechanism is new, they must be educated), and a well-designed voting experience (good UI, proper level of abstraction in decision making, etc.).

Ideally, these challenges have been addressed before the governance system has launched. It is often impossible to make major changes to the rules of a governance model once it is established, as those in positions of power are incentivized to oppose it. Many crypto projects are currently learning this the hard way.

6

I was thinking the same thing lol. Happy to jam anytime tho :slight_smile:

Re: QV, UX, and apathy… this can all be solved by starting first with small, fun, community driven experiments. Almost every (literally) conversation about DAOs and voting mechanisms gets derailed by thinking “oh well it won’t work in every scenario so let’s not.” This, I think, mostly comes from fear of the unknown. This can be mitigated by starting small, with a group of people who understand the mechanisms and want to participate, then growing from there. Rome was not built in a day and software did not eat the world in a day either. It takes time to grow these things, but we have to start somewhere. Low risk experiments with highly engaged communities are the best way to go. Start with a Rinkeby DAO, test it out, break it, refine it, move to main net, use it, get comfortable with it, add funds, decentralize.

  • TL;DR: signalling => decision making => fund management
3 Likes
7

I like this direction and the examples you came up with. I think it’s a great illustration of the potential flexibility of SourceCred, and a good “imagination primer” for people who are encountering SC for the first time and figuring out how it can be useful.

It would be cool to see each of these examples fleshed out a little bit more, with examples on how these would tie into the SourceCred graph structure.

1 Like
8

Ditto everything @decentralion said. Awesome way to make the abstract dynamic potential of SourceCred more tangible. A few demos/experiments showing/exploring this would really be great :slight_smile: