SourceCred Discord Plugin

Status: proposal

Champion?:

None yet. Help much appreciated.

Description:

Right now, it’s impossible to earn any cred from participating in our Discord. That’s unfortunate, since people definitely do create value there. An easy way to patch this would be to make certain reactions (e.g. the :sourcecred: reaction that @Beanow added) flow cred.

Concretely: we could have a plugin that reads the history of our Discord server, and finds every message with at least one :sourcecred: reaction. Then, it can create a node for that message, with edges connecting the reacters to the message that generated the reaction.

Before building this integration, as discussed here, we should make sure we want to commit to Discord. My sense is that since we’re using Discord for the community calls and the podcast production, we’re likely to stick with it.

Benefits:

It will make our Discord feel more cohesively part of the SourceCred plugin.

Implementation plan:

  • Explore Discord API and make sure we can get the messages and reactions
  • Write scraper (modeled on Discourse Plugin + GitHub mirror) that mirrors the Discord data into a local database
  • Create Discord plugin declaration, with message, reaction, and user types
  • Update identity plugin as needed with a new user alias type (discord/username).
  • Create a Discord createGraph method which generates the graph
  • Integrate plugin with loading, add necessary configuration options, etc
  • Possibly add another edge type (:sourcecred: and :supercred:?) for flowing more cred than a standard reaction. (Kind of like :+1: vs :heart:)

Deliverables:

  • SourceCred’s own cred will have a Discord plugin, with corresponding edges
  • There will be documentation on how other projects using Discord can activate this plugin

Dependencies:

References:

Contributions:

4 Likes

Discord would be a fun one to experiment with reactions for flowing cred. There’s 50 slots for arbitrary emoji’s like :sourcecred: we could play with to do this. Though since they aren’t standard, it’s important to have this as a setting, so other servers can use their own.

I’ve long thought that chat was a rich source of information. Exciting to see this

Created this emoji for Discourse just a few hours ago :slight_smile:

2 Likes

Ooh :sourcecred: I like it. Still think that UX wise Discord would be fun to try emoji experiments with, as you can use them for reactions. So you can just click them in the way that Discourse has likes and GitHub has reactions. Except Discord includes your custom emoji as reaction options. So you can come up with a lot of them with special meaning.

1 Like

Apparently we can do reactions on Discourse too, but for some reason the plugin isn’t loading through the admin GUI.

@decentralion do you know how to access the Discourse container/server that’s actually hosting this forum to install the plugin that way?

We’re on free hosting from Discourse org. I think that to install custom plugins, we would need to switch to self-hosted.

I thought that the self-hosted option was the free option?

Are we using one of the plans listed on the Discourse pricing page?

The Discourse folks were kind enough to give us a free instance that they are hosting.

1 Like

Doing some background work on this issue:

It seems getting the messages is possible: https://discordapp.com/developers/docs/resources/channel#get-channel-messages

Getting the reactions might be a little more onerous, as the reaction objects returned in messages don’t specify who created the reactions: https://discordapp.com/developers/docs/resources/channel#reaction-object

We could still get that info by making individual calls to a get_reaction route for each message: https://discordapp.com/developers/docs/resources/channel#get-reactions

@decentralion Do you think we should scrape the didathing channel (like I believe we mentioned on the last team call) or the whole sever/guild?

Something to consider is that the didathing channel is composed of a lot of references to things that will already have nodes, like reviews and pull requests.

1 Like

At the data layer, we should scrape everything. We may have a plugin config option to assign special meanings to special channels. Generally however, a valuable contribution can happen in any channel, so I think we should be scanning the whole server.

At the graph creation layer, I think that for now, creating a node and edges for every single message would be too much noise, especially since it’s not obvious what the edge structure between messages should be. Instead, let’s create nodes only for messages that receive :sourcecred: reactions with the following structure:

Nodes

  • message
  • user
  • reaction

Edges

  • user → creates → reaction
  • user → posts → message
  • reaction → reacts_to → message
  • message → references → (referencable)

Yep, the fact that we’ll get references is a feature not a bug – it means, for example, that if my posting the PR in didathing results in a lot of :sourcecred: reactions, then some more cred will flow to the PR.

This will be feasible thanks to @Beanow’s work on Unified Reference Detection.

I actually kind of like that the Discord doesn’t have any cred attached to it right now.

Our Discord server functions differently from our Discourse server. Discord is real-time and spontaneous, with interactions on the order of seconds. Discourse posts invite more thought: discussions happen on the order of hours to days at minimum, as people’s thoughts percolate.

I just sent a message in Discord that contained only the word “hi”, and followed it up with a message “what’s up?”. These aren’t exactly interactions that invite a rich structure of cred.

This isn’t to say that valuable and substantive interactions don’t come out of the Discord. They certainly can! In such cases, maybe the participants should create a “real artifact” on Discourse, anyway. Doing so:

  • …is itself an act of curation, signaling, “I think that this content is important enough that other people may be interested in it.”
  • …provides a venue for others to contribute to the discussion in a more structured form (e.g., with “quote reply” features, cross-linking, etc.).
  • …makes the content easily searchable and accessible even to people who aren’t logged in or don’t have a Discord account.
  • …encourages the post author to clean up and clarify the content, if necessary.

Of course, once it’s in Discourse, it’ll also be in scope for our normal Discourse cred.

There’s also a totally separate sense in which it’s nice to have a “cred-free zone”: it creates an environment without any weird transactional dynamics. I can freely use a :+1: reaction to indicate that I agree with a comment, or to vote in an ad hoc poll, or to simply convey that I’ve seen the message and don’t have anything to add, without having to think too much about what exactly I mean and what the ramifications of this are. I try not to give too much thought to these things, anyway, but they’re always there to some degree—and I know that others have felt this, too.

And, finally, a new plugin means new infrastructure: fetching APIs, persistence layer, graph generation, weight definitions, tests. That’s a serious amount of effort. I would want to be convinced that we’re gaining something valuable by taking this on.

What do you think?

2 Likes

@wchargin, I think we’re implicitly modeling very different scopes for the Discord plugin. The way I am thinking about it, only messages that receive :sourcecred: :sourcecred: reactions become nodes in the graph.

This means that Cred intersects with Discord only deliberately / intentionally rather than passively / implicitly. Discord would still be a different kind of space from Discourse–“Cred off by default” vs “Cred on by default”–but it would still be possible to flow cred based on the interactions.

In my mind, this is both an epistemically better model (some Discord interactions are valuable and do merit cred) and it feels more compassionate to SourceCred’s users. Saying “we aren’t going to allow anyone to value your contribution unless you go through the extra effort of writing a Discourse post” feels heavy-handed to me, and will bias cred flows towards people who are willing to both jump through hoops, and self promote.

In the past, some community members have expressed that they’re less comfortable posting on Discourse than on Discord; is it appropriate for people who mostly participate on Discord to not have any first-class way to get cred?

Also, to use a recent concrete example, you posted a helpful and informative message to the Discord earlier this week:

Floats are handled slightly differently in Python from JavaScript. Both attempt to conform to IEEE 754, but JavaScript has some edge cases. For instance, the spec says that pow(1.0, NaN) should be 1.0 , but JavaScript implements it as NaN .

You’re probably not going to hit these edge cases, but it is important to understand some of the intricacies of 754-compliant floating point arithmetic. Operations on floats do not have the properties that you expect: most famously, they’re not associative (1.0 + 1e-16 + 1e-16 is 1.0, but 1.0 + (1e-16 + 1e-16) is slightly greater than 1.0). So most of the trouble, I expect, is going to come from places where the Python and JS implementations are “clearly mathematically equivalent” with real numbers but not with floats. (For instance, summing a list of floats in forward order vs. in reverse order.)

It would be silly for me to write a Discourse post saying “thank you @wchargin for explaining some details about floating point arithmetic”. Both because the friction of doing would be a meaningful overhead, and because it would create a very low-signal notification. (SourceCred’s discourse audience doesn’t come to this site to read about floating point arithmetic; we shouldn’t build a system where regular community kudos crowd out deep discussion.)

While we’re still on the mint-on-reaction paradigm (cf Mint on Likes), I imagine that the Discord :sourcecred: reaction would have materially lower weight than the Discourse :heart: reaction – maybe 1/3 to 1/5th as much cred – so there would still be an incentive to record significant contributions onto the Discourse.

Of the concerns you raised in the post, this is the one I can’t directly address. Having cred minting on the Discord may change the vibe. (Why didn’t anyone :sourcecred: my message? It was so valuable? etc.)

That said, this isn’t really an issue that’s specific to having a Discord plugin–it’s a tension at the very heart of SourceCred as a social system. By turning SourceCred off in our Discord, we’re avoiding a rich source of dogfooding. As the people building SourceCred, I think we have a responsibility to lean fully into the paradigm, even in instances where it may be uncomfortable.

Suppose that having a Discord plugin really does create social problems for us, or harm our community. If so, it is right and proper that we discover this first–and try to understand how to improve it, or what boundaries should be set. If we don’t do this ourselves, but still create a plugin system that invites a Discord plugin, then we are inviting others to follow where we don’t lead.

I drafted the following reply earlier, but opted not to post it because I wasn’t sure how much energy you wanted to spend on the topic and wanted to be respectful of your time. But clearly you judge it valuable enough to merit a follow-up thread, so here it is. (Posting here rather than there because it addresses both the “cred-free” part and matters related more closely to the initiative itself.)


Thanks for the thoughtful response. Some responses and thoughts inline, but let me first say that I’d like to separate two things: (a) the degree to which we think that magically having cred on Discord would be a good thing, and (b) the degree to which we want to spend effort implementing it in the near future, as signaled by this initiative being marked “up for adoption” and help “much appreciated”. These are correlated but different. While we may have different perspectives on the magnitude of (a), I think that we’d agree that in the long term it is something that we want to have, and would be a strange gap if missing.

This is nice to have, but, as you hint, this doesn’t really solve the problem. “Ugh; they liked my comment enough to :+1: it but didn’t :sourcecred: it” currently does not pass through my mind on Discord at all.

Hmm… my intuition/strawman here—what do you think about it?—is that helpful and informative Discord posts usually create value by influencing the trajectory of other things that are more documented. If my comment about floating point numbers was useful to someone because they’re porting some JavaScript code to Python, the relevant PR description could include a “thanks” edge. If a comment shapes the development of an initiative, the “References” section of that initiative could include a “thanks” edge. We encourage these kinds of attribution, anyway.

It doesn’t require self-promotion—anyone can add the “thanks” edge, and I was actually imagining that most thanks would be added by the thanker rather than the thankee.

It is true that it raises the activation energy (and thus increases the signal-to-noise ratio, as it’ll be used less for “reflex” reactions). But contrariwise it also decreases the activation energy for actually posting on Discord.

As above, this doesn’t follow, because anyone can thank. It is true that if there were a clique of people who were only contributing on Discord and wanted to flow cred to each other, but nobody wanted to flow cred to them, then they would have a harder time doing that.

My two cents as the author of that message: I’m totally fine with not receiving cred for that post. I didn’t expect any while writing it. I posit that it is a feature to not create a culture of expecting remuneration for the simplest acts of helping each other.

This doesn’t sit quite right with me, and though I can’t pin it down exactly, let me try to share my point of view.

First, the nature of a plugin system is that we are inviting others to follow where we don’t lead. That’s explicitly the purpose. If it weren’t, then we wouldn’t need plugins; we would just bake in all the functionality to core ourselves.

Second, if we think that something is likely to create problems, that surely doesn’t mean that we must do it specifically to face those problems ourselves and break them down scientifically. It does mean that we should be transparent about known limitations of anything that we provide to others. But we’re well within our rights to say that we think that something is a bad idea and are not going to do it.

A critical part of “the paradigm” is recognizing where and how the paradigm should be applied. We need to lean into that, too.

Maybe I didn’t articulate them clearly enough, but other concerns that have not been addressed are that Discord is not an open platform (more closed than GitHub, as the entire content space is login-walled), and that implementing this would require a significant amount of work.

I’m ok with the idea that Discord comments receive cred only when they get an explicit “thanks” interaction, assuming one of the following:

  1. We make this extremely low friction, e.g. by making the “add thanks” interaction correspond to adding a reaction in Discord :wink:
  2. We apply this “need an out-of-band thanks edge to receive cred” consistently across the project, i.e. GitHub issue comments or reviews don’t receive cred except through the same process.

Remember that not everyone in this project has the same amount of privilege. Right now, GitHub contributors have a lot of privilege–they have high weights in the CredSperiment, and their contributions automatically mint cred. In contrast, contributors on Discord experience very little privilege, as their contributions quickly evaporate, and we don’t yet award any cred.

The proposal that Discord contributors will receive cred only by doing extra work, while other contributors recieve it automatically, means we are putting extra burdens on the people with least privilege – and that would not be aligned with our values.

Sure. In this case, we lean into that by (experimentally) extending into the uncomfortable areas of the paradigm. If we get burned, then we can dissuade others with confidence.

I don’t see any reason that plugins should only be written for fully open platforms. As a community, we could have chosen to use a different (more open) platform, e.g. Keybase or Matrix, but Discord had the features that we need and so we’re using it. If it’s good enough for us to depend on as critical community infrastructure, it’s good enough for us to write a plugin for.

It’s a significant amount of work, but also a different kind of work. A lot of the work in the project requires “trailblazing”, which tends to bottleneck on context transfers from 1-3 people who really know the system. IMO, it’s those context transfers that are the real development bottleneck. Since we have several good examples on how to write a plugin, it’s not trailblazing, and can be done in parallel with other work streams.

As part of CredCon, implemented a prototype in https://github.com/sourcecred/sourcecred/tree/discord Also available as a sourcecred/sourcecred:wip-discord docker image, to run it with GH actions.

Plus an example of our own instance: https://beanow.github.io/sce/discord-plugin/timeline/@sourcecred-discord/ (be sure to look at Discord - Members filter) This includes a boost of 2x on the #didathing and #props channels. And a 4x weight on mentions in the #props channel.

There’s a good few things already implemented, but needs more work to be production ready. Would love to go over the prototype with @brianlitwin and look at next steps :]

1 Like