Questions for Community feedback

Hello folks, Were putting together a series of questions the experience of joining and participating in the SC community, particularly the challenges around communication, community guidance and availability of body of knowledge resources. The purpose of this set of questions is to get data sets that would inform work that is being done around onboarding in the community cultivation channel. For more info on the work being done aroudn this check out this link (https://roamresearch.com/#/app/SourceCred/page/t_rZiNZBc) Please comment, discuss and contribute to this set of questions so that we have a well formulated survey that would yield relevant data. Here goes:

QUESTIONS FOR COMMUNITY

  • User profile
    • What is your area of interest in SC?
    • Area of Expertise - Dev, Design, UI/UX, content, Generalist (develop further)
    • How long have you been involved?
    • Degree of engagement - Wall flower, participate in discussion, contribute,
      • How often (1 is daily, 10 is monthly)
  • Onboarding process
    • How would you rate the current onboarding process (1 is terrible, 5 is awesome)
    • what is the biggest challenge you faced before you got comfortable contributing/interacting with the community
    • How long did it take you to :
      • Start participating in weekly calls or discussion?
      • Start contributing to one or more projects areas in SC?
    • What would have made your life easier during onboarding? What type of resource or assistance did you miss?
  • Body of knowledge/Content
    • How would you rate the current organisation of body of knowledge (1 is terrible, 5 is awesome)
    • What challenges have you faced while looking for required content/documentation?
    • What part of the body of knowledge had the largest value addition to you? What was your favorite segment of the over all body of knowledge?
    • How would you like the organisation of content to be different?
  • Communication channels
    • How intuitive has organisation of communication channels been? (1 is terrible, 5 is awesome)
    • What communication channel do you find yourself spending the most time in? Top three?
  • Community guidance
    • How would you rate the current quality of community assistance (1 is terrible, 5 is awesome)?
    • Was it easy to find who was in-charge of different projects?
    • Was community guidance/assistance easily available? Was it helpful?
    • How would you suggest that the community improves engagement?

@louis @LB, @Bex, @benoxmo

3 Likes

Ravi, this is phenomenal! Thanks for taking the time to formulate these questions and champion the survey, this is a great place to build from.

Here’s some of the things I’ve learned from some research about formulating surveys:

  1. Identify the answers we’re looking for and develop questions out of those. This helps make sure the data and questions will be relevant.
  2. Be simple, direct, and specific. Use language that has as little room for interpretation as possible, and is clearly and widely understood.
  3. Ask only one thing at a time. Break up more complex questions into multiple smaller, more direct questions.
  4. Keep questions as neutral as possible to avoid “leading questions”.
  5. Use interval questions to get nuanced data (1-5 scale).
  6. Use more “extreme” language to get more potent answers, like “most”, “worst”, “biggest”, etc.

So what is the data we’re trying to get? We want to know whether people: know what’s expected of them, have the resources they need to engage, feel supported by and engaged with the community, have the opportunity to do what excites them, feel like they belong here. (Loosely tied to the mountain climbing onboarding philosophy.)

Here is my reworking of the survey based on your original questions and the things I learned from my research, mixed with my current working onboarding philosophy:



Please answer these questions honestly in regards to your experience. The feedback will help us to improve our community operations. Thanks!


User Profile

  1. What is your expertise as a contributor?
  2. What part of SourceCred do you find the most engaging?
  3. When did you get involved in SourceCred? (less than a month ago, within the last 6 months, within the last year, more than a year ago.)
  4. How often do you engage with the project right now? (1 basically never, 2 rarely, 3 check in from time to time, 4 frequently around, 5 can’t stay away)
  5. In what ways do you engage with SourceCred? Choose all that apply: wallflower, text conversations in Discord, attending calls, posting thoughts in Discourse, working on a project/initiative, leading a team/championing an initiative, piece-meal work on my own, other _____.
  6. How connected do you feel to the vision of SourceCred? (1 not at all, 2 I’ve got other priorities here, 3 it resonates, 4 it really excites me, 5 it’s what I’m ABOUT)

Expectations

  1. I have at least a basic understanding of SourceCred. (Agree/Disagree 1-5)
  2. I have useful skills I want to contribute. (Agree/Disagree 1-5)
  3. I know how to start contributing. (Agree/Disagree 1-5)
  4. It was easy to start contributing to SourceCred. (Agree/Disagree 1-5)
  5. When I contribute, I know what’s expected of me. (Agree/Disagree 1-5)
  6. What is the most frustrating part about trying to understand the scope of SC?
  7. What is the biggest barrier you faced before you could start contributing?
  8. What do you need the most to be able to contribute effectively?

Resources

  1. I know what resources/tools are available to me as I learn about/interact with SourceCred. (Agree/Disagree 1-5)
  2. I can easily find the documentation I need. (Agree/Disagree 1-5)
  3. The documentation is useful and relevant. (Agree/Disagree 1-5)
  4. It’s easy to share ideas and have conversation in the Discord. (Agree/Disagree 1-5)
  5. It’s easy to get answers to my questions when I ask them. (Agree/Disagree 1-5)
  6. I know who leads what in SourceCred. (Agree/Disagree 1-5)
  7. I have the tools I need to work effectively in SourceCred. (Agree/Disagree 1-5)
  8. I know how much cred/grain I have. (Agree/Disagree 1-5)
  9. What tool, support, or resource would make the biggest impact in your ability to join or start a project in SourceCred?

Connection

  1. I feel comfortable in the SourceCred community. (Agree/Disagree 1-5)
  2. Someone in SourceCred cares about my experience. (Agree/Disagree 1-5)
  3. People in SourceCred are willing to help me. (Agree/Disagree 1-5)
  4. Others in SourceCred recognize the value I’m bringing to the project. (Agree/Disagree 1-5)
  5. I can use my skills to make SourceCred better. (Agree/Disagree 1-5)
  6. How did the experience of onboarding into SourceCred make you feel?
  7. What experience and/or resource would have improved your onboarding the most?
  8. How do you feel when you spend time with the SourceCred community?
  9. What would (or what does) make you feel like you belong in the SourceCred community?


Open to any suggestions around how to rephrase questions to make them more potent, how to condense this list more, or any other feedback!

3 Likes

Excellent… I love the additions. @louis and I did have the concern that there were too many questions, but the 1-5 rating reduces the cognitive burden

1 Like

That’s awesome. And idk about you guys but I know I would start being lazy to answer more than 10 questions at a time. So maybe we could start with a set of 10-15 questions on a specific topic, then share another one a week later or so.

One way to split it could be first asking about expectations and resources for contribution. i.e. If people:

know what’s expected of them, have the resources they need to engage

Questionnaire structure would be:

  1. User profile Qs
  2. Expectations Qs
  3. Resources Qs

Second about community assistance, belonging, connection, vision…

Questionnaire structure would be:

  1. User profile Qs (tailored to this survey)
  2. Connection Qs
  3. Vision Qs?
2 Likes

:+1:

Ideally, we would send the second questionnaire with the results (or some comments) from the first one.

Regarding responses, It could be helpful if we make a light one-to-one reminder :stuck_out_tongue: at some point. Good opportunity to make connections as well!

Alright, so how about this for the first questionnaire, oriented towards onboarding and connection with the community.

SourceCred Community Onboarding Survey

User Profile

  1. When did you get involved in SourceCred? (less than a month ago, within the last 6 months, within the last year, more than a year ago.)
  2. How often do you engage with the project right now? (1 basically never, 2 rarely, 3 check in from time to time, 4 frequently around, 5 can’t stay away)
  3. Is the vision of the project clear to you (1-5 scale)
  4. What part of SourceCred do you find the most engaging?

Onboarding

  1. How did the experience of onboarding into SourceCred make you feel?
  2. What resource helped you the most: documentation, chats, people?
  3. What experience and/or resource would have improved your onboarding the most?

Connection

  1. How connected do you feel to the vision of SourceCred? (1 not at all, 2 I’ve got other priorities here, 3 it resonates, 4 it really excites me, 5 it’s what I’m ABOUT)
  2. Someone in SourceCred cares about my experience. (Agree/Disagree 1-5)
  3. People in SourceCred are willing to help me. (Agree/Disagree 1-5)
  4. Others in SourceCred recognize the value I’m bringing to the project. (Agree/Disagree 1-5)
  5. How do you feel when you spend time with the SourceCred community?

If that makes sense I could go and put that into a Typeform.

The second questionnaire would be oriented towards expectations and contributions. We can use the questions above as well as the feedback we get from the first survey.

Cheers!

edit: does anyone have a paid subscription to Typeform?

for consistency sakes lets do “1 - less than a month, 2 - within the last 3 months, 3 - within the last 6 months, 4 - within the last year, 5 - more than a year ago”

3 Likes

@rpp63 @louis @LB Wow, watching the progression of these draft iterations is very inspiring. The gain in clarity and concision across each version is really something, great job yall! A couple of minor thoughts: 1) how long will it take someone to digest and answer these questions in total? My guess is that at more than 2 minutes total, you’ll see a sharp drop off in responses. I think the length is ok where it is now, but thinking in terms of “time-cost” is something I though I’d just mention. 2) One glaring conceptual hole seems to be this: stated simple, SourceCred automates compensation. None of these questions address that… There is no question along the lines of: “are you interested in earning grain”, “how much grain do you expect to earn”, “would you put in more time if you earned more grain”, etc. Now, maybe this is intentional, the operating assumption in all this seems to be that you are targeting “casual” community members who are motivated by social participation more so than earning grain. Maybe this is good, I just wanted to make the assumption explicit.

2 Likes

@DeltaFreq’s comment is insightful and resonates with the observer/user discussion between @decentralion & @LB.

I agree with (well coined btw)

and in the long run “earning cred/grain” will take more place. My suggestion for the questionnaire :

  • Replace Question 4

with “Earning grain in the next 2 years is: (not important / few bucks / enough to reinvest or play with / to approach a kind of wage/salary)”

  • keep the questionnaire as it is for those who responded in question 4 (not important or few bucks) but make a second track with light changes in the questions for those who responded (enough to reinvest… / wage )

  • the second track would have have the replacement of the questions 9 & 10 from track1 (inputs welcome from @DeltaFreq or @s_ben with the maker trial or some hypotheses we want to test)

9. Regarding the payout proposition, you will work if the slow/fast ratio is : fast only OR 25(slow)/75(fast) OR 50/50 OR 75(slow)/25(fast)

10. Regarding metrics, what are the most important ones (or the rationale) for you to make a contribution?

1 Like

Taht’s a great point haha. Asking questions about the product seems important. Would this be worth a standalone questionnaire?

This one could be run in co-communities like MetaGame as well

1 Like

@DeltaFreq I totally agree that product/purpose related questions are key. I wonder if they deserve their own standalone survey. Maybe we can add a question in the first, onboarding related, survey about purpose for joining community. That might give us an idea about high level motivations of the community, while a more detailed product related survey could dig in a little more. I must admit, it didn’t even occur to me :smile:

What if we have some grain attached to completing a survey? That could allow us to expend the set of questions to give us actual metrics. I do think we need to have a solid handle of what we hope to get from this survey, and then validate the questions one more time. @LB @louis @benoxmo @DeltaFreq

2 Likes

That could be a great motivation indeed. One way to do that is to have respondents signal that hey have answered the questionnaire in the Didathing channel

Edit: this would work with Cred, not grain. My bad!

Edit2: with Ravi we were thinking that maybe people could leave an Eth address at the end of the questionnaire and receive some grain

1 Like

Action steps from our Call with Ravi:

  • Wait for feedback around using grain to reward the respondents
  • Put questions in a Typeform
  • give the questionnaire to 1-2 test community members to get feeback