|
Venia is a regular organizer and panelist CHAOSScast, where we discuss community health, measurement, and more. As a part of this continuing partnership we'll regularly be cross-posting episodes in the CHAOSS community here for you to listen to!
If you'd like to see them all, head over to CHAOSS.community or to your favorite podcast app! For this episode "vanity metrics" have a bad wrap but we're defending when to use them, and when to leave them aside. avoid "vanity metrics with the QIA process
Modern marketing has the opposite problem of marketing in 2010. Where numbers used to be a vital and scantily accessed feature of business, now we have a world of big data where every metric could matter and it's hard to think through what they mean in tandem.
In today's world it's actually possible to be "metrics obsessed." "Vanity" metrics are metrics that tell you a line is going up and to the right or that you're doing well, but they don't really create any form of "action" on your end. They don't really tell you how you're doing, whether you should speed up, slow down, or pull over. Knowing which metrics matter for your marketing story isn't necessarily about finding the right metric to aid your progress. Instead, it's often about setting a line in the sand or throwing a dart at the board. It's about telling a story as you build your product, and figuring out the best way to represent that story in your overall plan. I use Chris Mercer's QIA process to decide what metrics really matter. First you will have a question. This needs to be thought of as "how will I know what action to take". Then you'll have an answer your want and several answers you don't want. As an example let's consider SociallyConstructed.Online's Social Currency Metrics System informational page. I want to know if it's answering the question, "do people get the SCMS?" Re-framing the question a few times provides more information as well. If people "get" it does that mean the landing page is converting or that people understand and move on? What behaviors and opinions are occurring that tell me this? Going trough this thought experiment I know what my potential answers might be. And because I understand the answers and how I might answer the question, our middle "information" column fills itself in. this way I know what metrics to pick, and that the "pageviews" metric for the SCMS page is probably more of a vanity metric - I don't care who hit the page, I care who read it.
Want to know more about the QIA process and Mercer's approach to analytics? Read our next blog where we interview him about the theory and future of metrics!
0 Comments
Venia is a regular organizer and panelist CHAOSScast, where we discuss community health and measurement . As a part of this continuing partnership we'll regularly be cross-posting episodes in the CHAOSS community here for you to listen to!
If you'd like to see them all, head over to CHAOSS.community or to your favorite podcast app! For this episode Ruth Cheesley of Mautic, an opensource marketing platform let us in on how she is measuring community health and growth for a non-coding audience. Mautic is a big community Ruth suggested Mautic make community metrics public.
Ruth's biggest issue with Mautic at Aquia has been an issue of identifying technical debt across several demographics throughout the large community.
One one end she has developers who must build solutions for marketers and on the other end marketers who will never be coding contributors. Her biggest focus was making sure that parts of the community who thought the project was dead or that changes were not being implemented, saw the parts of the community that was truly trying to resolve those issues. Her answer was a publicly available community dashboard so the community could directly interact with the data. She did this by using GrimoireLab's community health dashboards and CHAOSS metrics which conveniently include our own Social Currency Metrics System! In the podcast we are discussing her case study using the system to re-engage and combine parts of the community with others. Post your opinion below: How do you feel about publicly releasing Metrics to your community members and to the general public? would you feel safe doing that with your community?
We've gotten involved in the CHAOSScast open source community podcast as a regular part of our operations here a SC.O. Venia is an organizer and panelist on the podcast where we discuss community health, measurement, and more. Dylan edits and publishes some podcasts.
We'll regularly be cross-posting episodes in the CHAOSS community here for you to listen to. If you'd like to see them all, head over to CHAOSS.community or to your favorite podcast app! For this episode highlight we're also truly excited say we interviewed one of the most well-known community managers and strategists in the space - Jono Bacon
This is one of the best moments of my career really (Venia).
When i started my nonprofit RESCQU.NET and fell in love with Community Management, my partner was working for an open source company called Canonical. This was the same Canonical that Jono began his community management career in and over time I got the opportunity to speak with him about community. Over the course of my academic and industry careers I continued to stay in touch, speak with him, get advice, read his books the art of community and people powered, and now in this podcast I'm talking with him about the Customer Value Journey, Community Onboarding, what it means to be metrics-obsessed, and starting your career in community management. I am truly blessed to have done this podcast with Nicole and Brian. Thank you both. Jono bacon is an important inspiration for us. Here are two videos where he answers our questions directly!
We believe strongly in being a part of and improving already existing communities.
As a part of this we've have gotten involved in the CHAOSScast open source community podcast. Venia is an organizer and panelist. We'll regularly be cross-posting episodes in the CHAOSS community here for you to listen to. If you'd like to see them all, head over to CHAOSS.community or to your favorite podcast app! This week's podcast is episode 5: Defining Open Source Community Health
This is probably the most important podcast you could listen to for CHAOSS.
This podcast is integral to their mission of providing tools and strategies for defining what it means for a community or open source project to be healthy. It's an unexpectedly hard question to answer and for CHAOSS and SociallyConstructed.Online this is what we live for. We believe that your presence in a community should not hinder its growth and if it is, it's your job to ensure that doesn't happen. Over here the SCMS has been our answer to enhanced social listening and basing community health on the very voices, opinions, and feelings of the community members. But it's not just about what people are saying. It's also about what they are doing. Tracking your community through tried-and-true analytics provides a way for you to measure health from week to week and forecast where you need to focus on it next week. CHAOSS provides that other part of the equation by measuring it with a series of well conceived, written, and implemented metrics you can look through here. Applying some of these metrics to your community may shine a light on what your community is doing well, where it could improve, and what your next step will be to foster community growth. Question: How are you measuring community health? Do you report it to users? Submit your answers here or on our twitter feed@The_SCMS! Your answers could help CHAOSS and us make powerful new metrics.
SociallyConstructed.Online believes strongly in being a part of and improving already existing communities.
As a part of this Venia and Dylan have gotten involved in the CHAOSScastopen source community podcast and Venia is an organizer and panelist for episodes! Starting this week, we'll be releasing our episodes in the CHAOSS community here for you to listen to. If you'd like to see them all, head over to CHAOSS.community or to your favorite podcast app! We already announced the first podcast we were involved in on the Social Currency Metrics System, so this week we're starting with CHAOSScast episode 4: What is GrimoireLab? Why does GrimoireLab Matter?
GrimoireLab is the software suite that we use to run an implementation of the Social Currency Metrics System that Ria Gupta built. As an open source software GrimoireLab is capable of porting in channels of data, compiling and analyzing it, and producing powerful dashboards similar to Google Data Studio (GDS).
The difference and advantage its got on GDS is that it's 100% open source. That means once you have access you can request or make changes to any part of the platform which creates a lot of freedom and power for your ability to measure community health! GrimoireLab is powerful and that's why we recommend it as your next step after you've used Data Studio to its fullest.
Want to hear more about GrimoireLab and the SCMS implementation?
Darrick: We are a part of a lot of different groups and organizations. What does a "Meaningful Community" mean or look like to you?
Venia: For me there is a progression in your community's relationship with your members - A "useful" community provides value to its members. A "successful" community accomplishes the goals and mission of that community by imparting "value" to its members. An "engaged" community encourages relationships to grow beyond the value a member initially desired and encourages them to give back. The #1 metric I use to determine how "meaningful" a community is, though, is self-disclosure. A member's willingness to disclose more about themselves shows a genuine affinity with other members. A meaningful community is exemplified in those moments when a person's relationship with others in the community transcends the community's purpose or the value-added engagement that keeps them interacting. A meaningful community creates a real feeling of affiliation that proves to a member that this is where they want to spend their time. The #1 metric I use to determine how "meaningful" a community is self-disclosure. 
Darrick: How would you describe your role when it comes to the communities you work with?
Venia: As an online community manager and full-stack marketer, I usually work with brand communities and generally oversee a somewhat tenuous relationship between the community and its leadership. Many communities are viewed similarly to a beehive. Brands and more powerful stakeholders in a community can approach their community like a bear getting the honey, or a flower feeding it nectar. It's my job to tell a brand or executive team when they are being the bear, when they're being the flower, and when that's okay. \ Sometimes my job is to "speak for the bees" in their executive meetings, so the community has a voice. At other times it's to report cold facts on whether the community is providing real value to the people paying for it. I have to navigate my positions of authority and subservience to both the brand and community. Because of this tenuous relationship, my job primarily involves navigating misconceptions about how the community really works, using analytics and social science, and that's why my co-founder and I created SociallyConstructed.Online.
Darrick: What gives you joy when working with communities?
Venia: It's been my Raison D'etre (pretentious but true), to enter any community and know that I've left it better from my presence; to know I've improved people's lives somehow. I know it may be weird to think, but analytics has a special place in that. I love the social-scientific aspect involved in being a community manager. I really like the notion of measuring a community's health, reporting it to those in charge, and seeing those people implement one thing that will change that community for the better. Sometimes my job is to "speak for the bees" in executive meetings, so the community has a voice. Other times it's to report cold facts on whether the community is providing real value to the people paying for it.
Darrick: What are your biggest struggles when working with communities?
People like to think that because they spend every waking moment in their community, they know its pain points and how to improve it, but more often than not, once you get above a "tribe" or around ~250 people, they're wrong. I would say many people in their communities use the lived experience of their time in a community to make decisions rather than the learned experience of their community members. They see problems and advocate for them without determining the nature of that problem from other people's points of view. And this makes sense. As a community member you've become so fond of something, and you've gained enough clout to be considered a veteran or expert. It puts the blinders on so-to-speak. This further underscores the importance of processes, though. As community leaders, we have a responsibility to gauge and report out the metrics on our community's health - not just because everyone deserves to know, but because it's a social contract that keeps our decisions in check and makes it easier to figure out when we're the bear.
Darrick: Any last words or thoughts when it comes to meaningful communities?
Venia: Well that sounded a bit ominous. I guess I would say community is a social construct. Online spaces, especially communities, are built out of the very communication they facilitate. That means putting together a community charter of transparent practices and measuring your community success. You don't need to develop infrastructure for the future, but you need to have the infrastructure necessary to measure what's happening in your community today. You need to learn to listen. That's (not) all I wrote!
After this interview I also spent a good period of time reflecting on what exactly Darrick meant by "meaningful" in his conversation so I reflected a little more on my first question's response. I think a lot came out of it. Click below to read it and join me in the community catalysts group if you want to reflect more on this :)
This past Friday I was given the opportunity to do a small guest post for a local community here in Fort Collins, called "Community Catalysts."
The network is run by this amazing community expert and teacher - Darrick Hildman. I've placed my full interview in the image below, but he asked me a question I think is worth a good amount of introspection and exploration for your own community... "What does a 'meaningful community' look like to you?"
In my response I started with some silly response about how important it is that we be focused on the mission and values of our community, but before I knew it, I was second guessing what he meant by "meaningful".
The only way I could answer the question was by comparing the communities I've built to my family and friends. The people in my life that mean something. And my answer surprised even me: There is a progression in your community's relationship with your members. A "useful" community provides value to its members. A "successful" community accomplishes the goals and mission of that community, by imparting value to its members. An "engaged" community encourages relationships to grow beyond the value a member initially desires and encourages them to give back. But the #1 metric I use to determine how "meaningful" a community is... Self-disclosure. A member's willingness to disclose more about themselves shows a genuine affinity with other members. A meaningful community is exemplified in those moments when a person's relationship with others in the community transcends the community's purpose or the value-added engagement that keeps them interacting. A meaningful community creates a real feeling of affiliation that proves to a member that this is where they want to spend their time. Creating a space for self-disclosure is hard
Creating a plan that fosters a meaningful community is just as important as making one that performs according to the goals of the brand. It requires a plan and a good amount of thought.
Clearly, I hadn't thought about it enough. So, I thought I'd pay it forward to you - what tactics are you using to create "meaning" in the hearts and minds of your community members? How are you making the community better?
And while you're thinking about it, I invite you to read the rest of my post, join us in the community catalyst group and check out Darrick's new consultancy!
This week's collaboration meeting on the 26th was, prolific.
In this meeting we discussed the trends observed by tagging the data and discussed the future path of the SCMS's development. As expected, when we manually tagged data, out of all 5 tags, the most observed tag was “Utility”, because GrimoireLab provides very useful tools for software development analytics. Further, we discussed the noise elements observed in the data and what to do with them. IRC messages rendered some noise, and those records could not be tagged. Since we can't leave these records untagged, we have planned to remove such instances from the google sheets implementation, so that we could focus on meaningful texts. Some examples of noise "text" include: “abc_is now known as xyz” or “ChanServ sets mode: +o collabot`”
We also decided to utilize the SCMS' “Category” which is a way to classify our records for more fruitful analysis by breaking down information into specific keyword sets.
For instance, a comment in which a user is asking for help in any issue might be indicative of ‘troubleshooting’. For GrimoireLab, since the majority of GitHub comments were related to troubleshooting, we decided to split this category into 2 categories namely “Incoming Request” and “Technical Support”. This was because having categories as precise as possible would help in analyzing data more efficiently. Categories like “Interpersonal”, “Operational” and “Transactional” were also put down to be added later. Basic Visualisations
Apart from tagging data, I also spent the week learning the process of making a visualisation.
I made some of the basic visualizations using the index which contained randomly tagged data. Visualisations I've made so far included a pie chart of 5 different tags, a bar chart of the number of comments received per week, data indicating the number of conversations from different channels and I"ll produce more in the comming weeks. Thanks for reading! This week marks the end of the first coding period, It has been a wonderful month coding, brainstorming, blogging, interacting with the so-awesome mentors! ? That’s it for this week.? Make sure you have a look at the project updates on Github #ria18405/GSoC. All questions and comments are welcomed! Stay tuned for more weekly updates. ?
Airtable's limitations
In the 2nd week, I had exported the extracted data to an Airtable view.
I didn’t realise it until just last week but Airtable has a limit of 1200 records per base only for a free version and the SCMS could be implemented in any spreadsheet software as far as GrimoireLab is concerned. Our initial goal was to collect as much data as possible to represent the community’s sentiments holistically and make SCMS usable by other open-source communities as well. So we decided to shift the implementation to Google Sheets. Google Sheets has a limit at 5 million cells, which is adequately enormous [yeah....0.o]. So, I collected all data from Github, Mailing lists of Grimoirelab and IRC channel of CHAOSS. Then, randomly tagged it (as done in Week 2), and exported to Google Sheets using an API. After looking at the data carefully, we also noticed that we had Github comments by Coveralls indicating the coverage increased or decreased. Other than this, we had lots of IRC messages indicating a person has joined the channel or has left the channel. This data is not providing us with any additional information about the community and is not a user sentiment. So, we planned to remove all those unnecessary records. We collected around 5.6k filtered data from Github, Mailing list, IRC channel. 
The Codex
A major part of the week was planned to be devoted to building a Codex Sheet.
Codices help us to rely on Qualitative data in an objective sense over time much like Quantitative data. To reduce the subjectivity of this qualitative data, it is imperative to define a codex table which can help in tagging data points accurately by better defining and honing in on the purpose of tags. It also helps in collaborating with the team to keep similar ideas running. Codices contain the definitions of metrics within the organisation (here it is Grimoirelab), and an example to illustrate the definition. It also consists of “when to use” and “when-not-to-use” a metric. 
I also made a short rough draft of an overview of SCMS to be published for the CHAOSS blog!
Other than this, since we have a full version of data present in google sheets, I converted this entire Excel sheet data to the ElasticSearch index (precisely as done in Week 2). The only difference is the number of records being used. Earlier, I could only have limited records containing the “extra_scms_data” in the Enriched index. Now, every meaningful record (i.e. ignoring comments by coveralls) has an additional field present in its ElasticSearch index. And the weekly meeting...
Per usual, I had a weekly meeting with the mentors on 19 June’20, minutes are here.
We discussed Dashboarding and expanding on the SCMS. We’ll be having a collaboration meeting on Friday,26 for discussing the findings of tagged data. Till then, in the next week, I’ll be focusing on writing tests and making a basic dashboard. ? That's it for this week.? Make sure you have a look at the project updates on Github #ria18405/GSoC. All questions and comments are welcomed! Stay tuned for more weekly updates. ?
This week was oriented according to the timeline.
We had planned first to make the pipeline ready, and then to move forward with the rest of the procedures. In the first week of the coding period, I had extracted the required data from ElasticSearch and converted it to the default SCMS implementation's Airtable view. This week, I have randomly tagged the datasheet with five metrics and have linked the additional data back to ElasticSearch via a ‘study’ called ‘enrich_extra_data’. To explain what this exactly means and looks like, I’ll explain in detail the steps involved. The first step was to randomly tag the dataset by all possible combinations of the social currency parameters; Transparency, Utility, Consistency, Merit and Trust - in an Excel Sheet. Here, we have added another column in the excel sheet by the name of ‘scms_tags. 
The second step is making a python script Excel2Json which could convert Excel to a specific type of JSON which will be used as an input to the study. You can find the JSON here.
Now, in step 3, we need to execute a study called ‘enrich_extra_data’ in grimoirelab-ELK.
Edit Setup.cfg according to the study, input the URL of JSON made above. After successfully performing the study, we can see that ElasticSearch indexes have the extra parameter of scms_tags in the dump. ? The study appends an ‘extra’ to the field name, So the field name is ‘extra_scms_tags’. 
The importance of the SCMS' Codex
After building out the data set and meeting with mentors on June 12th, I understood the importance of codex sheet in training the tag set up. The minutes of the meeting are here.
Defining the codex table helps to increase universality and decrease the subjectivity of the data. It also allows us to rely more on qualitative data rather than quantitative data. After the training, we discussed the path for the next week. In the next week, I’ll be making a codex table which will contain the definitions of each trend observed and will contain the ‘when to-use’ and ‘when not-to-use’ cases. Additionally, I should note that we found the limit in the number of records in Airtable, so we planned to shift the implementation to Google Sheets. In future blogs we'll be using google. This week went off well, looking forwards to the next week! ? Make sure you have a look at the project updates on Github #ria18405/GSoC. All questions and comments are welcomed! Stay tuned for more weekly updates. ?
Putting Code to Pixel; My 1st week coding
I noticed that throughout my coding history, I had always undervalued the process of facing bugs, Now, I honestly feel that bugs make the entire coding process fun! After each set of debugging exercise, you tend to understand the main tasks in a better way, plus you get a sense of achievement, however small that might be. ?
So this past monday I put on my enthusiastic boots and set off to work. I first understood which data attributes need to be extracted to maintain consistency in the output data. After this, I made a new enricher for extracting Github data for the Social Currency Metrics System. I have collected the comments made under a Github issue and under a Pull request. We decided to keep the context for each GitHub comment, to bring more clarity and reduce idea redundancy to the table. So, I retrieved the ‘Title’ of a Pull request or ‘Title’ of an issue as the ‘context’. This is because the Title of the issue/PR conveys the same message as in the following comment. Similar steps were done while extracting mails from the mailing lists. The ‘subject’ of the mail was set as ‘context’ in this case. Now, after performing an ElasticDump operation, I wrote a script called “ES2Excel” which converts data from Elastic Search indexes into a CSV file, further into an Excel format, and then further extending to an Airtable view. Now, the data obtained by mails (MBox) and comments (Github) needed to be collected together in one excel sheet. So, for this, we perform “Aliasing” on ES indexes. We have 2 enriched indexes which we need to alias as a third index which can be used for creating CSV and Excel files. The CSV sheet is then converted to an Airtable view using Airtable API. We execute ES2Excel script mentioned above on the aliased index. This Airtable view is ready for performing all tagging procedures to it. The output can be seen below. 
Records of Github comments and mails in an Airtable view
Zooming a mail sent to Grimoirelabs Mailing list
Where we are and where we're going
during our meeting on 5 June 20th I was given training about the various backround social theories that made the SCMS inlucing the "Grounded Theory Analysis" method.
It was fun to relate those theories with the Social Currency Metric System. Dylan and Venia also emphasized understanding the process of Community interactions and further bringing it to a codable form. My mentors suggested that we must have a third data source other than Github and mailing lists, to avoid biased data, so we’ll also be looking towards the addition of Twitter or IRC data in the next week. For the most part next week, I’ll be focusing on converting the randomly tagged text data to ElasticSearch with the help of ‘Study’. The work done this week was nicely aligned with the timeline. Looking forward to more learning sessions! ?
Preparing for next week...
I had a meeting with my mentors on 22 May 2020 where we discussed the implementation of the Social Currency Metric System, how is the codex table created, and used.
I had done a pilot study to understand the pros and cons of using existing enriched indexes over creating new ad-hoc enriched index for SCMS. The results of this study favoured the creation of new enriched indexes. This week, I had tried to do a very interesting pre-coding period task to extract comics data from Marvel using grimoirelab tools like Perceval. This meant the creation of a new backend for marvel. [the single best sentence we've ever heard at SC.O ~ Venia] The repository can be found here. After cloning the Perceval directory, and executing perceval marvel, it will yield all comics data from perceval. Integration with ELK is left and will be continued during the coding periods at a secondary priority. After this, I had the last pre-coding period meeting with my mentors on 29 May 2020. We focused more on the implementation plan and timeline found here. We also had a detailed discussion about applying ‘keyword analysis’ to tag data on the basis of Social currency. We analysed the setbacks of using tags, one majorly being differentiating negative sentiments with positive ones. Imagine something like “I find this product to be very useful.” and “I did not find this product to be of any use.” Both these statements will have to be categorized under the parameter “Utility”, but separating these contrasting sentiments will help in creating the SCMS a more meaningful system. Finally, the time has come, which I had been looking forward to so long, the Coding Period! I hope to bring out my best during this journey ? Yay! Looking forward! ❤️
We've spent the past month launching the SCMS and SociallyConstructed.Online.
It's been a lot of work, but that work has paid off, and we're excited to unveil some of the fruits of our behind-the-scenes work, much of which has been dedicated to promoting our upcoming partnership with the CHAOSS project. This blog is promoting the 1st of many collaborative efforts between SociallyConstructed.Online and CHAOSS (Community Health Analytics Open Source Software). We are incredibly thankful for the opportunity to sit down and record a full hour-long podcast discussing the Social Currency Metrics System on CHAOSS Community's podcast, CHAOSScast. If you're interested in how SociallyConstructed.Online emerged, why we do what we do, and what we have in store in the future, give it a listen here and consider subscribing to the podcast! Venia is officially a regular on the show and we plan to have a long-standing relationship with CHAOSS moving forward. What does this mean for you?
A More Formal conversation with CHAOSS
Prior to this podcast we had a more internal presentation with the CHAOSS community after last year's Open Source Summit in San Diego. This provides a more structured, and more visual approach the Social Currency Metrics System, and CHAOSS.
Here is Week 2: Launching the SCMS in Airtable!
This blog has been cross-posted from Ria's blog with her permission... The social bonding period continues
I had a meeting with all of my four mentors on Friday, 15 May’20. The details are here.
We largely discussed the past 2 weeks' progress and understanding of SCMS (Social Currency Metric System) in more detail. It was similar to a training to understand the importance of qualitative data over quantitative. The main agenda of the training was “Why qualitative data is rejected in business, and how reframing its collection using the SCMS makes it useful in businesses?” It was a very informative presentation delivered by Samantha and Dylan.
Implementing SCMS on Twitter data of Amazon
For the next meeting, we’ll be discussing the concrete implementation of the project both technically and theoretically. For this, I have some implementation ideas written in my Project Proposal, and I had a meeting with Valerio to discuss the pros and cons of different approaches. One approach is to use the already present enriched index, and the other is to create new ad-hoc indexes. 
Output data extraction with specific parameters of CHAOSS mailbox
What I did this past week
The plan for next week
Next week plan is to advance the implementation of elk and include github issue and comments. Along with this, try to implement a method in which we can break customer reviews into 2/more sentiments without bringing incoherence or context break. This will involve checking NLTKs implementation and understanding MaxQDAs approach to such situations.
Don't miss out on more of Ria's Journey over the next several weeks! We have a series of these blogs throughout the process and you can start with the previous one here: 
Hey all, Dylan here!
This was supposed to come out last week, but the world's gone a bit crazy since then, so we decided to push it back to this week because #BlackLivesMatter more then this blog. Venia made a wonderful introductory video about installing the Social Currency Metric System, complete with detailed step-by-step instructions! Isn’t she great?! I fully encourage you to take 3 1/2 minutes to watch the video below, but I wanted to write a more in-depth guide for implementing the SCMS to be used as a reference you can refer to if you’re confused about what to do next. Quick Note: The protests in combination with the still dangerous pandemic are hitting us hard; physically, mentally and emotionally. We are with communities of color at this time and appreciate your action to make a better physical and digital space for minorities. For this reason we thought it best to remain silent and provide time for our communities to recover and fight. #BlackLivesMatter #AlertNotAnxious
While we can build your SCMS implementation in the two-spreadsheet system we're providing here for free, we also wanted to provide a more comprehensive option that may be a bit more robust and useful.
My step-by-step guide will work off of the assumption you’re using this template as it offers a whole bunch of awesome functionality that will serve you well as you work through setting up the SCMS system for your own organization. 
What You Need To Get Started
The SCMS was built to be a system scalable for businesses of any size, so what you need to get started depends largely on the scale of qualitative data you’re looking to analyze.
We would recommend you have at least the following items:
That’s it. Just a single spreadsheet-based template, and data to interpret. Cool! Now, Larger businesses or those with extensive amounts of qualitative data can implement the SCMS on a simple spreadsheet like this, but more extensive analytics tools may be necessary. My personal recommendation would be to check out MaxQDA, but we also offer a free consultation if you want to look at your options given your SaaS stack. Step 1: Plan Out Your Data Sources
What do we mean by “planning out” your data sources?
We encourage you to aim for a wide variety of data coming from a number of different contexts. Sure, most of your data may come from your YouTube, Twitter and Facebook feeds, but is the data you’re getting from those three feeds alone diverse enough to get a full picture of your target audience's Journey? In qualitative analysis, higher quality data is more valuable than more data. You may get good insight from the three channels mentioned above, but if you only have the capacity to focus on three main sources of qualitative data, look into substituting YouTube for incoming support tickets or forum posts that may provide longer, more well-reasoned information that comes from a more defined audience. The feedback you get from several different kinds of sources is guaranteed to be very different from data gathered solely from social media, which can help your organization to get a better picture of the different conversations being had in different parts of your organization. Step 2: Create Your “Shell”
In the accompanying video, Venia walks you through how to set up your two-sheet spreadsheet. As awesome an introduction as that is, we recommend you just make a copy of the template we’ve developed. It has further information regarding implementing the SCMS, and is a wonderful resource for those new to this whole process.
This is also the step where you take the channels outlined above and begin to port the sentiment (the comments, threads, posts, etc.) in them into the “SCMS” tab of your SCMS implementation. You can do this manually, by copying and pasting information from channel to spreadsheet, or exporting the data directly to your spreadsheet via API, plugin, or another automated option — which is our recommendation, as setting up an automated system to sync your SCMS with new data from these channels will make your life much easier. We recommendZapier, supermetrics, or automate.io depending on your software stack. Step 3.a: Prepare Your Codex
Your codex is the heart and soul of the SCMS.
The codex is where you note trends and patterns that will ultimately become your results. This is done by assigning keywords called tags and tracking their use to help you break down what exactly is happening with your data. These keywords take those trends and define them, under what context the trend should be used to depict the type of sentiment on display, and when the trend should not be used. The codex helps to ensure objectivity in the trends being tracked, which makes the sentiment measurable from a social-scientific standpoint. What do I mean by adding “trends” and “patterns” to the codex? As you read through data, you’ll eventually notice some parallels between the comments that your audience is offering to you as feedback. I tend to take note of trends after the third or fourth time they come up in rapid succession. You may be reading through comments and say to yourself, “Hey, I’ve already seen ten people mention something similar to this!” Come up with a word or short phrase that seems to capture what those comments are trying to weigh in on, and document it in the “term” segment of the codex. When you have the five metrics of the SCMS and a couple additional keywords in that “terms” column, take a moment to fill out the rest of the blanks before continuing on. This is going to take a while for you to wrap your head around. Don’t get frustrated. In fact, if you’re already feeling overwhelmed, Give us a call! We have a free hour-long consultation specifically for this so we can help you succeed right out of the gate! Step 3.b: Tag Your Sample Data
This is where you get to start having fun and actually working with the data you’ve collected! After porting it into your spreadsheet, read through the comments.
Let’s say you’re looking at 100 different comments, and 48 of them mention that they love your service, but they had problems with your payment system. Jot down “payment system” in your codex, and move on, making sure to include that tag in any pieces of feedback that mention it. Looking through those same 100 comments, you the notice that 79 of them denote positive interactions with your staff. Again, add “Staff” to your codex, and tag comments where it comes up. Don’t be afraid to tag comments with multiple labels. Last example, say that a single comment gives incredibly detailed, insightful information on how you can improve your consumer’s experience during product pickup. Should you tag this, even though it pops up only once and isn’t really a trend or a pattern? Absolutely. Who’s to say it won’t pop up again, and you won’t end up creating the tag eventually anyways? When the tag has amassed enough quantitative usage, it has become a significant theme or pattern. Any part of the conversations surrounding your organization that could contribute to its health is worth noting. Be careful about these smaller keywords, though. You don’t want to have so many of them that you can’t find anything in your codex. Especially if it only holds true for a handful of comments, see if it would make a good subsection of another trend that you can combine it with. Step 4: Scale Up Your Implementation
After you’ve got some practice tagging and analyzing your data, and you've got a codex of well defined keywords complete with examples of when to (and not to) use them, import the rest of your data from the channels you’ve isolated, and get to analyzing the rest of it!
Especially at first, this may seem a daunting task, like there’s too much information for you and your team to handle but just like other analytics systems, that’s okay. Do what you can. Don’t worry about getting everything analyzed as quickly as possible, feel free to chip away at the information as you can get to it. Remember, this is a cyclical system. Importing and analyzing new data is imperative, even if all of the old data hasn’t been fully sorted through yet. Because of how the SCMS gives your business real-time data from qualitative channels, making sure your implementation is up-to-date should be your primary concern before sitting down to tag. We recommend commiting your team to 1 hour per week of tagging, followed by 15 minutes going over what you've all discovered and deciding on new trends. If it's just you, 1 hour is fine. Again, this takes time for you to settle into a rhythm. That’s okay. Focus on doing the job right, and eventually you can work on doing it faster. Like anything else, qualitative data analysis is a skill that improves the more you do it. On Automation
We strongly recommend 1 hour a week at minimum going over this data. In team marketing meetings, this personal exposure will give depth and nuance to the views imparted in your data channels, allowing you to accurately and confidently communicate how those tricky-to-track participants on formerly inaccessible data channels think and feel.
What does this have to do with automation? Well, we suggest implementing automated functions to help organizations with the resources to analyze massive amounts of qualitative data quickly. However, Venia and I want to stress that the human element at play here is still vital, and there still needs to be interaction of real people with this system. Automated assistance is only as good as the information guiding it. You could be missing valuable data by not double-checking the work the system is doing, and the more work within the SCMS you do, the better it learns how to assist you. So your work doesn’t stop with the inclusion of automated processes like keyword analysis. It just makes it easier for large amounts of data processing to occur much more quickly. Which is something I, personally, think is pretty awesome! Now It’s Your Turn!
There you have it, a quick and easy crash-course into the SCMS and qualitative analysis.
Make sure to bookmark this page into a folder full of resources so you can reference it later. If you’re confused, I want to once more encourage you to take us up on our free one-hour consultation. It will help. We promise. If you have any questions or comments leading up to your consultation, or just want to reach out for clarification, just drop your feedback in the comment box below, or email me at dylan@sociallyconstructed.online, and I will do everything I can to help you out. Tune back in next week, for the long-awaited followup to the post “Who We Are, and Why You Should Care About SociallyConstructed.Online”!
We discussed some fundamental things first regarding the communication platform for meetings for the next 3 months. Along with this, we discussed the mode of updating progress and some tasks to be done prior to the coding period. He explained every component of CHAOSS, how any data point is moved from one to another, to get an overview of the community. I was made aware of several Working Groups present in the community and the official mailing lists. He proposed some tasks to help me get familiar with grimoirelab and ELK.
He gave excellent advice to make a project log Github repository and keep updating it with time. It can act like a project-tracker repository and will make it easier to track the project’s developments. It will contain all blogs and a summary of all weekly meetings. On Tuesday I met the rest of my mentors. It was absolutely a pleasure meeting them. We had a friendly session to discuss how things are going to look in the next three months. I bombarded them with my multiple questions about SCMS(Social Currency Metric System), to which they explained the concepts and the idea behind the metric system in greater detail. We discussed some details on communication platforms and blog details too. Meeting Conclusions:
The Community-Bonding Plan:
After meeting my mentors, I could feel a huge bubble of inspiration within me!!
In this blog series, at opensource.com, and on her own page, Ria is going to update us on her project every week!
Before we launch in to her first blog though...Ria, It is absolutely wonderful to have you! This blog has been cross-posted from Ria's blog with her permission... “Magic happens when you don't give up, even though you want to. The universe falls in love with a stubborn heart.” 
It had been my first encounter with the open-source world.
Despite very encouraging mentors, it was very intimidating at first. I can still remember the numbness I felt after submitting my first PR. There were times when I self-doubted myself, but as it is said, the mornings bring a ray of hope along, I got back to my feet to work. I had applied for both GSoC and Outreachy under the organisation CHAOSS, and on May 04, I was euphoric after getting selected in both! I decided to continue with GSoC during the summer of 2020. The account of me getting selected for GSOC’20 came as a piece of pleasant news to me. I got selected for GSOC under the organisation CHAOSS, for the project “Implementing Social Currency Metric System in Grimoirelabs”. I will be working for the next 3 months, starting from June- Aug’20 remotely on this project for the software improvement position. My project has 4 mentors from different geographical locations.
I am glad to be a part of such a welcoming and encouraging community.
Want to see Ria's progress with the project in more real-time? check out her project tracker here!
About the Project
Community sentiment is based on the opinions and expectations of community members which is very important for framing decisions. Member input in essential decisions to public health is very important and thus, helps in better decisions which in turn help in the better framework and execution. Collecting and processing all the data like emails, comments, issues, pull requests, tweets etc. will allow community leaders to make key quality decisions regarding transparency and actionability of open source project health. Data is tagged with respect to social currency constructs like Merit, Trust, Transparency, Utility, Consistency.
By Implementing the Social Currency Metric System, we will be able to measure the value of community interactions to accurately gauge the ‘reputation’ of a community. Implementing Social Currency Metric System (SCMS) will be a huge milestone in providing a better view of project health in the open-source community. That is because, through SCMS, we are adding another dimension of social currency in the metric. Measuring community interactions, analysing it will definitely take CHAOSS one step ahead to measure project health. My Approach
My Role in the Project
Stretch Goals
Thank you
I thank my family, friends and super helping mentors for supporting me and encouraging to work harder. ❤
I plan to dedicate myself completely to complete the project efficiently. I hope I do due justice to the project and the community :) Looking forward to loads of fun and a fantastic learning experience! “ I’m not afraid of storms, for I’m learning how to sail my ship!”
Follow the rest of her journey!
This past January, our friends over at CHAOSS announced the newest round of community metrics to go into their community analytics system, GrimoireLab, and nestled in those new metric updates was a type of metric that has not yet been featured before in CHAOSS... Our Social Currency Metrics System (SCMS)!
Then on May 4th, we were told that we had been accepted alongside side software developer Ria Gupta for the Google Summer of Code, to write the SCMS directly into GrimoireLab as a metric! In this blog we'll explain why we’ve been added to GrimoireLab, why we think it matters for you, and most importantly what the next steps are in this exciting new process, so you can implement the system, for free, starting today! Why the SCMS was added,
|
|
To do this The Social Currency Metrics System augments your quantitative data using a framework based on the theory of Social Currency, to measure moments your community exhibits trust in you.
If you're interested in learning more on the system and setting it up for free let's get to next steps. Here's how it works in Grimoire Labs. |
"At the heart of a community’s value, it’s not about what people have done, but how community members express themselves in the process of doing." |
What's next for us, and for you?
There have been some amazing and wonderous developments before we've even been able to launch SociallyConstructed.Online's brand. But now, with so much momentum, we've got a lot coming down the pipeline:
1st, remember that Software developer, Ria Gupta, you heard about?
She just started her own full series releasing every sunday detailing how she's building the SCMS in Grimoire lab on here, her personal blog, and on the CHAOSS Blog!
2nd, listen to the new CHAOSScast podcast!
The CHAOSScast podcast just launched with Georg Link, Matt Broberg, and other magnificent panelists so give it a listen and the SCMS is the second episode premiering Friday May 29th!
Venia will also be a regular panelist on the podcast so join us to talk about all things community analytics!
1st, remember that Software developer, Ria Gupta, you heard about?
She just started her own full series releasing every sunday detailing how she's building the SCMS in Grimoire lab on here, her personal blog, and on the CHAOSS Blog!
2nd, listen to the new CHAOSScast podcast!
The CHAOSScast podcast just launched with Georg Link, Matt Broberg, and other magnificent panelists so give it a listen and the SCMS is the second episode premiering Friday May 29th!
Venia will also be a regular panelist on the podcast so join us to talk about all things community analytics!
3rd, start up the SCMS for FREE using the new instructions over at CHAOSS!
We are providing this system as a methodology for free because we truly believe that augmenting your data with qualtiative social listening will make a better, safer, more secure online community.
We are providing this system as a methodology for free because we truly believe that augmenting your data with qualtiative social listening will make a better, safer, more secure online community.
And of course if you'd like to learn more, get a personalized demo, or have us set up your SCMS, we offer a free 1-hour consultation as well!
|
Part 1 of a 3 part series on how we're changing analytics.
|
Part 2: What is social currency?
|
How can online marketing, an industry that’s infamous for its metrics-obsessed capacity to be downright predatory, become as much a benefit to consumer communities as it is for the business?
SociallyConstructed.Online was founded to answer this question.
By focusing on bettering brands through participation and collaboration with the communities supporting them, Samantha “Venia” Logan sought to lessen the marketing industry’s reliance on morally gray, overzealous behavioral tracking systems.
After offhandedly mentioning this idea to yours truly (Dylan), we took a rough idea and built it from pipe dream to a business focused on improving the industry’s relationship with online communities using our own data analytics system — the Social Currency Metric System.
SociallyConstructed.Online’s purpose today is simple:
We equip those in charge of online communities with tools that ethically “listen” to what those contributing to conversations online are saying.
The end goal is to take these insights, and help community managers, data analysts, influencers and business owners collaborate with the communities supporting them. We provide real-time qualitative data to help orient themselves in a direction that will promote business growth and consumer goodwill.
This collaborative effort relies primarily upon passive social listening of data channels such as Facebook feeds or support ticket queues. By relying on qualitative data from conversations surrounding your brand instead of intrusive quantitative tracking, we’ve taken lessons from academia. The social sciences' use of data analysis methods to study cultures and other such groups has given us a decades’ old tried-and-true toolbox to work with.
Tools that we have made user-friendly enough for daily business and analytics use.
SociallyConstructed.Online was founded to answer this question.
By focusing on bettering brands through participation and collaboration with the communities supporting them, Samantha “Venia” Logan sought to lessen the marketing industry’s reliance on morally gray, overzealous behavioral tracking systems.
After offhandedly mentioning this idea to yours truly (Dylan), we took a rough idea and built it from pipe dream to a business focused on improving the industry’s relationship with online communities using our own data analytics system — the Social Currency Metric System.
SociallyConstructed.Online’s purpose today is simple:
We equip those in charge of online communities with tools that ethically “listen” to what those contributing to conversations online are saying.
The end goal is to take these insights, and help community managers, data analysts, influencers and business owners collaborate with the communities supporting them. We provide real-time qualitative data to help orient themselves in a direction that will promote business growth and consumer goodwill.
This collaborative effort relies primarily upon passive social listening of data channels such as Facebook feeds or support ticket queues. By relying on qualitative data from conversations surrounding your brand instead of intrusive quantitative tracking, we’ve taken lessons from academia. The social sciences' use of data analysis methods to study cultures and other such groups has given us a decades’ old tried-and-true toolbox to work with.
Tools that we have made user-friendly enough for daily business and analytics use.
Why Qualitative Data?
|
Qualitative data analysis is much more difficult than interpreting quantitative data.
Instead of extrapolating something from a number of people taking specific actions analyzing qualitative data seems to require an individual analysis of every data point in a given set before trends can be seen. Without a comprehensive strategy in place or someone trained in evaluating qualitative data, it can be a time-consuming and expensive process, that seems less-than scientific and difficult to act on. Why even attempt it, then? |
John Sculley didn't exactly call that one right did he?
|
While quantitative data can infer how people generally feel, qualitative data gives those feelings context enough to make powerful and insightful observations. It helps to shine light on trends that quantitative data isn’t able to pick up on, and proper collection of qualitative data can reduce or eliminate the cons to qualitative data analysis we covered earlier.
Most, if not all of these, can be mitigated by doing it properly.
As Venia discussed in our blog on the pitfalls of surveys in qualitative data, surveys are prone to failure because they collect “active” data — data that is explicitly requested — rather than “passive data,” or data that is generated as a result of regular participation.
Qualitative data analysis of comments and suggestions left on channels such as Facebook don’t rely on those souls kind enough to take your survey — it’s feedback straight from those who are freely sharing their thoughts and feelings with you.
The more your customers feel like they’re being heard, the more they’ll be willing to share their thoughts and opinions with you in hopes that their pain points will be addressed. The question, then, is how to begin this loop of passive consumer feedback you can act on to improve your business and garner yet more feedback.
SC.O’s answer to that question lies in social capital and our Social Currency Metrics System.
Qualitative data analysis of comments and suggestions left on channels such as Facebook don’t rely on those souls kind enough to take your survey — it’s feedback straight from those who are freely sharing their thoughts and feelings with you.
The more your customers feel like they’re being heard, the more they’ll be willing to share their thoughts and opinions with you in hopes that their pain points will be addressed. The question, then, is how to begin this loop of passive consumer feedback you can act on to improve your business and garner yet more feedback.
SC.O’s answer to that question lies in social capital and our Social Currency Metrics System.
The Social Currency Metrics System
The Social Currency Metrics System, or the SCMS, for short, is how SociallyConstructed.Online helps businesses involve their consumers directly in their reputation management and decision making processes.
Reputation is a paramount concern of community managers and businesses alike, as it plays an increasingly tangible role in the success of a business, and the likeliness of patrons willing to support them. Many people no longer just purchase products, but rather seek to support their friends and family as well as those whose values and goals match up with their own.
A key component of that “relatable small-business” mentality is the ability to understand the wants and needs of your consumer base in real time, so you can pivot to meet your clientele’s needs as they arise.
SC.O does that by an evaluation of five specific “traits” used in garnering reputation:
Reputation is a paramount concern of community managers and businesses alike, as it plays an increasingly tangible role in the success of a business, and the likeliness of patrons willing to support them. Many people no longer just purchase products, but rather seek to support their friends and family as well as those whose values and goals match up with their own.
A key component of that “relatable small-business” mentality is the ability to understand the wants and needs of your consumer base in real time, so you can pivot to meet your clientele’s needs as they arise.
SC.O does that by an evaluation of five specific “traits” used in garnering reputation:
|
TRANSPARENCY
UTILITY CONSISTENCY MERIT TRUST |
Is your brand easily understandable?
Is your brand contributing value? Are you being reliable and dependable? Does your brand merit respect and attention? Can people trust that you'll continue to provide value? |
Businesses begin by identifying what each of these metrics mean to them, and then they can begin to sort through the qualitative data they collect in order to see where their consumers think they shine, and where they could use some work.
The system is an extraordinarily simple one that can lead to surprisingly in-depth qualitative data analysis. It is capable of offering up an obscene amount of information at the cost of only 1-2 hours a week, once the system is set up and ready to go.
Week-to-week “snapshots” of how businesses are doing in the eyes of their consumers can help businesses brainstorm plans of attack. By utilizing the SCMS to form actionable next steps in accordance with the wishes of their clientele, businesses can achieve tangible and measurable gains in customer sentiment, practically in real time.
Quantitative data has reigned supreme since data-harvesting tools and the sort of demographic information made available by companies like Facebook made the information so easy to come by.
Completely rewriting the playbook is a big ask.
That’s why we aren’t asking you to change the way you market to your audiences. We’re only suggesting that you supplement it with the tactics and strategies we have to offer.
The system is an extraordinarily simple one that can lead to surprisingly in-depth qualitative data analysis. It is capable of offering up an obscene amount of information at the cost of only 1-2 hours a week, once the system is set up and ready to go.
Week-to-week “snapshots” of how businesses are doing in the eyes of their consumers can help businesses brainstorm plans of attack. By utilizing the SCMS to form actionable next steps in accordance with the wishes of their clientele, businesses can achieve tangible and measurable gains in customer sentiment, practically in real time.
Quantitative data has reigned supreme since data-harvesting tools and the sort of demographic information made available by companies like Facebook made the information so easy to come by.
Completely rewriting the playbook is a big ask.
That’s why we aren’t asking you to change the way you market to your audiences. We’re only suggesting that you supplement it with the tactics and strategies we have to offer.
SC.O is Not Just An Analytics agency
SociallyConstructed.Online’s biggest strength is not just in our role as the inventors of the SCMS analytics platform. We're also experienced small-business consultants with almost 20 collective years of experience in the digital marketing industry.
We are skilled and practiced in areas such as rebranding, development or optimization of marketing funnels, content creation, SEO, and more.
None of these are any less important than sentiment analysis when it comes to digital marketing. So while the integration of the SCMS into existing data analytics strategies is our primary service, SC.O takes a full-stack approach to our digital marketing services so you can pick and elevate the strategies that will reap the most benefit.
Sales pitch over.
The point I’m trying to make is that Venia and I built SociallyConstructed.Online because we as marketers need to do better.
We came to this conclusion because of growing disillusionment with the status quo of digital marketing. The wild, wild west days of digital marketing are over. Laws like the GDPR are having a profound effect on the sorts of information companies are allowed access to and how they’re stored.
Choosing not to participate with these regulations is no longer an option. SC.O’s SCMS system makes adhering to these guidelines easily achievable, by rendering the morally gray behavior tracking implementations that collect information on consumers they don’t want you to have, unnecessary.
Part 2 of this series will cover social currency as it pertains to qualitative marketing, and how it has become the backbone of our SCMS system. Sign up for our newsletter to keep in touch with our new releases, and make sure to stay tuned, for more exciting content is coming down the pipe soon!
We are skilled and practiced in areas such as rebranding, development or optimization of marketing funnels, content creation, SEO, and more.
None of these are any less important than sentiment analysis when it comes to digital marketing. So while the integration of the SCMS into existing data analytics strategies is our primary service, SC.O takes a full-stack approach to our digital marketing services so you can pick and elevate the strategies that will reap the most benefit.
Sales pitch over.
The point I’m trying to make is that Venia and I built SociallyConstructed.Online because we as marketers need to do better.
We came to this conclusion because of growing disillusionment with the status quo of digital marketing. The wild, wild west days of digital marketing are over. Laws like the GDPR are having a profound effect on the sorts of information companies are allowed access to and how they’re stored.
Choosing not to participate with these regulations is no longer an option. SC.O’s SCMS system makes adhering to these guidelines easily achievable, by rendering the morally gray behavior tracking implementations that collect information on consumers they don’t want you to have, unnecessary.
Part 2 of this series will cover social currency as it pertains to qualitative marketing, and how it has become the backbone of our SCMS system. Sign up for our newsletter to keep in touch with our new releases, and make sure to stay tuned, for more exciting content is coming down the pipe soon!
popular posts
|
|
|
Part 3 of a 3 part series on community surveys.
|
We’re hypocrites. We can admit it.
We just spent an inordinate amount of time across the previous 2 blogs touting the huge success of a 2-question survey, the Net Promoter Score (NPS).
For 2 weeks we’ve covered how and why the Net Promoter Score was one of the single best systems to start measuring your community audiences... and here we are now, telling you the NPS survey is, although useful, fundamentally flawed.
What gives?
Here’s the truth:
We just spent an inordinate amount of time across the previous 2 blogs touting the huge success of a 2-question survey, the Net Promoter Score (NPS).
For 2 weeks we’ve covered how and why the Net Promoter Score was one of the single best systems to start measuring your community audiences... and here we are now, telling you the NPS survey is, although useful, fundamentally flawed.
What gives?
Here’s the truth:
Social scientists and businesses use surveys
very differently, and businesses usually do it wrong.
Don’t get us wrong. Survey systems like the Net Promoter Score, Customer Satisfaction, and Sense of Community have been used for a long time in business, to great success.
But the reason most CEOs and Data Analysts just take a glance at the graphs, rip percentages out of context, and abandon them in their Survey Monkey account until next year, is because there are some fundamental problems with how businesses view the almighty survey.
In this blog, we’re going to go over 3 pitfalls to survey production, delivery, and analysis that have caused the average Marketer and CEO to distrust their community’s responses.
But the reason most CEOs and Data Analysts just take a glance at the graphs, rip percentages out of context, and abandon them in their Survey Monkey account until next year, is because there are some fundamental problems with how businesses view the almighty survey.
In this blog, we’re going to go over 3 pitfalls to survey production, delivery, and analysis that have caused the average Marketer and CEO to distrust their community’s responses.
But…
To prove we’re no negative Nancy and to stand by our promotion of the Net Promoter Score in the past two blogs, we’re also going to provide you grounded and simple solutions to avoid, fix, or altogether improve your survey implementations and convince your higher-ups to trust your respondents’ feedback.
Here we go!
To prove we’re no negative Nancy and to stand by our promotion of the Net Promoter Score in the past two blogs, we’re also going to provide you grounded and simple solutions to avoid, fix, or altogether improve your survey implementations and convince your higher-ups to trust your respondents’ feedback.
Here we go!
|
Problem 1: There's too much detailed data
to sort through & not enough time
RESCQU.NET Report
When I was working at RESCQU.NET, our typical "community census" involved weeks of meetings and a lot of back and forth between myself and my volunteers.
We would spend tons of time figuring out what we needed to ask of our members, how best to ask it, and what we were looking for in an answer that would really matter.
We then spent several weeks working tirelessly to market the thing. We'd put out preliminary feelers, publish the survey, and bam, just like that, we'd get 1500 strongly worded opinions responses we had no real idea what to do with.
One of the ways we made processing easier on us was to limit the amount of "qualitative data" we'd collect because each question equaled an opinion, and that meant 1500 people times 10 comment boxes.
It was simply too much, so we structured the survey to make it easier for us to handle.
I've found that the same general approach happens in any company of any scale. Qualitative data is just a fire hose no one wants to turn on and if you do, no one wants to go through it. Even if it holds information that will save you a dumpster fire PR disaster, or generally improve your service, excellent. It's still not likely that you'll go through it.
The issue here is that most people pick questions intended to reduce the workload later on and, in doing so, limit the responses you receive. Alternatively, many go the other way and produce a survey with so much data that making sense of it becomes insurmountable.
As a result you're incentivized to ignore the qualitative data, which is the bulk of the survey's value.
To solve this problem instead of working to avoid it, learn how to best process the information you're receiving. We recommend learning how to "abstract" or tag your data for quick and easy tallying later.
We'll have a full blog on how to tag your data and "abstract" it into easy-to-digest themes in a few weeks so be sure to check back here for that, but here's the gist:
We would spend tons of time figuring out what we needed to ask of our members, how best to ask it, and what we were looking for in an answer that would really matter.
We then spent several weeks working tirelessly to market the thing. We'd put out preliminary feelers, publish the survey, and bam, just like that, we'd get 1500 strongly worded opinions responses we had no real idea what to do with.
One of the ways we made processing easier on us was to limit the amount of "qualitative data" we'd collect because each question equaled an opinion, and that meant 1500 people times 10 comment boxes.
It was simply too much, so we structured the survey to make it easier for us to handle.
I've found that the same general approach happens in any company of any scale. Qualitative data is just a fire hose no one wants to turn on and if you do, no one wants to go through it. Even if it holds information that will save you a dumpster fire PR disaster, or generally improve your service, excellent. It's still not likely that you'll go through it.
The issue here is that most people pick questions intended to reduce the workload later on and, in doing so, limit the responses you receive. Alternatively, many go the other way and produce a survey with so much data that making sense of it becomes insurmountable.
As a result you're incentivized to ignore the qualitative data, which is the bulk of the survey's value.
To solve this problem instead of working to avoid it, learn how to best process the information you're receiving. We recommend learning how to "abstract" or tag your data for quick and easy tallying later.
We'll have a full blog on how to tag your data and "abstract" it into easy-to-digest themes in a few weeks so be sure to check back here for that, but here's the gist:
|
|
To abstract data easily, port the data into a Word doc or spreadsheet and place comments on the feedback you find interesting. In that comment, use a simple 1 or 2-word phrase that encapsulates the theme of the statement. Note down what you mean by that term and then use that term each time you see similar feedback. Eventually, you'll see that theme pop out of the text frequently.
|
It's like highlighting the important ideas in a book. When a term comes up 40 times, it's probably more important than terms that turn up once or twice. Eventually, you'll have a count of themes and concepts that jump out at you as trends and patterns you can act on.
Problem 2: Only a few get to
(or even want to) speak
“The only people who will take your survey,
are people who take surveys."
This next issue has less to do with setting up the survey to succeed and more to do with the audience receiving the survey.
One of the most common issues with surveys is that they require a person to take time out of their day to do something they weren’t planning to do, and put in effort they weren’t initially anticipating.
3 different kinds of “fallacies” rear their ugly heads here and build on each other to create a nasty issue with your resulting qualitative data set.
And if you can’t sidestep these fallacies, your survey is bunk. These fallacies are the main reasons data nerds cite when they poo-poo the idea of collecting opinions via survey.
I’ll explain each before we get into ways to look out for them.
Fallacy 1: Vocal Minorities or Polarized Involvement
In general, only about 2% of any community will be labeled “power-users.” These are the people who are ALWAYS talking and always giving opinions. Usually, they’re also the ones you interact with and trust the most.
On the flip side, detractors tend to be hyper-vocal about their opinions. As the saying goes, “Negative PR is about 10 times stronger than good PR.”
And then there’s the middle.
Fence-sitters tend to be less vocal and less invested. So getting their opinion is difficult. That means you’ll get biased answers from your polarized users, and fewer from them.
Fallacy 2: Survey Fatigue or more broadly The Diminishing Value of Work
You’ve likely run into the term Survey Fatigue before, but you probably haven’t spent much time digging into the theory behind it.
The diminishing value of work refers to the initial value a respondent feels completing the survey is worth at the beginning, and how that value is impacted as they move through it. There is a certain amount of commitment required for a person to perform any action, and this occurs every time a member participates in your community.
Each survey question is an additional amount of work. As effort is put into the survey, the value of that survey may become “less worth it.” Eventually, the value of the survey and how much effort they’ve put into it is no longer justified, and they click off.
Many people view this as a survey’s length and how long the questions are, but in reality, short or long doesn’t matter. It’s about imparting enough value before, during, and after they fill out the survey, that they feel their action is still worthwhile by the last question.
Fallacy 3: The Spiral of Silence
This last fallacy is less known, but you can think of it as the ultimate consequence of letting fallacies 1 and 2 get too far out of hand.
The vocal bias skews our data to favor the involved. Fence-sitters won’t see as much value but are still important. If you make decisions based on the more vocal than over time fence-sitters lose any sense of influence they did have and begin to think their opinion, if they had provided it, wouldn’t have made a difference.
So they start to think their opinion isn’t valued or you won’t listen to them. Then they intentionally refuse to count it. As a result, their ideas aren’t heard, and their voices DO become of less value.
If this sounds a lot like a certain country’s political situation - you’re right. It’s the exact same mechanism, and it happens at every level of a community; small group to policy.
Now let’s talk about solutions.
To get around these fallacies there are a lot of tactics and fail-safes you can implement. A lot of organizations will apply extrinsic rewards like raffles and badges that have a more stable “value” to their surveys to ensure the value is viewed more fairly.
It should be no surprise as community managers at SC.O that we don’t recommend that approach. Extrinsic reward is a great way to devalue the intrinsic value of influence by way of participation. It’s got less value for the work if you give them something detached from your brand.
On top of this, the quality of the submissions you get from those who simply want the reward at the end of the survey may not give responses that match in quality with those who are driven by intrinsic motivations.
Instead we recommend making the work smaller and spread out over time by adding 1-2 question surveys like the NPS to your regular community management or social media campaigns. Then have real public conversations that credit those thinkers and use those results to perform a transparent action.
The questions will encourage “passive engagement” rather than require active commitment so the effort is lower and the conversation is viewed as valuable. It will also pull some of your “lurkers” and “fence-sitters” out of their holes if you spin the conversation toward them. Consider priming an audience before the survey with a#LoveOurLurkers campaign!
PROBLEM 3: MOST SURVEYS ARE noT
"SCIENce ENOUGH” TO JOIN THE SCIENCE CLUB
Let’s get elementary now..
“If your survey uses the scientific method over
the social-scientific process, you’re not collecting
your data correctly, at all."
The scientific method is predicated on the systematic manipulation of “variables".
What is the cause-effect relationship between your studied thing and your hypothesis? The idea is to control as many variables as possible and test the relationships between 1-3 unknown variables. This allows you to solidify correlations into findings and then theories.
This works great in lab environments, on problems with clearly defined answers, when different approaches have clear upsides and downsides, or with scientific principles that are the same no matter where you go.
But that’s simply not reality when you start adding people, culture, and social structures throughout the big wide world to the mix.
People are too diverse and do things for too many different reasons. Often their actions can only be defined correlationally.
And that is why the great social-scientists of the 1900s built on the scientific method with the lesser known but ridiculously impactful “social-scientific process”.
What is the cause-effect relationship between your studied thing and your hypothesis? The idea is to control as many variables as possible and test the relationships between 1-3 unknown variables. This allows you to solidify correlations into findings and then theories.
This works great in lab environments, on problems with clearly defined answers, when different approaches have clear upsides and downsides, or with scientific principles that are the same no matter where you go.
But that’s simply not reality when you start adding people, culture, and social structures throughout the big wide world to the mix.
People are too diverse and do things for too many different reasons. Often their actions can only be defined correlationally.
And that is why the great social-scientists of the 1900s built on the scientific method with the lesser known but ridiculously impactful “social-scientific process”.
The primary reason people believe that data collected from surveys is highly subjective is because the data stopped at step 3 in this larger process. You collected it, looked at it, found some cool stuff, and said, “huh, looks like this is a thing”. It doesn’t have the ability to solidify any correlations you make into clear causal effects.
The social scientific process creates objective data out of subjective data by taking the cause-effect relationship of the scientific method further; it tests the environmental factors at the same time as the variables using the rule of generalization.
For example, on the traditional scientific method your survey goes around it once:
This same process makes a really solid go at steps 1-3 of the social method, but it stops at the rule of generalization.
It doesn’t investigate the limitations of those hypotheses, it doesn’t root out fallacious conclusions, it doesn’t generalize to wider audiences, and it doesn’t test the limits of what you’ve learned so you know where the correlation ends.
If you do the survey using the social-scientific process it goes around the scientific method a full 3 times before you get “results”.
So, this social-scientific method is the reason we love the Net Promoter Score. If you implement the NPS like we taught you in our prior blogs, it will cover a full go-around of the social-scientific process as it happens over and over again.
The social scientific process creates objective data out of subjective data by taking the cause-effect relationship of the scientific method further; it tests the environmental factors at the same time as the variables using the rule of generalization.
For example, on the traditional scientific method your survey goes around it once:
- You observed trends in your community
- You wrote a survey about it
- You wrote your questions specifically to suss out those variables
- You got your results, analyzed them, and made a report
- You disseminated them to the powers that be
This same process makes a really solid go at steps 1-3 of the social method, but it stops at the rule of generalization.
It doesn’t investigate the limitations of those hypotheses, it doesn’t root out fallacious conclusions, it doesn’t generalize to wider audiences, and it doesn’t test the limits of what you’ve learned so you know where the correlation ends.
If you do the survey using the social-scientific process it goes around the scientific method a full 3 times before you get “results”.
So, this social-scientific method is the reason we love the Net Promoter Score. If you implement the NPS like we taught you in our prior blogs, it will cover a full go-around of the social-scientific process as it happens over and over again.
To Conclude
The Net Promoter Score is great because it makes active data collection passive collection, it is continually available, it only prompts for comments when people really want to provide them, and it's based on emotions rather than pre-assessed logic.
So to conclude, we don’t want to discourage you from implementing these awesome community management tools. We are not against surveys.
What we are saying is that the way you implement these community analytics tools needs to be done with these issues in mind. Each of these problems is a reason people have started to mistrust qualitative data for the past several decades.
So to conclude, we don’t want to discourage you from implementing these awesome community management tools. We are not against surveys.
What we are saying is that the way you implement these community analytics tools needs to be done with these issues in mind. Each of these problems is a reason people have started to mistrust qualitative data for the past several decades.
|
We aim to fix that by making qualitative data easier to collect and analyze, more objective, and harder to read falsely into by taking your use of qualitative data further with our Social Currency Metrics System.
Check out the system and how to build your own for free here, or read the previous two parts of this blog! |
|
This 3-Step Guide Will Install an NPS System In Your Community.
|
Part 2 of a 3 part series on community surveys.
|
The Net Promoter Score, or the NPS, is a very unique 2-question survey borne out of the customer service industries of Amazon, Netflix, and other internet giants.
The NPS asks for your target audience to describe how they feel about you. But it asks them at several intervals of their customer value journey, and then each time it asks, it offers a completely optional and unassuming (but hugely rewarding) comment box below it.
It may seem simple, but the Net Promoter Score is almost single-handedly responsible for measuring essential quantitative data in the biggest companies, and as we covered in part 1 of this blog series, it's no exaggeration to say that it saved Comcast's reputation.
This blog will guide you through planning, designing, and implementing the Net Promoter Score for your community, brand, and/or team in 5 steps.
The NPS asks for your target audience to describe how they feel about you. But it asks them at several intervals of their customer value journey, and then each time it asks, it offers a completely optional and unassuming (but hugely rewarding) comment box below it.
It may seem simple, but the Net Promoter Score is almost single-handedly responsible for measuring essential quantitative data in the biggest companies, and as we covered in part 1 of this blog series, it's no exaggeration to say that it saved Comcast's reputation.
This blog will guide you through planning, designing, and implementing the Net Promoter Score for your community, brand, and/or team in 5 steps.
What Implementing The NPS Will Take:
We expect that executing these steps to will take 1-2 weeks of fairly light work.
Preparing to implement the NPS is mostly about getting all of the stakeholders into one room to decide where the survey should be placed, and how it will be supported.
Once the system is set up, you can likely expect it to take about 1 hour per week to analyze what comes out of it.
Note: This blog won't cover the contents of the survey. Review part 1 of this blog series for that.
Preparing to implement the NPS is mostly about getting all of the stakeholders into one room to decide where the survey should be placed, and how it will be supported.
Once the system is set up, you can likely expect it to take about 1 hour per week to analyze what comes out of it.
Note: This blog won't cover the contents of the survey. Review part 1 of this blog series for that.
1. Where Does the Target Audience Meet You?
As with all new analytics systems, planning is your first step.
List all the direct points of contact you have with your community, customers, or target audience. This is where your people are talking to them. Regular emails? Support tickets? Over the phone? Where are your touch-points on social media?
Then list all the indirect points of contact your target audience has with your brand. This means they’ve found your brand and your project, but they’re not interacting with your people. Examples of indirect contact include landing pages, websites, FAQ pages, YouTube how-to videos, and so on.
Next, consider who the people supporting these channels may be. They could be support representatives, lead posters on a forum, volunteers, and employees, but also contractors, admins, community members, and the like. Now you know who you'll need to have a discussion with to implement an NPS system across your organization.
So set up a meeting with those who are involved in running each of these channels. Reach out to them to discuss what your goals are before you set up an NPS system. Share this blog with them to get them to buy in, and get some feedback on how they would prefer to receive the NPS survey from those filling it out.
By the end of Step 1's meeting, you should have an idea of how and where you should implement the NPS survey most effectively. This implementation will probably look different for each of the channels you target, but since it's just a question and a comment box, it’s usually pretty low effort.
List all the direct points of contact you have with your community, customers, or target audience. This is where your people are talking to them. Regular emails? Support tickets? Over the phone? Where are your touch-points on social media?
Then list all the indirect points of contact your target audience has with your brand. This means they’ve found your brand and your project, but they’re not interacting with your people. Examples of indirect contact include landing pages, websites, FAQ pages, YouTube how-to videos, and so on.
Next, consider who the people supporting these channels may be. They could be support representatives, lead posters on a forum, volunteers, and employees, but also contractors, admins, community members, and the like. Now you know who you'll need to have a discussion with to implement an NPS system across your organization.
So set up a meeting with those who are involved in running each of these channels. Reach out to them to discuss what your goals are before you set up an NPS system. Share this blog with them to get them to buy in, and get some feedback on how they would prefer to receive the NPS survey from those filling it out.
By the end of Step 1's meeting, you should have an idea of how and where you should implement the NPS survey most effectively. This implementation will probably look different for each of the channels you target, but since it's just a question and a comment box, it’s usually pretty low effort.
For example...
Take an email going out to those that directly interact with you, like a weekly newsletter. You can embed the survey directly into the bottom of the email and each of the numbered faces would lead to a different landing page where you take their comment.
If that isn't going to be particularly helpful because you don't rely on emails to communicate within your community, you can put the NPS at the bottom of a Support landing page, or it could be a pop-up on a thank-you page once they've completed a specific action.
It could even be a podium, as with this McDonald’s Kiosk. Regardless, it should be implemented in the easiest way possible for people to provide their responses.
Take an email going out to those that directly interact with you, like a weekly newsletter. You can embed the survey directly into the bottom of the email and each of the numbered faces would lead to a different landing page where you take their comment.
If that isn't going to be particularly helpful because you don't rely on emails to communicate within your community, you can put the NPS at the bottom of a Support landing page, or it could be a pop-up on a thank-you page once they've completed a specific action.
It could even be a podium, as with this McDonald’s Kiosk. Regardless, it should be implemented in the easiest way possible for people to provide their responses.
2. Implement and Prepare To Score the Data
The next step is to put the NPS in the places you've just isolated. Once you've done that, you're ready to go!
Well... almost ready to go. Now you have to figure out how to support it. Don't worry, we can help!
When the reports come rolling in, don’t consider this a one-and-done survey situation. If you don't use it to discover community member pitfalls and figure out ways to fix them, installing the NPS was useless.
People are not leaving comments to blow off steam. This is a social contract. When they write a comment, they are doing that with the expectation that you will read and consider what they have to say. Especially if they aren't the only person giving that type of feedback.
NPS is an around-the-clock system. Keep it going and add it to your analytics meeting every week. Score the data according to the following chart. Then discuss it with your project team.
Well... almost ready to go. Now you have to figure out how to support it. Don't worry, we can help!
When the reports come rolling in, don’t consider this a one-and-done survey situation. If you don't use it to discover community member pitfalls and figure out ways to fix them, installing the NPS was useless.
People are not leaving comments to blow off steam. This is a social contract. When they write a comment, they are doing that with the expectation that you will read and consider what they have to say. Especially if they aren't the only person giving that type of feedback.
NPS is an around-the-clock system. Keep it going and add it to your analytics meeting every week. Score the data according to the following chart. Then discuss it with your project team.
How scoring works...
Promoters are the only people who really count for NPS. The weight puts the score in your audiences’ favor. These are the people who are giving you the social reputation you need to really progress your company. Most people will probably be either dissatisfied or kind of “meh” about your product. Put them in a group labelled “needs improvement.”
Subtract the percentage of your promoters from your general detractors and voila: You’ll get a number that your CEO should be happy with, when compared to others.
Promoters are the only people who really count for NPS. The weight puts the score in your audiences’ favor. These are the people who are giving you the social reputation you need to really progress your company. Most people will probably be either dissatisfied or kind of “meh” about your product. Put them in a group labelled “needs improvement.”
Subtract the percentage of your promoters from your general detractors and voila: You’ll get a number that your CEO should be happy with, when compared to others.
3. Leverage NPS Results to Improve the Score
NPS is a powerful tool that introduces a treasure trove of insights that qualitative data can provide if utilized properly.
By tracking your NPS score over time and then using the trends you find to make changes in your business, you'll find that you can make a change on Monday and see a tangible increase or decrease in your NPS score as soon as the next Friday. Over time, you can challenge your teams to hit goals based on the feedback you recieve.
In truth, NPS is pretty low-hanging but very juicy fruit among qualitative analysts. There’s tons of insights left on the table for you to pick up as you get better.
As Chris Mercer at MeasurementMarketing says;
“First get good. Then get better.”
So, if you’re ready to see what NPS can do for your company, implement it!
but....
By tracking your NPS score over time and then using the trends you find to make changes in your business, you'll find that you can make a change on Monday and see a tangible increase or decrease in your NPS score as soon as the next Friday. Over time, you can challenge your teams to hit goals based on the feedback you recieve.
In truth, NPS is pretty low-hanging but very juicy fruit among qualitative analysts. There’s tons of insights left on the table for you to pick up as you get better.
As Chris Mercer at MeasurementMarketing says;
“First get good. Then get better.”
So, if you’re ready to see what NPS can do for your company, implement it!
but....
The limits of the Net Promoter Score
The problem with the NPS is that it is limited to those who are willing to speak and to listen. It's impact hinges on whether you take action to fix it. So Do not reduce the NPS to a simple set of numbers, or you’re doing it wrong. |
One of the most important concepts we at SC.O push is called the rule of generalization:
|
A Concept Applies to B population only so far as C limitation.
|
To truly leverage the Net Promoter Score, you need a plan for understanding the qualitative data these comments give you objectively and at scale.
It’s one thing to read the comments you receive and act on them when they impact your team. It’s quite another thing to recognize trends in the comments are limited by a few harsh realities for any survey you ever use:
Because this is a survey that relies on an action, 70% of your audience is not actively telling you what they think, just like a regular sales funnel.
So, in part 3 of this series on surveys, we're going to cover what those limitations are, and what you can do to get around them. Stay tuned for next week!
Let us know if you need help getting started, find hang-ups, or have any questions. I’ll watch the comments below and you can email me at Samantha@SociallyConstructed.Online.
It’s one thing to read the comments you receive and act on them when they impact your team. It’s quite another thing to recognize trends in the comments are limited by a few harsh realities for any survey you ever use:
Because this is a survey that relies on an action, 70% of your audience is not actively telling you what they think, just like a regular sales funnel.
So, in part 3 of this series on surveys, we're going to cover what those limitations are, and what you can do to get around them. Stay tuned for next week!
Let us know if you need help getting started, find hang-ups, or have any questions. I’ll watch the comments below and you can email me at Samantha@SociallyConstructed.Online.
THE 2-question survey THAT saved comcast.
|
Part 1 of a 3 part series on community surveys.
|
In 2014, a technology reviewer recorded a customer service call with Comcast to cancel his account.
The incident went so horribly viral that TIME magazine wrote about it - and the fallout did not end well. Comcast ended 2014 with the distinction of being listed below Monsanto as the worst company in America.
So you might imagine my apprehension when, down on my luck and without enough freelance clients to support myself, I went to work for a newly-formed Comcast Xfinity call center in Fort Collins, Colorado.
I expected a repeat of the nightmare above.
Instead, I joined one of the best companies to work for in 2019.
Comcast - A veritable titanic of a company renowned for its “meh” service packages and its awful customer service - went from “worse than Monsanto” to one of INC.’s top 100 companies to work for in the space of just 5 years.
The incident went so horribly viral that TIME magazine wrote about it - and the fallout did not end well. Comcast ended 2014 with the distinction of being listed below Monsanto as the worst company in America.
So you might imagine my apprehension when, down on my luck and without enough freelance clients to support myself, I went to work for a newly-formed Comcast Xfinity call center in Fort Collins, Colorado.
I expected a repeat of the nightmare above.
Instead, I joined one of the best companies to work for in 2019.
Comcast - A veritable titanic of a company renowned for its “meh” service packages and its awful customer service - went from “worse than Monsanto” to one of INC.’s top 100 companies to work for in the space of just 5 years.
Using NPS, Comcast turned a doomed-to-sink titanic headed for an iceberg, on a dime like it was a 10-person rowboat.
How did Comcast change its reputation? More importantly, what can this teach us about how to run projects, companies, and communities?
TL;DR: Comcast used a 2-question survey called the Net Promoter Score. Implementing it can connect you to your constituents, empower them to speak up, and give you the community feedback you need to make big decisions. Used correctly, the NPS can revive entire communities - and, as in Comcast’s case, brand reputation.
In this blog we're going to discuss how the NPS works, and what makes it so powerful. Then in Part 2 of this 3-part blog on community surveys, we're going to walk you through how you can implement it.
TL;DR: Comcast used a 2-question survey called the Net Promoter Score. Implementing it can connect you to your constituents, empower them to speak up, and give you the community feedback you need to make big decisions. Used correctly, the NPS can revive entire communities - and, as in Comcast’s case, brand reputation.
In this blog we're going to discuss how the NPS works, and what makes it so powerful. Then in Part 2 of this 3-part blog on community surveys, we're going to walk you through how you can implement it.
WHY DOES THE NET PROMOTER SCORE WORK?
The Net Promoter score is a small revision to a common customer service survey question.
It takes this question:
It takes this question:
|
“On a scale of 1-10, how likely are you to recommend us to a friend or family member?”
|
And turns it into this question:
|
“How happy are you with [1. the experience you had], [2. our brand], or [3. working for us]?”
|
The average statistics nerd might see a few key design differences:
- It speaks directly to the person
- It puts emotion before numbers
- It requires 3 different points of contact
- It doesn’t push for more detail
These design changes to the survey are the magic behind the NPS system.
First, the point is not to see if your customers talk to others about your brand: They do. Instead, this question is about how they feel. Removing the “numbers” cuts to the emotional hind-brain, so the NPS uses emoticons so people will flag how they intrinsically feel.
The last two changes are more complex.
The question asked in the NPS has three different options to end it, but that’s not because you’re asking them three separate questions. It’s actually three completely separate surveys.
The Net Promoter Score accounts for the differences between:
Net Promoter Score doesn’t just account for customer feedback one time. It captures sentiment at multiple points along the Customer Value Journey, and it can capture the employees’ sides of the story as well. Collecting data at each of these junctures is the only way you can tease apart the complex feelings a person has about your brand or project.
Let’s go back to the Comcast example.
Contrary to popular belief, people are largely satisfied with Comcast’s internet services right up until the moment they’re not. People hate calling into Comcast for simple things like rebooting a modem. They dread the call, from being put on hold to actually dealing with a representative. Then, the representative they talk to has a large influence on how they feel about the company for days afterwards. Each progressive call mounts emotions on top of emotions.
A customer could get the absolute BEST customer service - but biased interactions might get in the way. If they already loathed calling in because the product broke or their service is spotty and they were expecting a nightmare call, their satisfaction rating likely won’t reflect their overall experience appropriately.
The NPS allows each customer to rate you several times, in different circumstances. Then, it compares that customer’s response to the employee’s. Still, it doesn’t matter how many times a person is surveyed: You’re still reducing a rich, full experience to an abstract, dry number.
So, let’s turn to the importance of that third change to the NPS question: the comment box.
First, the point is not to see if your customers talk to others about your brand: They do. Instead, this question is about how they feel. Removing the “numbers” cuts to the emotional hind-brain, so the NPS uses emoticons so people will flag how they intrinsically feel.
The last two changes are more complex.
The question asked in the NPS has three different options to end it, but that’s not because you’re asking them three separate questions. It’s actually three completely separate surveys.
The Net Promoter Score accounts for the differences between:
- How customers feel about the company overall and over time.
- How customers feel about individual decisions a company makes.
- How customers feel about (the representative) speaking on behalf of the company.
Net Promoter Score doesn’t just account for customer feedback one time. It captures sentiment at multiple points along the Customer Value Journey, and it can capture the employees’ sides of the story as well. Collecting data at each of these junctures is the only way you can tease apart the complex feelings a person has about your brand or project.
Let’s go back to the Comcast example.
Contrary to popular belief, people are largely satisfied with Comcast’s internet services right up until the moment they’re not. People hate calling into Comcast for simple things like rebooting a modem. They dread the call, from being put on hold to actually dealing with a representative. Then, the representative they talk to has a large influence on how they feel about the company for days afterwards. Each progressive call mounts emotions on top of emotions.
A customer could get the absolute BEST customer service - but biased interactions might get in the way. If they already loathed calling in because the product broke or their service is spotty and they were expecting a nightmare call, their satisfaction rating likely won’t reflect their overall experience appropriately.
The NPS allows each customer to rate you several times, in different circumstances. Then, it compares that customer’s response to the employee’s. Still, it doesn’t matter how many times a person is surveyed: You’re still reducing a rich, full experience to an abstract, dry number.
So, let’s turn to the importance of that third change to the NPS question: the comment box.
why is the comments box so important for nps?
Physical NPS stand outside a McDonalds in DIA - Denver
As a marketer and community manager it’s difficult to stress to software developers, DevOps teams, and executives how important qualitative data like social media comments are. So, they’re often ignored as some wasteful customer service thing.
In truth, this data can be hugely helpful - if it’s used correctly.
In previous research by GetApp - "How big data is used on today’s IT teams" - the #1 recommendation for using big data to make business decisions was to implement a qualitative data system.
The NPS cleverly integrates qualitative data collection front and center without asking too much from respondents. Measuring it is another story as we'll discuss in Part 3, but this is a solid step.
By not asking people to elaborate on their responses, in the initial question, NPS doesn’t scare away those who otherwise wouldn't spend a moment taking a survey. It allows people to click the emoji they identify with at the end of the interaction and move on. As a result, you get more - and more natural - responses than just the people willing to take the survey.
At the same time however, the people who truly do have things to say have an unlimited amount of space in a defined box to say it.
This is where companies can find the biggest, most important insights.
In truth, this data can be hugely helpful - if it’s used correctly.
In previous research by GetApp - "How big data is used on today’s IT teams" - the #1 recommendation for using big data to make business decisions was to implement a qualitative data system.
The NPS cleverly integrates qualitative data collection front and center without asking too much from respondents. Measuring it is another story as we'll discuss in Part 3, but this is a solid step.
By not asking people to elaborate on their responses, in the initial question, NPS doesn’t scare away those who otherwise wouldn't spend a moment taking a survey. It allows people to click the emoji they identify with at the end of the interaction and move on. As a result, you get more - and more natural - responses than just the people willing to take the survey.
At the same time however, the people who truly do have things to say have an unlimited amount of space in a defined box to say it.
This is where companies can find the biggest, most important insights.
the value of qualitative data
While I was working as the Marketer-in-Residence at DigitalMarketer.com a customer posted on their facebook group about a concern with DigitalMarketer over-emailing him. His post was longer than this article. Hundreds of people responded in the comments and the community manager let the conversation role. A flood of people who otherwise wouldn't have spoken, did.
This qualitative comment off the back of an email caused so much commotion that we spent 5 hours across the company discussing that Facebook post and the replies it garnered. DigitalMarketer’s email team built an entirely new email marketing deliverable out of that feedback and the company turned faulty email segmentation into one of its greatest successes.
Imagine if that person had access to a system similar to the NPS prior to his comment. Digital Marketer could have gotten that response months before hand.
This qualitative comment off the back of an email caused so much commotion that we spent 5 hours across the company discussing that Facebook post and the replies it garnered. DigitalMarketer’s email team built an entirely new email marketing deliverable out of that feedback and the company turned faulty email segmentation into one of its greatest successes.
Imagine if that person had access to a system similar to the NPS prior to his comment. Digital Marketer could have gotten that response months before hand.
Conclusion:
The NPS is a powerful social listening tool that introduces you to the treasure trove of insights qualitative data can offer to you.
But in truth, the NPS is pretty low-hanging fruit among qualitative analysts. You can install it in a day if you have all your ducks in a row and there’s tons of insights left on the table for you to pick up as you get better. As the old adage goes;
“First get good. Then get great.”
In Part 2 of this blog series, we'll show you how to install an NPS system yourself.
Let us know if you need help getting started, find hang-ups, or have any questions! I’ll watch the comments section below, and if you have something more involved you would like to discuss, you can email me at Samantha@SociallyConstructed.Online.
But in truth, the NPS is pretty low-hanging fruit among qualitative analysts. You can install it in a day if you have all your ducks in a row and there’s tons of insights left on the table for you to pick up as you get better. As the old adage goes;
“First get good. Then get great.”
In Part 2 of this blog series, we'll show you how to install an NPS system yourself.
Let us know if you need help getting started, find hang-ups, or have any questions! I’ll watch the comments section below, and if you have something more involved you would like to discuss, you can email me at Samantha@SociallyConstructed.Online.
RSS Feed
