Your Questions, Answered.
Thanks so much for attending our Webinar: Beyond the Data, User Research for Complex Information Challenges. We were truly honored to share it with so many attendees.
Here’s a link to access the webinar recording, if you’d like to watch it again. Please feel free to share it with your colleagues.
We got some good feedback after the Webinar, and we wanted to share our responses. Below are some questions we received that we wanted to answer:
Collecting and standardizing attribute data
“I would love to know more about the example of all the attributes that were collected that were not standardized enough to be useful for buyers to browse with. How did you change that data or how it was collected to make it successful?”
[SARA’S RESPONSE] This is a great question! Our role in the project we discussed in the webinar was to assess the data structures, workflows and governance processes related to the management of that data, and make recommendations on how to move forward. he unstandardized attributes were only one small aspect of the bigger issues they faced with their product taxonomy.
But, to answer your question, the process we generally follow looks like this:
- Identify the field in the database with the unstructured attributes.
- Create a new taxonomy for that attribute based on the existing values.
- Map all the existing values to the new taxonomy.
- Determine if there are other attributes in this system or other systems that utilize the same domain (i.e. Geography, Audience, Products, Color, etc).
- Rationalize the taxonomy to meet the needs of all the systems (as possible).
- Modify the data entry platform such that the field uses a controlled value list based on the new taxonomy rather than a free text field.
- Correct the existing data through a process of find and replace, essentially, using your mapping. This of course should be done very carefully, and only by the database administrator. Changing your existing data in this way can have huge ramifications for all sorts of other things. What else is that data linked to? Trending reports? Form fields? This needs to be investigated thoroughly, and any resulting side effects should be weighed in terms of the pros and cons.
Research Ethics and Informed Consent
“How do you go about gathering qualitative data on your own customers without them necessarily knowing that you are doing so?”
[SARA’S RESPONSE] My gut reaction to this question is to say simply, you don’t! Informed consent is important. Here’s a decent introductory article on ethics in user research: https://www.interaction-design.org/literature/article/conducting-ethical-user-research
However, here are some things you can do:
- Surveys with free text response fields that allow anonymous responses.
- Naturalistic observation in public spaces. Here’s an article I found that could be helpful on this: https://www.uxmatters.com/mt/archives/2013/11/becoming-a-spy-covert-naturalistic-observation.php#comments
- Under certain circumstances you can conduct a blind study using a third party (e.g. a consulting firm like Factor). Our informed consent, includes (among other things) letting the participant know that we are working for a client, who is in [industry type]. We explain that we are learning about [consumer behavior, etc], and that our results, which may include the session notes and recordings, will be shared with this client. If they find this situation objectionable, it allows them the opportunity to opt out of participating.
Overstepping boundaries?
“If people in sales were out in the field and asked questions to clients, how do you go about conducting those conversations without overstepping boundaries if these customers didn’t necessarily volunteer to be apart of a “study”.
[SARA’S RESPONSE] I feel you. Many organizations aren’t willing to pay for research, so we look for workarounds. Here are some things to consider:
- A sales rep’s job is to sell. Whatever you ask them to do on top of this, be respectful of their time and the added burden it places on them.
- Be clear with the reps about the goals of the research. What exactly are you trying to learn?
- Don’t assume you need to be sneaky—it’s quite likely that the customer is more than happy to provide feedback. Informed consent matters here, too. If the rep is going to pivot the conversation towards research questions, they should let the customer know. If you want the session recorded, the participant needs to approve to that as well.
- Be clear with the rep as well as the customer about how the data will be used. Make sure the rep informs the customer that, while their feedback is appreciated, it may not be possible to make the product improvements they desire.
- Limit the number of questions – maybe five or less. Here’s more advice on that: https://medium.com/user-research/never-ask-what-they-want-3-better-questions-to-ask-in-user-interviews-aeddd2a2101e#.izil93jqf
- Consider how asking about product designs might lead to topics of discussion that the reps aren’t prepared to answer for (e.g. why hasn’t this bug been fixed yet?). It can make them look bad when they don’t know every last thing about the product, and can’t solve all the problems, which could damage their relationship with the customer and hurt their morale. On the other hand, it may provide them with an opportunity to demonstrate that they value their customers’ insight and that the company is actively trying to improve things. And as long as they are prepared for any sticky subjects that come up, it should be fine.
Quant vs. Qual
“The balance of quantitative and qualitative is great! I have things to say when people just talk about quantitative.”
[BRAM’S RESPONSE] We’re happy to find that the message is resonating. Please stay in touch, and let’s continue the conversation!
