www.userinterviews.com
Open in
urlscan Pro
2606:4700:10::6816:1ce2
Public Scan
Submitted URL: https://userinterviews.orpluto.com/api/mailings/click/PMRGSZBCHIYTQNJUGQ2TELBCOVZGYIR2EJUHI5DQOM5C6L3XO53S45LTMVZGS3TUMVZHM2LFO5ZS4...
Effective URL: https://www.userinterviews.com/blog/the-state-of-user-research-2022-roberta-dombrowski
Submission: On April 19 via api from US — Scanned from DE
Effective URL: https://www.userinterviews.com/blog/the-state-of-user-research-2022-roberta-dombrowski
Submission: On April 19 via api from US — Scanned from DE
Form analysis 7 forms found in the DOM
Name: wf-form-Blog-Modal-Form — GET
<form method="get" data-mpformtrack="Lead captured" data-mpformtrack-form-name="Blog Modal Form" name="wf-form-Blog-Modal-Form" data-name="Blog Modal Form" id="wf-form-Blog-Modal-Form" class="blog-modal-form" aria-label="Blog Modal Form"
data-hs-cf-bound="true"><input type="email" class="blog-modal-field w-input" maxlength="256" name="email" data-name="email" placeholder="Enter your email" id="email-8" required=""><input type="submit" id="blogModalSubmitButton" value="Submit"
data-wait="Please wait..." class="button-new teal w-button"></form>Name: wf-form-ROI-Calc-Blog-Modal-Form — GET
<form method="get" data-mpformtrack="Lead captured" data-mpformtrack-form-name="ROI Calc Blog Modal Form" name="wf-form-ROI-Calc-Blog-Modal-Form" data-name="ROI Calc Blog Modal Form" id="wf-form-ROI-Calc-Blog-Modal-Form" class="blog-modal-form"
aria-label="ROI Calc Blog Modal Form" data-hs-cf-bound="true"><input type="email" class="blog-modal-field w-input" maxlength="256" name="email-10" data-name="Email 10" placeholder="Enter your email" id="email-10" required=""><input type="submit"
id="blogModalSubmitButton" value="Submit" data-wait="Please wait..." class="button-new teal w-button"></form>/search
<form action="/search" class="c-search w-clearfix w-form" data-hs-cf-bound="true"><input type="search" class="c-searchinput w-input" maxlength="256" name="query" placeholder="Search…" id="search" required=""><img
src="https://global-uploads.webflow.com/59ace8427353c50001765cbd/5d816bb3831435246bed1c0e_search.svg" alt="" class="c-searchicon"><input type="submit" value="" class="c-searchsubmit w-button"></form>Name: wf-form-Blog-Sidebar-Form — GET
<form method="get" data-mpformtrack="Lead captured" data-mpformtrack-form-name="Blog Sidebar Form" name="wf-form-Blog-Sidebar-Form" data-name="Blog Sidebar Form" data-mpformtrack-position="sidebar" id="wf-form-Blog-Sidebar-Form"
class="c-formemailsubscribe" aria-label="Blog Sidebar Form" data-hs-cf-bound="true"><input type="email" class="c-input w-input" maxlength="256" name="email" data-name="email" aria-label="Enter your email to subscribe" placeholder="Enter your email"
id="email-5" required=""><input type="submit" value=" " data-wait="Please wait..." class="c-buttonsubmit w-button"><img src="https://global-uploads.webflow.com/59ace8427353c50001765cbd/5d8173d66e11c0668a25b7ae_arrow.svg" alt="" class="c-arrow">
</form>Name: wf-form-Blog-Inline-Form---General — GET
<form method="get" data-mpformtrack="Lead captured" data-mpformtrack-form-name="Blog Inline Form - General" name="wf-form-Blog-Inline-Form---General" data-name="Blog Inline Form - General" data-mpformtrack-position="main"
id="wf-form-Blog-Inline-Form---General" class="c-forminputs" aria-label="Blog Inline Form - General" data-hs-cf-bound="true">
<div class="c-formemail"><input type="email" class="c-input c-inputlaunch w-input" maxlength="256" name="email" data-name="email" aria-label="Enter your email to subscribe" placeholder="Enter your email" id="email-7" required=""><input
type="submit" value="Subscribe" data-wait="Please wait..." class="c-buttonprimary c-buttonwhite c-launchcta teal w-button"></div>
</form>Name: wf-form-Blog-Inline-Download-Form — GET
<form method="get" data-mpformtrack="Lead captured" data-mpformtrack-form-name="Blog Inline Download Form" name="wf-form-Blog-Inline-Download-Form" data-name="Blog Inline Download Form" id="wf-form-Blog-Inline-Download-Form"
class="blog-inline-download-form" aria-label="Blog Inline Download Form" data-hs-cf-bound="true"><input type="email" class="blog-inline-form-txt w-input" maxlength="256" name="email" data-name="email" placeholder="Enter your email" id="email-9"
required=""><input type="submit" value="Submit" data-wait="Please wait..." class="button-new teal blog-inline-btn w-button"></form>Name: wf-form-wf-Disco-Form-Modal — GET
<form id="wf-form-wf-Disco-Form-Modal" name="wf-form-wf-Disco-Form-Modal" data-name="wf-Disco-Form-Modal" method="get" data-mpformtrack="Disco Form Submit - Modal" class="cpformentrydiv modal" aria-label="wf-Disco-Form-Modal" data-hs-cf-bound="true">
<div data-w-id="0c52d349-bcb6-a1e3-c0e0-ebb381ac0804" class="modalclose"><img src="https://global-uploads.webflow.com/59ace8427353c50001765cbd/61ccb684961ff8128b59c5be_times-solid.png" loading="lazy" alt="" class="image-47"></div>
<div class="rich-text-block-38 w-richtext">
<h4>What day is best for you?</h4>
</div>
<div class="cpemaildiv"><label for="emailModal" class="eyebrow-text teal table left">Work Email*</label><input type="email" class="cpforminput2 w-input" maxlength="256" name="email" data-name="email" placeholder="" id="emailModal" required="">
</div>
<div class="cpnamediv">
<div class="firstnamenarrow"><label for="FirstNameModal" class="eyebrow-text teal table left">First Name*</label><input type="text" class="cpforminput2 w-input" maxlength="256" name="FirstName" data-name="FirstName" placeholder=""
id="FirstNameModal" required=""></div>
<div class="lastnamenarrow"><label for="LastNameModal" class="eyebrow-text teal table left">Last Name*</label><input type="text" class="cpforminput2 w-input" maxlength="256" name="LastName" data-name="LastName" placeholder="" id="LastNameModal"
required=""></div>
</div>
<div class="cphiddenformfields w-embed w-script"><!-- Disco UTMs -->
<!-- UTM parameters to set on disco -->
<input type="hidden" id="utm_sourceModal" name="utm_source" data-name="utm_source" value="">
<input type="hidden" id="utm_mediumModal" name="utm_medium" data-name="utm_medium" value="">
<input type="hidden" id="utm_campaignModal" name="utm_campaign" data-name="utm_campaign" value="">
<input type="hidden" id="utm_contentModal" name="utm_content" data-name="utm_content" value="">
<input type="hidden" id="url_pathModal" name="url_path" data-name="url_path" value="">
<!-- Javascript to set UTM parameters -->
<script type="text/javascript">
// Get the UTM parameters from the URL
const urlParams = new URLSearchParams(window.location.search);
const utm_source = urlParams.get("utm_source");
const utm_medium = urlParams.get("utm_medium");
const utm_campaign = urlParams.get("utm_campaign");
const utm_content = urlParams.get("utm_content");
// Set the values of the hidden form fields
document.getElementById("utm_sourceModal").value = utm_source;
document.getElementById("utm_mediumModal").value = utm_medium;
document.getElementById("utm_campaignModal").value = utm_campaign;
document.getElementById("utm_contentModal").value = utm_content;
</script>
<!-- Javascript to set URL Path -->
<script type="text/javascript">
window.onload = function() { //when the page is loaded, set the disco URL path field to the visitor URL path
document.getElementById("url_pathModal").setAttribute("value", window.location.pathname); //Set url path
}
</script>
</div>
<div class="daypickerandbutton"><input type="submit" value="Pick a time" data-wait="Please wait..." class="submit-button w-button">
<div class="dayrow2">
<div class="daybox2 daybox0">
<div id="day0dm"><span>Wed<br>19<br>Apr</span></div>
</div>
<div class="daybox2">
<div id="day1dm"><span>Thu<br>20<br>Apr</span></div>
</div>
<div class="daybox2">
<div id="day2dm"><span>Fri<br>21<br>Apr</span></div>
</div>
<div class="daybox2 rightmobile">
<div id="day3dm"><span style="color:#ececec;">Sat<br>22<br>Apr</span></div>
</div>
<div class="daybox2 right">
<div id="day4dm"><span style="color:#ececec;">Sun<br>23<br>Apr</span></div>
</div>
<div class="dayrightarrow2">></div>
</div>
<div class="button-new-discosubmit">Pick a time</div>
</div>
</form>Text Content
x GET FRESH VIEWS—THE UXR NEWSLETTER THAT KEEPS IT, WELL, FRESH! Join over 150k subscribers to get the latest articles, podcast episodes, and data-packed reports—in your inbox, every week. Thanks! See you in your inbox. User Interviews is the fastest, easiest way to recruit research participants. Source from our panel with Recruit or build your own panel with the Research Hub CRM. Sign up freeGet a demo Oops! Something went wrong while submitting the form. SUBSCRIBE TO FRESH VIEWS AND GET A COPY OF THE 2023 ROI REPORT. Join over 150k subscribers to get the latest articles, podcast episodes, and data-packed reports—in your inbox, every week. Thanks! See you in your inbox. User Interviews is the fastest, easiest way to recruit research participants. Source from our panel with Recruit or build your own panel with the Research Hub CRM. Sign up freeGet a demo Oops! Something went wrong while submitting the form. “This is the most relatable email I’ve received, perhaps ever.” – Holly S. Product Our products Recruit Participant recruitment that’s unbelievably fast Research Hub Powerful panel software for scaled user research Features Spotlight Unmoderated Research NEW Recruit participants for usability tests & surveys Security UI = the secure solution for user research Integrations Connect to apps in your research workflow Product Updates See what's new in our release notes Hub API Beta Program NEW Feed live data into Research Hub with the API Talk to users today! Sign Up Free Pricing Company All about UI About Us Read about our story, team, mission, and culture Partners NEW Want to collab? Join our partner ecosystem! Customers We're trusted by research teams big and small Careers We're hiring! Read about our current openings. Spotlight Proudly annoucing our $27.5M Series B! Read about our latest fundraising round Talk to users today! Sign Up Free Resources Learn Blog Fresh UXR content, every week Podcast Tune into Awkward Silences Field Guide An in-depth guide to UXR methods Explore Launch Kits Kick off your next study Incentive Calculator How much should you pay? ROI Calculator Is it worth it? Help Center Need support? We got you. Featured 2022 UXR Tools Map Explore the UXR software landscape 2022 State of User Research Data from our annual survey 2022 User Research Yearbook Meet UXR leaders and change makers Talk to users today! Sign Up Free Participants Participant resources Browse online studies Find the type of study that works for you Learn more about participating Getting started is simple and fast Participant FAQs Need help? Start here Contact participant support Talk to our dedicated participant support team Join our Panel Want to get paid to participate? Sign up free today → Share your opinions, get paid. Sign Up to Participate Sign In Sign Up Sign Up Sign In All posts Announcements Announcements Announcements Announcements Announcements Announcements Announcements Announcements Announcements Announcements Announcements UX Research Topics Research Methods & Deliverables Research Ops & Tools Recruiting Interviewing & Research Skills Leadership & Strategy Profiles & Interviews Culture & Remote Life Podcasts Video Data & Reports Field Guide SUBSCRIBE TO OUR NEWSLETTER Thank you! You are all signed up. Oops! Something went wrong while submitting the form. FOLLOW US Blog > Awkward Silences > The State of User Research 2022 with Roberta Dombrowski Awkward Silences * Last Updated: November 1, 2022 THE STATE OF USER RESEARCH 2022 WITH ROBERTA DOMBROWSKI An insider’s look into how The State of User Research 2022 Report was made and what one creator thinks of the insights. Erin May share on social | The State of User Research 2022 Report unpacks the data from our international survey of UX researchers and people who do research as part of their jobs. This week, we met with Roberta Dombrowski, VP of User Research at User Interviews and one of the creators of the report, to talk about how it was made and what she thinks of the insights. She talked about the rise of buy-in and demand for research, what democratization means for professional researchers, and her predictions for the future of user research. Roberta talked about… * An insider’s look into how this year’s survey was conducted * Signals of growth in User Research, including salaries and team structures * Why it’s a great time to be a user researcher Psst—can't get enough of podcasts? Here are 30+ more of the best UX and User Research podcasts to add to your listening queue. ✴︎ FRESH UXR INSIGHTS IN YOUR INBOX ✴︎ Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. Thank you! Your submission has been received! Oops! Something went wrong while submitting the form. SEE THE ROI OF USER INTERVIEWS FOR YOURSELF Create a free account and start recruiting the participants you need in hours, not weeks. Sign up freeDownload the ROI report HIGHLIGHTS [02:01] An insider’s look into how this year’s State of User Research was conducted [05:19] The rise of organizational buy-in [06:06] Measuring the different facets of impact [08:44] Demand as a signal for impact [10:45] The growing role of research operations [14:11] Reporting and team structures [16:07] What democratization means for professional researchers [18:19] Helping distributed teams limit bias and analyze data [19:57] Why it’s a great time to be a user researcher [29:07] Growing momentum in your practice [30:44] Predictions for the future of user research ABOUT OUR GUEST Roberta Dombrowski is the VP of User Research at User Interviews (and a User Research Yearbook Class of 2022 member—check out her profile here). In her free time, Roberta is an adjunct professor through Boise State University's Organizational Performance and Workplace Learning (OPWL) program and mindfulness teacher. RESOURCES MENTIONED IN THIS EPISODE * The State of User Research (SOUR) 2022 Report * How to Track the Impact of Your UX Research * Kate Towsey on Starting a Research Ops Practice * Distributed, Democratized, Decentralized: Finding a Research Model to Support Your Org TRANSCRIPT [00:00] Roberta: Personally. I have a belief that a lot of the creativity and experimentation in the industry will come from research ops in ways that we can think about solving a lot of the problems that everyone is going through. And that's also like, I have to catch my bias a lot when I'm doing research. If I identify themes from the surveys or even like talking to researchers, that's something I have to gut check all the time. [00:29] Erin: This is Erin May. [00:33] JH: I'm John Henry Forster. And this is Awkward… Silences. [00:43] Erin: Hello, everybody. Welcome back to Awkward Silences. We are here today. We've got Roberta Dombroski, VP of UX research at User Interviews, the biggest company in the world, back for the second time. Welcome back, Roberta. [00:58] Roberta: Thanks for having me again. [01:00] JH: Here to, yeah, we called the State of User Research SOUR, internalizing an acronym, and I'm just seeing a thing right now for this recording that says Roberta SOUR. So Roberta’s actually in a good mood, but it's just a funny combo. [01:11] Erin: We are talking about SOUR today, the state of user research. This is now in its fourth year, which is fun because we get some sort of longitudinal data here in terms of, you know, things we've been asking about for years and have observed sort of anecdotally, but now can quantify somewhat. So we'll dig into that. And some of the changes we've seen over time, and this is Roberta's first year being part of the State of User Research, both to talk about what we've seen as a researcher, but also, as someone who helped put the report and the survey together. So let's just take a moment, not too long, but let's get meta for a minute and just kind of talk about: How did we put the State of User Research 2022 together? And how were you involved? What were you excited to do this year versus maybe being on the other side, having read it in previous years? [02:01] Roberta: You know, this was a very meta experience for me. I have been reading the report for… really since it started. And so I was really excited to dive in this year. Definitely a huge collaboration with the marketing team, Katryna, who's not on the call, really championed and led the whole effort. And so I worked very closely with them on the plan for the survey putting together what outcomes we were trying to get. We wanted to definitely measure benchmarks year over year to see trends. But there were also some areas I wanted to explore that I had heard in qualitative interviews from customers too. So we added some new areas this year, which was exciting. And so yeah, we put together the plan, like any research study. Crafted the survey and it went live and all the work after that came in analysis, and it was a beast. Say it a lot. It was all hands on deck, but it was a really great experience. And seeing it live now is just it's the first time, Erin, you've had like really marketing visuals, like Holly was on it for the first time, too, our graphic designer, and it just looks amazing. And when I saw it, it's just like a proud mom moment, a proud research moment whenever you see findings from a study, so I was really excited. [03:20] Erin: I'm just, I'm wishing maybe we can spin up a userinterviews.com/sour. I think we should probably do that, but for now you can find it, Google it, go to our website, we promoted everywhere. Yeah. Holly did an awesome job. Thanks for mentioning both Holly and Katryna, who definitely definitely did a ton of heavy lifting here. So one of the things you mentioned was you're excited to not just benchmark, baseline, compare across previous years, but also ask some new stuff. So what were some of the things you're excited to add in? [03:50] Roberta: Oh my gosh. So a personal area that I've been wanting to explore is a research impact. There was a blog post that Lizzy, one of our content team members, made a few months ago and I've been obsessing about it. And so I added a section in the survey on impact. How do you measure the impact of your team as a research? I also wanted to dive deeper into tooling. We do the tooling map every year. I've been obsessed with the map and the visuals every year. Internally, we're thinking about integration partners. So I changed some of the sections of how we asked the questions and thought about groupings of different types of tools too, so that we could use that data. And then also that the way that we were talking about it, aligned with how researchers were speaking about it from conversations and calls with them. I also have been thinking a lot. I have this in the back of my head, maturity is coming up as a team of research practices. And I hear a lot about org design like team structures from researchers. So I included a new section around reporting. What team, what function do you report into inside of your organizations? Is it product research? What is it to kind of get like themes there as well? So there were quite a few like new sections this year. But luckily it didn't feel all that different from previous years, which was nice. [05:19] JH: I have a very interesting thing I'm curious to get your take on. There was the question or the section around, you know, the impact of research. And one thing that was surprising to me when I went through this for the first time, was that around a third of researchers report, basically aren’t tracking the impact of the research that they did. Right. But then we've also seen year over year. Is that there's more buy-in and support for research at an organization level than ever before. Right? Like we see that improving year over year. I can't figure out if these are contradictory or complementary, like, is it that just because everyone's bought in, that research is valuable and it's really hard to quantify, like the ROI or something, people just aren't doing it. And there's just kind of like they've made the leap of faith and like that's happening or is there like an actual gap? Yeah. No, we should be closing the loop and like saying that this research was impactful and, and this is the type of decisions or things that had informed. I'm curious how you think about it. [06:06] Roberta: It's funny because I'm coming off of, I do a monthly research leader group, and this week we talked about the impact of research because I've been obsessing about it. I think there is something there where part of the impact is, is it measurable? There's different facets of impact that I'm starting to just have as a practitioner, there is the actual you're influencing decisions around research, which is more quantifiable. You can count the studies. You can count the, oh, we learned this, we changed this in a design, but then there's also the qualitative part. That's like, oh, we changed the culture. Like we have more buy-in for research. It's really hard to measure that it's things that as a leader, you see happening over time and it takes longer too. It takes like two years, at least like it's any time in growing a team. And then there's like the external cultural impact of research, like in society, which is also really hard to measure. And so I think part of it is like, impact is really hard to measure. Depending upon the facet factor of impact. And then also like when you have more buy-in for research, like we have a great team here. I don't need to justify the number of studies that I'm running, like in counting the number of customers. Obviously I want the, as a data point to tell the story, and some of the researchers that we heard from, cause this was like, the impact question was actually free response. So there was a lot of coding data, but a lot of people talked about like, I don't measure it or I do, when I have a research repository, people will engage with it. And so that's impactful. That's a signal that I can use. And so I think people are also trying to figure out what the signal is. Because research is still new in many ways. So the signals will vary based on your context, always. [08:02] JH: Yeah. The product answered everything. That's like the top part needs to interview PMs. They just say, it depends on everything you think will change next year. Do you think it'll go up or down in terms of the number of people reporting? [08:14] Roberta: I hope so. I mean, for me as a practitioner, I hope that like every single researcher I speak to every week, my perspective is changing. And I think the more that we talk about it as practitioners, I think other people will be experimenting. And so it's likely we'll see a change in it next year. I hope we do. [08:33] JH: You think you'll see more measurements next year? [08:35] Roberta: I hope so. It might not be a quantitative measure. It might be more qualitative observations of signals in a structured way. [08:44] Erin: I think the demand signal is really important. Like that's something you see in the data is that. People are demanding more research. And of course, researchers are going to do more research, that's their job is to do research, but people who do research are doing more research. People who do research, and I'm talking about powders, we call them PWDRs, but anyway, they're doing research and they're doing it because they value it. Right. It's not like they have time for it. If it's not valuable, as opposed to researchers where it's like, this is my job. This is what I do, and you're seeing more demand for it. Right. And so that to me is a very clear signal of impact because why would you demand something that takes time if it isn't useful to you, or if you don't perceive it to be useful for you? So that to me is one of the strongest signals to your point. We're still trying to figure out as research teams, like how do we measure impact in a structured way? But that is something that's happening. That I think might account for some of that seeming paradox of like, we don't know how to measure it yet, but it feels like there's more buy-in. It feels like there's more, buy-in people are doing more research. They want more of it. [09:47] JH: Yeah. Yeah. This is a silly way of maybe thinking about it. But the first thing that comes to mind is like, if you think about driving and you're like, what's the value of paying attention while you drive or having your eyes open? It's like, you don't get in an accident and you don't quantify, like, if I had crashed into that thing, you know, I would have had to replace my car in this. Research has kind of that, right? Cause it's like we were going to ship this crappy thing. We did research and we shipped something that's better. And like quantifying that delta every time I think it's just really challenging. And so like, once you get people to believe that like, oh, it's good to drive with your eyes open and pay attention. It's like, oh yeah, it's good to do a lot of research as we're developing this stuff. And that seems to be kinda what you were saying. Erin is like, you can measure it more on the demand side, maybe than the impact on every signal occasion. [10:21] Erin: R and D. It's not D. It's R and D. [10:24] Roberta: Yeah. And I think, like you're saying demand is a signal, I think the growing headcount is also a signal that we're having an impact. Like companies wouldn't be approving, hiring more researchers if they weren't being impactful in some way. And then also we wouldn't see a rise in non-researchers doing research too, if they weren't seeing value in it. Right. [10:45] Erin: Well, let's talk about that. Let's talk about, there's a rise in non-researchers doing research there's a rise in headcount of researchers. All that kind of touches on a couple of related themes, one democratization and two research ops. So let's dig into those, you know, research operations as a field was kind of like that, is that a thing? When I started here four years ago, when we first started the state of user research report. And now it very much is a thing. It's not a thing every organization has someone doing in a dedicated capacity, but it is a discipline, whether it's a dedicated role or not. So what jumped out in the report as far as research ops and democratization to you, Roberta? [11:28] Roberta: Yeah, I think for me, and even speaking to research ops practitioners too, is that it's very rare still. Like it's definitely growing, research ops is becoming more prevalent. It's very rare to have specific roles for research ops. We find that they're in larger organizations and then in smaller sized organizations, it’s always various obviously, but in smaller size organizations, research ops ends up being a component of a researcher's job or a person who does a researcher’s job. Hey, you need to do recruitment and talk to a customer. Like it's just part of the way that you're doing your work and getting value. And then it's really wonderful as companies start to scale, you want to streamline all that work that's getting done. And that's when like the specialization, the specialist role can come in, which is really exciting to see. I personally have a belief that a lot of the creativity and experimentation in the industry will come from research ops. And ways that we can think about solving a lot of the problems that everyone is going through. And that's also like I have to catch my bias a lot when I'm doing research. If I identify themes from the surveys or even like talking to researchers, that's something I have to gut check all the time. [12:46] JH: I'm thinking back on the conversation we had with Kate Towsey from Atlassian a while back where I think the rule of thumb she had around research ops was like, once you hit like eight researchers, ops starts to be valuable. That was her measuring stick. And to your point about, it's not that common yet. I was like, well, it's probably not that many organizations that have eight full-time researchers, but I wonder if something that maybe people haven't figured out the calculus on is like how you account for people who are doing research. They're not dedicated researchers. So if you have four dedicated researchers, but you have 12 PMs and designers doing research, are you in a spot where you actually need ops at that point? Maybe the dust hasn't settled on that thinking yet. [13:20] Roberta: Oh, totally. And even when we were analyzing the data, I was like, oh, I wish I asked this in a different way when we were trying to figure out, like, org structure. Because it is very nuanced. So trying to count that you're, it's really trying to think of the whole system inside of a company and who's doing it and who's running what part of research, the practice, are they doing? And it's hard to really capture that in a few questions or even a conversation with someone. [13:50] JH: And the adjacent part here is around the whole thing of people who do research powders and the democratization or whatever you want to call it. It's kind of linked into this, right with, you know, we have a lot of demand. And so there's more researchers, the research ops becomes more prevalent, but you also have people who are not researchers doing research. And so you have these different models happening. What were some of the takeaways or trends in that area? [14:11] Roberta: It's interesting. So much of my brain is exploding right now. I think one thing that came to mind was team structures and reporting structures too, with all of these different groups of people who are doing research. One thing that we found is that people who do research are really distributed across teams, like it could be product managers, designers. When you get to a point of having research operations, it's typically that person might be reporting into research or design. It really varies by like org structure. And so there's like hybrid models that are actually coming to be where a lot of teams will have like centralized research teams or centralized design teams or product teams, wherever researchers reporting into. But then researchers will also be embedded as well into the context that they're working. And that has everything to do with different facets of how research work is being done when like tooling. How are you quantifying studies, measuring the impact, all of that. And that's also why I wanted to dig into the nuance as much as I could of the team structure, because it influences how the work is being done so much. [15:24] Erin: I just thought it would be interesting to triangulate our SOUR data with our internal app data and see the job titles and the sizes of these different teams. And who's actually on the platform and using it right, because when you respond to a survey, you know what you know about what's happening in your organization, but you don't know what you don't know about. [15:47] Roberta: Next year! That was the stretch goal that I had for SOUR this year. So we'll see if we can get to it, but yeah. [15:53] Erin: Democratization. Research ops. What else jumps out to you when you were looking through either comparing this year's trends versus previous or something that you were kind of interested in on the new side? [16:07] Roberta: So before we did this SOUR report too, I talked a little bit about bias because I'm a researcher researching research. I actually created a document with a massive list of assumptions that I had so that I could compare and contrast what I'm noticing in my work. And then what's actually happening. Like, is this accurate? Is this an observation? And one of the gut feels that I had was we're seeing this rise of people who are doing research. It's likely going to be like evaluative research rather than discovery type research. And that was really cool to get validated on that. That was a theme, like a pattern that did emerge. And the survey this year is that with the rise of democratization, when researchers are doing research, it's typically generative and then other teams, people who are doing research will do more evaluative design, usability study type work, which is really great. I think there's a lot of fear around democratization for researchers. I think about it all the time. And so with that, what that means is that if we do open up, which we've already been doing, open up research to not researchers, it means that we can focus on strategic work. Which is like what everyone as a researcher loves doing. Like, that's why we study our craft so much. So that was really cool to see a pattern that emerged. [17:34] JH: I feel like maybe just because I do this for most things, I can kind of talk myself into different beliefs and points of view. I could almost see that going the other way to some degree. Right. Cause like when you're in the stage and doing usability stuff, that probably means the designer or the PM has like some sort of concept or something that they brought to life that's like relatively firmed up and they're like kind of attached to it and they probably have some developers like ready for work and it feels like they could go out and then really like butcher the research. Cause they're just trying to hear what they want to hear. Whereas, like if you plugged into a professional at that stage, hopefully that would go better. Whereas that's probably still true in the generative, like discoveries I do, but maybe less. So like if the, if the PM is out there, just like hearing pain points and starting to like pattern match, I don't know, but I kind of go back and forth. [18:19] Erin: That's why I know, Roberta, in our recent panel conversation at the advancing research event, we talked about, how can centralized researchers really help these distributed teams? And one of the things that came up was helping teams to say eliminate bias because that's impossible, but to check their bias, acknowledge their bias. And that's where it feels like that could really come in. If, you know, evaluative research is something that people who do research are doing the most stuff that they're finding the most useful. The most sort of like fits into their workflow, like, okay, do it, but write down your doc of assumptions or whatever it is to make the research worthwhile. [18:59] Roberta: Accurate. And it's something that I've been thinking about for our team internally, too, cause we're exploring more evaluative work. And how do you have things like analysis anchors or like quantifiable measures when you're doing evaluative that all ties into the quality, the rigor, putting guard rails on things. I think about this very deeply. Yeah. [19:24] Erin: Well, because that feels like a risk on the generative side. If you have, you know, people not being researchers, people are doing research, doing generative where on the one hand, anyone can find a user with something to say and ask them good questions and learn something, but then taking all of that qualitative data and turning it into signals. Okay. Okay. What should we do with this? Really useful to have someone trained on analyzing qualitative data to help with that part of it? Right. Good time or bad time to be a UX researcher. [19:57] Roberta: Good time. I always tell my team, especially after the Sara report came out, I was like, well, we have job security. We are in demand right now. And it's really cool to see. Like qualitatively, you know, we get the updates, we're seeing articles throughout the year about job postings, but, and even salaries too. It was really cool to see that people are really happy, a really strong signal of people being happy with their roles this year. We also saw strong signal as salary increases, which there's been a lot going on with a great rising nation this year. But yeah, it's really cool to see. The trends that have happened over the past year and how it's surfacing into the research space itself. [20:41] Erin: It looks like over half of folks with five years experience or less are making a hundred, 250,000 salary. Two years ago, just under 25% were making the same. So, you know, definitely reflective of the macro trends, but also seeing that within UX research. [21:00] JH: I dunno if we have the data on the company side stuff there, but I wonder if we can pull that up too, with, as the big companies that grow their teams, if sometimes there's a little bit of a higher salary band. I wonder if they're also as they invest further and further and research helping lift that up. [21:31] Roberta: Yeah, and we did some slices. I don't have it in front of me right now, but in the fuller report, we talk a little bit about it and I believe we're going to do some follow-up reports as well into different data points that we collected. But we looked at role level and salaries. So, differences between director and management. And then I see is, and we also looked at company size since we have that too. [21:37] JH: All right. Quick awkward interruption. It's fun to talk about user research, but you know, what's really fun is doing user research and we want to help you with that. We want to help you so much that we have created a special place. It's called user interviews.com/awkward for you to get your first three participants free. We all know we should be talking to users more. So we went ahead and removed as many barriers as possible. It's going to be easy. It's going to be quick. You're going to love it. So get over there and check it out. [22:06] Erin: And then when you're done with that, go on over to your favorite podcasting app and leave us a review, please. [22:14] JH: I guess the question I have to weave some of these things together is… it's a good time to be a user researcher. You know, companies are bought and salaries are going up, people see the value in it, but then you ask to have all these non-researchers doing research. Maybe just like from the conversations that you have with that community, Roberta, and maybe more anecdotally, like, is that like a good thing? People are liking it. Because there's more awareness across departments. That research is valuable and people want to roll up their sleeves. Is it still a little bit weird of like, there's kind of two camps and a little bit of judginess of like, you're not doing that very well. What's your like, pulse or vibe on that? [22:46] Roberta: Yeah, it's interesting. I'm just coming off of a conversation with John Cuttler yesterday. We actually dug into the different lenses at which research is happening inside of our organizations. I gotta preface, I'm really weird. I've worn every hat. I've been a designer. I've been a product manager and decided to really specialize in research. So I understand all the perspectives and how research is different in each of those contexts. Generally, what I've noticed is that from speaking to researchers, there's fear around things like the quality of research. When we talked about bias earlier, when you start to open the door of, oh, more people can do research, more can go wrong. And I think also there's fear tied to what does that mean for me? I've spent so long perfecting my craft. Many times people go to master's programs to get a PhD and it’s like, we don't learn this overnight. And so. Part of it's fear around, like, what does that mean when we have other people doing research? And that's where I think like, it's happening, we know it's happening. And so as practitioners as a researcher, what does that mean for me? Like we have the agency to figure that out, which is really awesome. Like that's the energizing part. As a leader, I can help figure that out. I can create structures and training to put guard rails, and really think about the nuances of how research is being done on our team in different contexts. And so that's one thing that, you know, that's my own bias too, in my perspective, but that's something that I want to encourage researchers to think about is like, how do we view it from an opportunity standpoint, in an enabling standpoint? Because it should be done. It is a craft and it should be done well, but where is also the flexibility in how it's being done for other contexts too? [24:45] Erin: What do you worry about when you worry about more people doing research? You're talking about kind of, you know, a lot of folks have advanced degrees and a ton of training and it's not something you just come out knowing how, how to do research. Right? It's a skill. So what are the risks? [25:03] Roberta: Yeah. I think for me, what I think about is seeing a signal. What you think is a signal and then confirmation bias. It's not actually a large signal. You're just noticing this to confirm something that you were checking, like you had a gut instinct for us. So that's very scary. I think also the analysis part that we've talked about before too, is depending upon how you're doing research, adapting, if there's different ways to analyze things. I love down and dirty affinity mapping. I prefer it, actually. I'm also someone who's very flexible in my own methods because I come from a different background. So I love that, but I do get scared about, are we quantifying the qualitative insights in the best way? Is this representative of wider signals? Those are probably my personal biggest fears for things. And then I also like—Roberta controlling research. I want to be involved in everything. I love research. That's what I want to do. I want to be with the PMs like nitty gritty and ring out the evaluative studies. I love doing that just as much as I love doing the generative work. It's like a reality check for me too. Like you can't do it all. And I think that's part of it for researchers too, when you democratize, it's like, you actually can't do it all. You can't use research to inform every single decision in the business, if it's just you doing it, but how do we enable other people to use it as well? And then also maybe we'll hire more researchers. So yeah, those are some of my personal things that I struggle with. [25:47] JH: Yeah. Yeah. I think the fear in the other direction is like, I'll put on my, a powder hat of the chronic side. I think a lot of product people don't necessarily want to own, like specializations that are not their wheelhouse. I'm thinking about myself, right? Like I've done design work in a pinch because we were short-staffed or whatever. Right. I find that stuff interesting. And I like learning it, but no, I'm not a true designer. I've got examples. I'm happy to let like a real designer come in and do a much better job. Right. And same thing with research was like, I think this is a problem that benefits from research and I'm happy to be scrappy and do it myself and get some signal that would probably be better than doing nothing. But, you know, having somebody who's a real practitioner come in and do it would certainly be. But I think the fear is that when you, as many in there's going to be like, somehow that's going to slow things down where it's like me scrappily doing the design poorly. I think I can get it done in two weeks and we can move forward. Then somebody comes in and does design full-time and tells me it's gonna take six weeks. We got to meet in the middle here somewhere. And I think that's probably a little bit what people are working through on the research side, too, of like, I want you to do this. I'm sure you're going to do it better than me, but I also can't wait two months. So we got to figure out a way to, to get on the same page. Like the turnaround and the approach and all that sort of stuff. [27:54] Roberta: Yeah, for sure. It's funny. You mentioned that. Cause I was talking to Paolo earlier this week. Who's one of our senior PMs. We, on a recurring basis, I mean like jam on discovery approaches to discovery. And he actually said that. And usually when I'm interacting with a person who does research, like a designer or PM, they'll come to me with something that they're interested in discovering, and then I'll go through like, oh, you could do it this way, this way, this way, this way options to try and mitigate that. Like if I had more time, I could do it this way, but you can do it this way really quickly and cut some corners. And that's where I feel really grateful for my design and product background. And also, I think I'm not sure how other researchers outside of like user interviews, if they're getting down into like here's option ABC. But I could see that being really helpful as they're starting to build out how they're working with other people and like democratization is helping. Yeah. [28:55] Erin: Like that, go find your pragmatic researcher who will help you cut the right corners to get just enough signal. But sometimes it's like a need. Right. And that's how you work together in real life. [29:07] JH: Do you think about a contractor or something? Right. Like, I want some help, like doing my interior design and my living room or whatever. And you reach out to people in your, like, I have a budget of like, I don't know, $2,000. And people are coming back to you and they're like, I'll do it, but it's gonna be $15,000. And we're gonna knock down a wall. You're like, that's just too far away from what I am grounded on. And like, maybe we can do a project together. That's close to my budget. We stretch up to 2,500 whatever. And then next time we'd use something I'm willing to stretch more because I trust you. And I saw like, what a good job, but like, you can't just come in. And I'm like wildly at differences of spectrums. And I feel like sometimes that's what people in these different roles can get themselves into a little bit. [29:39] Roberta: I am a huge fan of, especially when partnerships are new and democratizing research is new and different, like growing into a new team. I'm all about the down and dirty, quick wins. Do we do useability tests? Let's get feedback, let's share it with the team. And then once you have that trust, you can start to expand into the more like, hey, we could do this really cool study with all these methods and triangulate and all these super interesting ways, but we're not gonna be able to get to that if we can't help the team inform a decision quickly sometimes. Yeah. [30:13] JH: There was a great phrase I saw on Twitter the other day from somebody. I'm not gonna be able to attribute it cause I forget where it came from. But, it was describing what you're doing in product, in product development. Just being like, you're trying to just develop like a momentum engine. Like you just want the team to have momentum on solving problems. And I think that goes with how research comes in and how design works. And it's not just like, you need to feel like you're making progress against whatever it is you're trying to deliver. And like momentum is relative. Like how fast you need to be going or what that looks like is going to be situational. But, I thought that was just a really nice way of expressing, like what you're trying to get the team to do together is just like being in rhythm and building momentum. Cool. [30:44] Erin: So first year, doing user research, what are you…. or are you resting and recharging, but what are you thinking about next year for, or what are you curious to see? What might change any predictions? [31:03] Roberta: My predictions. So I definitely see just the number of researchers growing and the impact that we're having on teams continuing to grow. I really envision the people who do research. So. Product managers, product marketers, designers. I see that rising. I don't see that stopping anytime soon. I see salaries continuing to grow too. I think it's because I get obsessed. Like that's what makes me a researcher. I get obsessed with these random things and I just want to keep tearing them apart, but like, I really want to dig into how we're measuring impact. I really want to try and figure out the signals and start to triangulate the signals more of org structure, team models and stuff like that. It's the greater context, like how a team is structured and who they report into that influences happiness. That someone has within their role and how they perform in their job, if they're able to do a specific type of work or that yeah, work structure was the biggest reason for unhappiness. And so, I'm really curious to see. Our org structure is going to shift as democratization shifts who researchers are reporting into on teams. Is that going to change over time? Right now, I report centrally to the CEO. Are we going to see more of that? As research becomes more prevalent because the company has realized the impact. It's not just product decisions or designs that they can influence, but it's sent to her company. I'm a little biased, but that is a trend I would like to see and I think will happen over time. [32:52] Erin: What will grow the most? Between now and next year research shops, researchers, or people who do research as a group, people who do research, I think. [33:03] Roberta: Yeah. We shall see. Yeah, I think so. [33:11] JH: I think so too, but then I'm also like, I feel like just from the people I know, like in product and design rules, across from companies, like they're already doing it. So we might be like a little sash room. I've heard of being tapped out on the side. I don't know. We'll see. [33:24] Roberta: I do think that the number of researchers is gonna continue to grow too. I think it's all gonna grow, but I think those two. [33:33] Erin: Yeah, it could be interesting next year to look at just the amount of time spent doing research among these different categories and the percent of research happening in an Oregon who is doing it. And by doing it, we really have to break that into parts, right? The planning of talking to users, that analysis. Right, because there's so much there. So unpack in terms of how has the research been getting done? How has the research been used? Who's doing it? Who's spending their time on it? Because to JH's point, probably a lot of people. I think we put it in terms of like, are you spending 10% of your time on research that makes you a person who does research? It's not the majority of your time, but it's, you know, a consistent piece of what you do. But there's obviously a lot of range between 10 and a hundred. And you know, maybe what will happen with some of these hybrid models again, predicting, I don't know, but I'm curious to see if researchers are spending more of their time helping and assisting people who do research or people who do research saying, I'm not the expert here. I don't want to spend more than 10% of my time on this, but I very much want to spend 10% of my time on this. So I think that'll be interesting to see how that breaks. [34:46] Roberta: I think we'll also see a rise in research coach roles. The more companies start to democratize because we're even shifting to that. I'm finding my percentage of time is shifting to doing more coaching. And I know it's certain types of companies. Some enterprise teams are hiring specifically research coaches to come in, which is exciting to see. [35:08] JH: So yeah, a couple of things I'm curious about are one, if we'll see organizations try to more explicitly define ratios of like researchers to engineers or other functions, like you see that with like design and stuff sometimes, right? Like for every X engineer, we want a designer and like we're going to budget for that and stuff. I wonder if that will come from research. I'm very curious if we'll see movement between roles, like people who do research, who really like it and have some aptitude for it, move into full-time research or full-time researchers who are embedded and really like the strategy piece, jump over to product or vice versa. I'm curious if we'll be able to pick anything up there. I'm super curious. The general tool, like prevalence of zoom, just being so common because of the pandemic and ever moving remote. Everyone knows how to use it. And we already pay for it. Like if that will maintain or we'll see more specialization creep out. And the last one, this is kind of adjacent, but I think this year, I think somebody is going to crack the repository stuff in a way that really clicks in is like, you know, not like goes viral, but like catches on. Like, I think you just saw that with design systems, like there was a lot of movement and tooling around it and like thought, and then at some point it kind of tipped over. Okay. Everyone is kind of doing design systems this way now in terms of how they manage components with developers, everything, I think we're close to that point on the research outside. [36:19] Erin: Yeah. Well, let's take that further. Cause when I, back to four years ago, I remember JH one of the first articles you shared with me was—remind me his name at, yeah, exactly. The nuggets. And I know there's still that atomic research, the air table, and you know, some of that's definitely still happening, but of course there's blow back to that and all sorts of different views on, well, you can't, decontextualize an insight and it has to be part of a larger sort of study and in context or whatever it is. So, yeah, I'm curious if anyone has. That's JH, are you going to crack the code? [36:58] Roberta: I smiled when you said that, because we're knee deep in repository work. And I was like, ah, it's working. He's thinking about it. [37:07] JH: No, I'm not. I don't think I'm going to be going to do it, but I just think we're close. It just feels like there's been moments that have stuck and resonated with people like that. Kind of like an atomic unit insight thing. There's a lot of tools right now for managing inside. Everyone's kind of capturing their research artifacts digitally or in a way that is storable. And so that's kind of exploded and people are wrestling with it. So it just feels like there's a lot of like the factors kind of there that someone's going to like to put those ingredients together in a way that I think really resonates with folks. But we'll see. [37:37] Erin: I agree. I mean, there's definitely pain there. Opportunity. I don't know what it looks like. I'm thinking of Grain, which I was using earlier today, is just such a delight to use for just pulling clips, which is very different than pulling a high volume of insights across a high volume of projects. Right. Then there's AI. Like how does that fit in? I mean, all we talked about in my last job was, AI this and Mr. That and everything, but it does feel like we're hitting a new wave in web three. We haven't even talked about web three. So let's say another episode, but does AI have a place in when you think about just evaluating this sheer volume of qualitative insights and how humans could possibly ever, at scale, do that. I don't know what the solution is either, but I’m curious. [38:25] Roberta: It makes me think about when we were analyzing the survey data, we had a lot of open response questions and I just put up my hoodie and I went on to coding. So I do think there's a lot of opportunity. I did it in a few days. Cause I was like let's park. [38:48] JH: I was listening to a neurologist on a podcast recently. And he was talking about how people can focus and stuff. And it was, this is just a super random thing, but he did mention that pulling up a hoodie honestly helps. Because it blocks your field of vision. It's like, there's a physiological response of, by narrowing your field of vision, you focus more, so good work. [39:06] Roberta: You should see my setup. When I do analysis, I got everything. Yeah. [39:11] Erin: I think we've got viral potential here. We're going, gonna send us your sick hoodie analysis pics, put them on Instagram, hashtag research. [39:23] Roberta: It'll be good. [39:25] JH: It's state of the union strong. [39:33] Erin: We sit strong somewhere until next year or until next time we'll get you back on sometime sooner. You're listening if you made it this far, and there's a topic you want to hear Roberta talk about, find somehow on Twitter or elsewhere and we'll put it in the mix. [39:52] Roberta: Yeah. Yeah. That's good. Awesome. Thanks. [40:00] Erin: Thanks for listening to Awkward Silences brought to you by User Interviews. [40:05] JH: Theme music by fragile gang. Book a demo Explore Research Hub SCALE YOUR RESEARCH WITH POWERFUL, TARGETED RECRUITING See the ROI of User Interviews for yourself—create an account and start talking to users in hours, not weeks. Sign up free ERIN MAY VP, Growth & Marketing Left brained, right brained. Customer and user advocate. Writer and editor. Lifelong learner. Strong opinions, weakly held. More from this author MORE FROM AWKWARD SILENCES Awkward Silences April 19, 2023 DEFINING A RESEARCH STRATEGY WITH DEVIN HAROLD OF CAPITAL ONE Devin Harold, the Director of Research at Capital One, joins us to share the ins and outs of crafting, implementing, and refining an impactful UX research strategy. Erin May Awkward Silences April 12, 2023 THE POWER OF ATOMIC RESEARCH WITH DANIEL PIDCOCK OF GLEAN.LY Daniel shares how Glean.ly and applying the atomic research framework helps UX researchers organize large amounts of data and efficiently extract insights. Erin May Awkward Silences March 29, 2023 EXPLORING UX RESEARCH CAREER OPTIONS WITH MARIEKE MCCLOSKEY OF LINKEDIN Marieke shares eye-opening insights and details about the experiences that shaped her career as a UX researcher, consultant, and trainer. Erin May Talk to users today. Seriously, we're fast. It's free to sign up! Sign up Solutions Recruit Research Hub Pricing Integrations Roadmap Security User Interviews vs. UserTesting 🌟 Get a Demo Researcher Resources Support Field Guide Blog Podcast Launch Kits Participant Resources Support Browse All Studies Focus Groups Studies Diary Studies Interviews Studies User Tests & Surveys Refer a Participant Company About Us Customers Partnerships Careers © 2023 User Interviews Inc. All rights reserved. Researcher Terms | Participant Terms | Privacy Policy | CA Privacy Notice Cookies Settings WHAT DAY IS BEST FOR YOU? Work Email* First Name* Last Name* Wed 19 Apr Thu 20 Apr Fri 21 Apr Sat 22 Apr Sun 23 Apr > Pick a time Thanks! Loading the calendar... Oops! Something went wrong while submitting the form. By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. Cookies Settings Accept All Cookies PRIVACY PREFERENCE CENTER When you visit any website, it may store or retrieve information on your browser, mostly in the form of cookies. This information might be about you, your preferences or your device and is mostly used to make the site work as you expect it to. The information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies. Click on the different category headings to find out more and change our default settings. However, blocking some types of cookies may impact your experience of the site and the services we are able to offer. Allow All MANAGE CONSENT PREFERENCES ANALYTICS COOKIES Analytics Cookies We use third-party cookies such as Google Analytics, Mixpanel, and Clarity to gather information about website user activities. These analytics cookies help us analyze and improve the performance of the User Interviews website. They also help us understand how you use our products and services, and which types of devices you use to access them. Third-party analytics cookies may use IP addresses, but are generally not tied to other personal information. STRICTLY NECESSARY COOKIES Always Active Strictly Necessary Cookies These cookies are used for the operation of our website and products and services, authentication of visitors, and to help remember your preferences. These cookies may not be disabled and are required for the operation of our User Interviews websites and products and services. FUNCTIONAL COOKIES Functional Cookies These cookies enable our website and products and services to provide enhanced functionality and personalization. They may be set by User Interviews or by third-party providers. If you do not allow these cookies, some services may not function properly. TARGETING COOKIES Targeting Cookies We use third-party cookies from companies like Facebook, Google, and others in order to send you personalized promotions and advertisements through these platforms. If you do not allow these cookies, you will experience less targeted advertising from us. Confirm My Choices Back Button Back PERFORMANCE COOKIES Vendor Search Search Icon Filter Icon Clear Filters Information storage and access Apply Consent Leg.Interest All Consent Allowed Select All Vendors Select All Vendors All Consent Allowed Confirm My Choices