Episode 4: AI for UX research 🔍💜

Welcome back to the AI Goodies community! 👋

Research feels like that last area that AI would be well-positioned to replace, but it many ways it is the one it can support best. AI technologies are good for a couple of things, and making sense of large volumes of data is one of them.

But when you think about involving AI in research a couple of questions immediately pop up:

“What are the tools and scenarios that I can confidently employ?”

“Can I trust AI to produce and interpret research data?”

“Is it ethical?”

We’ll be looking into what are the opportunities AI presents us with when it comes to research and how to navigate this tricky problem space in an ethical, safe way.

⚠️ Caution: The Limitations of these tools

We know that AI is still infant technology, so we are mostly aware that it can easily produce inaccurate results. But there’s much more to worry us when it comes to research. A couple of problems:

1. Lack of citations & accuracy of information: When generating quotes and recommendations, some tools fail to reference their sources, being unable to attribute quotes or link them to specific clips. In a recent study by Nielsen Norman the tools they tested couldn’t distinguish between the researcher’s notes and the session transcript, which raises important ethical issues since it's crucial to differentiate between interpretations and participant statements.

   Furthermore, this lack of citation prevents researchers from verifying if the tool’s summaries or recommendations are based on actual research.

2. AI is biased: The biases of AI has can be divided in these categories: systematic, statistical, computational, and human. Read more about Open AI efforts to reduce the biases of ChatGPT.

3. AI lacks context: Most AI-powered research tools don’t ask for, and can’t accept things such as the study goals, insights and tags from previous research, background information about the users, and so on. This lack of a broader context can lead to a multitude of research failures. For example, with no understanding of the research goals, the tool can’t know what is important or relevant to the researcher.

4. Current AI tools offer vague insights and recommendations: The actions proposed by these tools usually are vague and lack clarity, so they’re mostly not actionable.

On top of all these points, there’s also the problem of user research tools over-promising, creating false expectations while delivering often unreliable results.

One of the best article to dive into these limitations is this research by the NN Group.

🔥 Some great AI tools for your research process

The most popular slide from my AI talk! It essentially imagines and E2E design process augmented by AI at stages where that makes sense. Of course, the key lens through which to look at it is that you are still at the center of orchestrating all of this.

If we look at the first line, we can interpret it in the context of discovery and research work, and I want to invite you all to an exercise: think about the activities that go into your research efforts (maybe better to write them down) and then start exploring if and how they might get supported (safely) with the help of AI.

Below I share a couple of use-cases that I’m exploring in detail in my course about AI for Designers.

📊 Use case #1: AI for Market Research & Competitive Analysis

Yes, you’ve read it right! We can use ChatGPT or Gemini to gather preliminary information on consumer needs, trends & key behaviours. (⚠️ Just make sure to fact check all information that it provides).

How do we do that?

Ask the following questions from ChatGPT to get to know your product’s target market better, let’s find out the most common problems they face.

Your product: Productivity mobile app

“I would like to understand the most common problems that people face when it comes to everyday productivity. What are their common struggles?”

Of the results generated above, you could narrow down the topic for more relevant results:

“I would like to learn more about procrastination. Can you expand on this?”

We can also use ChatGPT to help us carry out a Competitive analysis.

Let’s ask:

“Are there any apps currently helping people with the problem of procrastination?”

“What is the most popular productivity app right now?”

🙋‍♀️ Use case #2: AI-powered User Interviews

You can essentially use AI in two ways to support you with user interviews:

• Generate an interview script based on your research goals and questions

• Create AI-generated personas (risky and with important ethical considerations to bear in mind)

Tools to use: ChatGPT, Gemini, Synthetic Users

ChatGPT can support you with a good draft for your user interview guide. If you want to explore “braver” applications of AI around interviews, you can try out a (controversial but interesting!) tool called Synthetic Users.

I personally think it can be an useful tool that can complement nicely real user interviews. Imagine you have a set of insights out of which some have not been sufficiently validated or explored, and are working under time and resource constrains. You can tap into the power of these AI-constructed archetypes, essentially manifestations of your representative user, and have an interesting dialogue with an entity constructed based on the demographic and statistical information you feed it.

⚠️ Caution! These personas may not fully reflect reality. They also don’t cover users with disabilities, with specific edge cases needs, as AI provides the average user persona. Make sure to validate the information provided with real people. Never leave out real user testing.

🌪️ Use case #3: Data processing & synthesis

AI can be very useful in helping you process large sets of data and notice trends & key information. It can also extract learnings from your interview notes & recordings and provide suggested actions.

1. Tableau: Offers data visualisation and business intelligence tools with built-in AI features for data analysis.

2. Microsoft Power BI: Has the ability to sort through data and visualise it to search for insights.

3. H2O.ai: Provides an open-source AI platform for machine learning and AI-driven data analysis.

4. Miro AI: Miro Assist can support you in research synthesis: bring all the data on your board and then use AI to help you process and distill it into the essential points and transform insights into shareable formats.

5. ChatGPT: (or) you can simply paste interview transcripts into ChatGPT and ask it to extract the key insights.

🧐 Use case #4: Usability testing with AI-powered Tools

There are various tools that you can use to predict user performance based on past data, generate predictive heat maps, provide UX audit or help synthesise large sets of data.

1. Attention Insight: This tool predicts where users will look after engaging with your design. Apart from predictive heat maps, it can help you compare designs—you can use alternate versions of the same design or even a competitor’s design to evaluate how well your interface performs.

2. Predict by Neurons: generates predictive heatmaps of where your audience might look at your designs and has very similar features to Attention Insight. The main difference between the two tools lies in the AI’s underlying “training”. While Attention Insight bases its predictions on eye-tracking studies, Predict uses data from eye-tracking and neuroscientific experiments.

3. 3M’s Visual Attention System (VAS): This tool is similar to the above tools and extends to physical environments in addition to digital interfaces.

4. Heurix: A heuristic analysis tool that conducts a UX audit of a website and automatically generates a report highlighting the areas of improvement.

5. UserTesting AI: Helps synthesise large volumes of video, text and behavioural data to identify key themes, patterns and insights. The tool’s sentiment analysis feature focuses on what users say during a testing session. Its friction detection focuses on behavioural cues (what users actually do) to pinpoint the exact places where participants had difficulty interacting with websites or prototypes during tests.

🌟 Final thoughts

These tools can offer a great helping hand for researchers. However, the AI-augmented research process shouldn’t miss the humanity of actually talking to users, seeing their emotions while testing products & discussing how it makes them truly feel.

🔥 Let’s collaborate: Would you like to appear in my newsletter?

Check out my sponsor deck to find out details about how your brand can reach it’s potential with my Newsletter’s audience 👇🏼

✨ Relevant articles

💜 Next on AI Goodies: Episode 5 - Brainstorming with AI

In the next newsletter I will cover the ways in which AI can help us Brainstorm product ideas & solutions for product problems.

Send this Newsletter to a friend who would be interested, link below 👇🏼

✨ For more Goodies, follow me on:

AI & Design Love, Ioana ♥️