Nielsen Norman Group Explains Research Adaptation To AI Development
Research must adapt to AI-driven development speed, according to Nielsen Norman Group. This change enables researchers to provide value to product development in new ways.
Original reporting · nngroup.com
Original: “Research-In-The-Loop: How UXR Adapts to Development at AI Speed”
Read “Research-In-The-Loop: How UXR Adapts to Development at AI Speed” at Nielsen Norman Group →
How this was written. GDS wrote this article with a language model from Nielsen Norman Group’s reporting (model: @cf/meta/llama-3.3-70b-instruct-fp8-fast); it describes that reporting rather than adding to it. The original, linked above, is the full story.
Key points from Nielsen Norman Group’s report
- AI-driven development is changing the way software is built and released, requiring research to adapt to faster development cycles.
- Research-in-the-loop is an approach that integrates UX research into AI-assisted software development, using AI's ability to build evaluation mechanisms alongside product features.
- This approach enables researchers to provide valuable insights to inform development decisions and improve product outcomes, despite the rapid development cycle.
The increasing use of AI in software development is transforming the way products are built and released. Nielsen Norman Group reported that this shift is challenging traditional research methods, which often cannot keep pace with the rapid development cycle.
In the past, software development was a costly and time-consuming process, with significant resources invested in each new feature. However, with the advent of AI coding agents, the cost of building and changing software has decreased dramatically, allowing for faster iteration and release.
As a result, many teams now adopt a 'ship first, ask questions later' approach, which can lead to a disconnect between development and research. Researchers can no longer conduct lengthy studies before a product is released, and instead must find new ways to provide valuable insights to inform development decisions.
According to Nielsen Norman Group, research can still play a crucial role in AI-driven development by defining what each release should measure and what it can help teams discover. This approach, known as 'research-in-the-loop,' involves integrating UX research into the development process, using AI's ability to build evaluation mechanisms alongside product features.
Research-in-the-loop enables researchers to shape what a release should capture, implement data tracking alongside the feature, and study the results after release. This approach supports different types of research activities, including evaluation, investigation, and discovery, which can be used to inform development decisions and improve product outcomes.
To implement research-in-the-loop, teams can specify research requirements alongside product features, using a single specification that coding agents can build from. This allows researchers to use resulting data to inform design improvements and continue running other types of research as needed.
Nielsen Norman Group suggests that building research instrumentation alongside product features is a promising starting point for research-in-the-loop. By adding a research section to the requirements specification, teams can ensure that research is integrated into the development process from the outset.
Questions and answers
What is research-in-the-loop?
Research-in-the-loop is an approach that integrates UX research into AI-assisted software development, using AI's ability to build evaluation mechanisms alongside product features.
How can research-in-the-loop be implemented?
Teams can specify research requirements alongside product features, using a single specification that coding agents can build from, and add a research section to the requirements specification.
What are the benefits of research-in-the-loop?
Research-in-the-loop enables researchers to provide valuable insights to inform development decisions and improve product outcomes, despite the rapid development cycle.
.png)


