New technologies of the past 25 years are facilitating and improving processes of product research and development (R&D). They enhance methodologies and create novel possibilities for testing new products, pre-launch and post-launch. The technology-enhanced research methods (marketing-driven) apply also to products that per se are not technology-embedded (e.g., food & drinks, cosmetics & personal care). Special importance is assigned to testing products among prospect users.

This post will focus on research methods that aim to get the perspective of consumers. Research studies conducted in the course of product R&D span different stages, including reviewing needs or tasks (‘jobs’) consumers seek to perform, generating ideas, measuring preferences (e.g., attributes, functions, intended uses), and product testing at different levels of progress. Different techniques are available at the disposal of researchers in which consumers may participate actively (e.g., surveys, experiments) and through analyses of existing sources of Big Data.

The forms of interaction with consumers are mainly digital (online platforms and mobile applications) through structured and semi-structured questionnaires, behavioural (e.g., in experiments), diaries and other feedback tasks. In more recent years, the introduction of artificial intelligence (AI) and machine learning (ML) created new options with furthermore advanced methods, including virtual models and simulations (e.g., VR and AR), analytics (e.g., learning algorithms), and techniques of generative AI. Most of these methods are relevant to and can be recruited for studies of product testing.

A key factor in product development and innovation is time pressure, that is, firms are increasingly pressed to release products faster (across industries). The challenge it poses calls for several careful considerations: examining the driving factors of the need to release products faster; identifying the strategies for accelerating product development and testing without compromising quality; while balancing at the same time between speed and the importance of thorough testing and validation; and accounting for the potential risks and challenges arising from rapid product release [Greenbook, 1]. A rush to be first in the market can result in product flaws or weaknesses which a follower has an opportunity to ‘fix’ and improve on, and to win over (i.e., eroding the ‘pioneer advantage’).

The impact of technology on product research, and particularly testing, has multiple facets. It applies to both physical and digital products, and to new products as well as modified or improved models of an original product. Technology enables real-time data collection, automated analyses, and wider user reach; the latter is achieved by utilising smartphone apps, AI-powered platforms, and remote testing solutions that aid in eliminating logistical barriers associated with traditional testing methods (e.g., inviting users to companies’ labs or other facilities) [1]. Subsequently, advanced analytic methods can help to glean deeper insights from consumer data, including product testing data. In advanced analytic methods and modelling, we refer to statistical analyses (e.g., generalised linear and non-linear regression models, clustering algorithms) and to AI-enabled methods (e.g., machine learning, language models, visualisation). Since data is a crucial resource for those analytical models, it is of utmost importance to guard the quality of data (using ‘clean’ data) and how it is employed (with care by skilled and dedicated teams). On aspects other than product testing, insights from consumer data can be gained. for instance, in revealing latent needs or discovering product opportunities [Financial Times, 2].

Pre-launch testing is essential for assuring that the product released is safe and sound, of the quality standard planned, and it is performing flawlessly as expected. It is also important for discovering before launch any problems or disturbances prompted by consumers-users testing the product which the developers (e.g., scientists, engineers) did not anticipate. However, the communication channels available today (e.g., company’s website platform, social media) allow for collecting effective feedback even post-launch from product users, being helpful in sorting out remaining issues and making additional improvements to the product. It is particularly relevant for digital products or services installed on computers and mobile devices or delivered online (e.g., cloud-based Software-as-Service, SaaS), where updates can be entered almost continuously. Additionally, more than a few physical products now include digital components and displays (e.g., home appliances, entertainment systems); when connected to the Internet (i.e., as Internet of Things, IoT) they may also be updated remotely.

Much can be done nonetheless in regard to physical products whose production is flexible, modular and more heavily reliant on current consumer data. We should bear in mind that products are increasingly developed for a “segment of one”. The application of this approach is heavily reliant on detailed, specific and timely data at the individual consumer level. For example: (1) adjusting a cosmetic product to the skin tone of the consumer, using image recognition and ML (Fenty Beauty); (2) allowing customers to create their own trainers from a palette of styles, colours and designs (Nike by You — formerly known as NikeID) [2]. The creation of customised products paves the way for on-going feedback from users, and thereof continuous improvement of the selection of customisable features as preferred by consumers.

Manufacturers have been concerned with a challenge of how to let consumers-users engage with and test product items during the development process without creating full-play versions of the product (i.e., in materials and functions) for those tests. Furthermore, it might be better if those tests could be done remotely without transporting the participating consumers to a company’s facilities. First, creating semi or incomplete versions of a product model can help in reducing costs and logistical hurdles (creating and employing ‘complete’ product items for recurring tests may be expensive in some categories). Second, letting consumers use and test a product in a familiar and normal environment of their daily lives would feel more natural, convenient and free of oversight.

Various solutions have already been generated over the years, such as the creation of mock-up and prototype versions of a product in development, which is especially relevant for experimenting and testing in intermediary stages of the process. As the product concept evolves, it may be sufficient to introduce a ‘less-than-perfect’ version of it for testing. Remote digital testing has been around for a while for digital products (e.g., ‘beta’ versions of software), and the approach has been expanded to virtual online service platforms — but more interesting, with new technologies ‘digital’ testing is applied to digital models of physical products. The AI-powered technologies, methods and tools are adopted for even more advanced forms of ‘virtual’ testing (e.g., simulations) and other applications (e.g., analytics).

The increasing prominence of digital product testing is supported by a rise in SaaS (cloud-based) platforms, agile development and user driven innovation. It further led to greater application of A/B testing, remote usability studies, and behavioural analytics of user interactions with online interfaces [1]. The approach of online A/B testing (a basic form of experimentation) is probably more familiar from testing marketing strategies, for example, advert versions, or website menus and interfaces (e.g., e-commerce) — it is thereon being extended to product development, such as testing for variations in materials and design [2]. Perhaps even more influential is customers-users’ participation or collaboration in projects of product innovation (also known as ‘open innovation’): Collaboration Software can be employed for fostering communication and collaborations between teams from different units of a business [2], but it may also help in co-creation initiatives with customers for developing product ideas up-to generating physical working prototypes.

It is now easier and more inviting for companies to let consumers perform in-home testing of their physical products, and the option is also more attractive for many consumers-users. The range of popular products admissible to in-home testing is wide, covering for example household maintenance goods, personal care products, robotic vacuum cleaners, energy-saving laundry machines, kitchen gadgets, and more. Making the living room a laboratory for trying new products allows for testing them in real-life scenarios in the convenience and familiarity of one’s home. It benefits consumers-testers with hands-on experience with new products, and companies with valuable feedback in fine-tuning products at an advanced stage of their development before they land in stores. New technological platforms for distributing products, engaging users, and communicating feedback from testers to the manufacturers streamline this process. The feedback from consumers-testers “drives improvements in design and functionality, influencing product development and innovation”. [3]

• In addition to the opportunity to try new products before they become widely available, consumers may be driven to participating for personal interest or curiosity or be drawn by the benefit of thrill of being among the first to experience innovations [1]. Yet, consumers-users may come with different levels of prior knowledge and proficiency in the product category, and that requires companies to sort and monitor users-testers according to the type and value of contribution they expect to receive from them [cf. 3].

Companies have been moving from producing physical prototype artefacts to creating pure digital prototypes (e.g., enabled by ML, used in digital experiments). Another form of virtual models is used in Digital Twinning, a simulation approach of enacting a replica of the physical world: “a digital twin is a virtual model of a planned or actual real-world product or process”. The simulation is ‘leveled-up’ from “straight digital simulation” since it enables optimised testing, monitoring and integration of facilities (i.e., it creates settings that more fully mimic real-world conditions). Additionally, companies are increasingly applying 3D printing for creating mock-up models of products with different materials (e.g., plastic, clay, ceramic), that are still more similar to end-products in planning than possible before and yet less costly for producing a physical item for real-world testing and demonstration. [2] (Note: The techniques and methods described above do not necessarily involve consumers in the tests.)

Earlier product simulations displayed a drawing or photo-like image of a product in progress on a screen — the image was made ‘live’ or interactive so that consumers in a study could rotate the object (e.g., bicycles) for viewing from different angles, zooming-in to focus on specific parts, and perhaps additional operations. In the framework of the Virtual Customer programme at MIT, Dahan and Hauser [4] proposed six methods based on customer input for assisting the product development process, two of them epitomised well the contribution of customers in the design and testing of product concepts: User Design (UD) and Virtual Concept Testing (VCT), respectively.

• The UD method focused on the selection (adding or removing) of features to a product, showing participants how the visual design (and price) is adapted as the product is modified; the VCT method is intended for a later stage in the process (nearing the end product), where more complete model concepts are presented, in holistic design styles and with ready compositions of features, and participants choose which ones they would buy at different quoted prices.

The advanced Virtual Reality (VR) and Augmented Reality (AR) technologies are applied to create enhanced user experience simulations wherein they enable a much more immersive and realistic experience of engaging with digital-virtual product models, visually and for performing real-like operations or manipulations with the object. They can help developers to identify potential design flaws or unveil usability issues early in the development process [1]. The VR methodology may be especially cost-effective in testing simulations of rich design spaces and operation of more complex equipment (indoors and outdoors, for work or leisure).

Simulations enabled by AI/ML can be used to generate more complex scenarios for testing the functionality and performance of products as in realistic environments or evaluate the receptivity of consumers to the new products (e.g., setting competitive market scenarios). However, companies appear reluctant to adopt AI/ML simulations and tend to prefer the more traditional tested-and-proven methods. Impediments revealed are mainly relative lack in talent/skills, understanding of the benefits, and trust in the AI/ML advanced methodologies [2 — based on a study of McKinsey].

Generative AI is associated primarily with generating ideas, identifying product opportunities and brainstorming. It is advised to deploy the capabilities of Gen AI as a support aid to humans for enhancing these processes, not to rely solely on Gen AI to perform them. Gen AI techniques can used, for instance, to kick-start a conversation, but collaboration between human developers and consumers-users will continue to be very productive and beneficial. Hence, the general thinking for the future should be “Human plus AI”, especially for R&D. Sean Ammirati, professor of entrepreneurship at Carnegie Mellon University (PA), elaborates: “The right mental model is to think of generative AI as a cofounder, or what is often described as ‘Human in the loop of AI’. So think of it more like a brainstroming buddy, more like someone who automates first drafts than does all the work for you” [2].

Digital technologies of recent years, that continue to advance, produce new capabilities and offer enhanced possibilities for assisting companies in the product development process. They entail different options of research and analytics, and particularly different modes of product testing. It remains essential to keep consumers engaged in the process — from generating ideas, through learning about their preferences, and to product testing — and to benefit from their contribution in insights and feedback.

Ron Ventura, Ph.D. (Marketing)

References:

[1] “Faster, Smarter, and More Affordable: How Technology Is Revolutionizing Product Testing”, Ashley Shedlock (Senior Content Coordinator), Greenbook, 5 March 2025

[2] “AI and the R&D Revolution”, Lucy Calback, Financial Times (FT.com), 27 November 2024

[3] “What Is In-Home Product Testing?”, Ashley Shedlock (Senior Content Coordinator), Greenbook, 28 August 2024

[4] The Virtual Customer; Eli Dahan and John R. Hauser, 2002; The Journal of Production Innovation Management, 19, pp. 332-353