Testing and feedback: Establish tight feedback loops and conduct user tests to gather insights for iteration and optimization. Ethnography and prototype testing can provide valuable insights on user perspectives, such as:
- User lens: Ethnographic research offers the ability to look at a product from the point of view of a user, and not just as a technology solution.
- User experience: Collect feedback on UX, and in the case of conversational AI, how to strike the right conversational tone, number of questions to response ratio, chat lengths, and use of media formats.
- Hooks: A deeper understanding of what would get users to return to the solution, and double down on these value-adding features.
- Edge-cases: Identifying edge-cases and scenarios that have not been accounted for in the design process, which also provide the opportunity to troubleshoot new areas to set up guardrails around.
Build fast and adapt: Adopt an iterative approach to building and scaling genAI solutions, enabling rapid evaluation and the ability to course-correct. Given the rapid evolution of this nascent technology, organizing work into short sprints with lean teams ensures agility and adaptability.
Pillar #2: Rethink solution development in the GenAI era
Traditional approaches to solution design need to evolve from linear to open design principles when working with genAI solutions. In traditional digital solutions, the user experience and customer journey are defined by the product builders in a linear fashion. This logic has been applied to traditional chatbots and makes for a frustrating and limiting experience where the bot cannot handle unstructured requests.
However, common to many generative AI solutions is that their relationship to the user is non-linear. For human interactions (even with a bot) to feel natural, conversations need to be open and free, which challenges the traditional approach to building digital products. This is what conversational AI models like ChatGPT and Bard have succeeded at; they excel at natural interactions by mimicking human conversations and creating a sense of freedom. However, although this newfound freedom poses design challenges for building applications around genAI, genAI systems are still software systems: you still need build an integration and deployment pipeline for the engineering team, you still need to do QA and deploy services to a production environment. GenAI systems still need to be maintained once they are running live in production.
Based on our experience in developing, testing and building genAI-based solutions, here are some key recommendations:
Demonstrate value: Users may not be aware of the full capabilities of your genAI solution. Allow the AI to demonstrate its value by showcasing additional features beyond the user's initial request. These capabilities can be communicated in different formats, such as on the landing pages of the solution (e.g., ChatGPT in OpenAI), through disclaimers around the chat window, or throughout the chat with the AI highlighting, where relevant, its capabilities (as well as its limitations).
Time to value: Maintain user interest by delivering value early in an interaction. Traditional solutions rely on gathering detailed information through user inputs. For instance, when ordering food, an app needs data on location, cuisine, restaurant, specific food items to be ordered, delivery instructions, user details, credit card information etc. In contrast, genAI allows for more natural dialogue. Offer recommendations or useful information upfront and incrementally answer the final request through an interactive conversation. This concept of ‘earning the right to ask for information’ establishes a give-and-take dynamic between the user and the AI, creating a balance in the amount of information shared by and with the user. Doing so will help improve a crucially overlooked statistic; approximately 40% of users drop out of conversations with chatbots after the first text, with a further 25% leaving after the second.
AI-led interaction: GenAI holds the power to shift the interaction dynamic from being simply question and response based to be more proactive, by being AI-driven. Conversational AI models can ask follow-up questions based on predicted behavior patterns or the context of the conversation. This shift in interaction dynamic opens possibilities for deeper customer relationships and engagement, deeper customer understanding enabling businesses to align their offerings with customer objectives. A study of chatbots in the banking industry found that genAI-powered bots had 55% better engagement rates and that 72% of customers rated personalization as highly important.
Relationship and retention: GenAI has the potential to transform the way users interact with businesses and their solutions. Each chat session can be seen as a building block in a relationship, with every new session building upon previous ones to deepen the understanding of a user’s needs. This understanding is a key requirement to enable personalization which builds loyalty and retention in ways that were previously reserved only for personalization-focused tech solutions, such as Spotify and Netflix. Leveraging generative AI's memory capabilities, such as those seen in LangChain, creates a personalized experience, and strengthens information relevance and user loyalty. Increased usage enhances the recommendation engine, creating a higher switching cost for users who do not want to lose benefits accumulated as a result of having a conversation history.
Modular blocks: Conversational chat can be organized around modules and combined in various permutations to create unique user journeys. These conversational blocks may include (but are not limited to):
- User request: This request often initiates the relationship and opens a dialogue with your genAI solution. With this request, a user can take a conversation in any direction.
- Contextual analysis: Once your user’s request has been submitted, it is key for your genAI engine to understand the context and provide an appropriate response, while simultaneously identifying potential information gaps that need to be closed to optimize the response.
- AI-driven response/follow-up: This is your opportunity to showcase your genAI solution’s value. If the user has not provided enough context or information in their request, it is important to demonstrate value and balance the number of follow-ups with the time to value principle. This is also an opportunity to end each message with a call to action or an open-ended question to initiate a new conversation into an adjacent topic.
- User response and/or follow-up: This is where you receive proof that your AI-driven response has delivered (or shown promise to deliver) value to the user, and that the user is interested in continuing to engage with your solution.
- AI-driven unprompted chat initiation: Once you have established a relationship between the user and your conversational AI solution, make the most of your AI’s memory to follow-up and re-engage the user in a meaningful way. The importance is to strike the right balance between usefulness and spam.
- AI-driven closing: When the conversation comes to a natural end, your genAI solution can close the conversation by reminding the user that it is always available and ready to provide support, thereby conditioning the user to return.
The flexibility of modular design allows various permutations, creating a unique journey for every individual user, depending on context, conversation initiation, and topic of interest.