Ozon AI assistant
Created a little mate to help sellers increase revenue and reduce the workload on technical support
Meet the team
Problem
Ozon is a large marketplace with plenty of sales opportunities. Sellers are constantly asking for tips on how to boost their sales, and personal account managers and support staff can't keep up with all the requests. Here are a few reasons why we decided to address these issues with an AI assistant: we will provide a single point of access for all help and support information regarding a seller's personal account; we can increase Ozon's GMV through personalized sales recommendations; and during interviews, sellers mentioned several times that they use AI, which is a good reason to move some of these tasks to our interface.
Why AI-assistant?
We already have a production ML model that works with all the data sellers need. It can provide a single point of access for all help and support information regarding a seller's personal account, and increase Ozon's GMV through personalized sales recommendations. During interviews, sellers mentioned several times that they use AI, which is a good reason to move some of these tasks to our interface.
Discovery
Before I began designing the user flow, I studied how modern AI tools are designed so I wouldn't overlook anything important. By the end of my research, I had identified several principles that would serve as the foundation for the entire AI assistant interface:
AI is a teammate, not a tool
AI feels like a helpful assistant that works behind the scenes and supports users without requiring their attention
AI should empower people, not make things more complicated
Users don't need to learn how to use AI. The best projects seamlessly integrate AI, enhancing the experience without adding unnecessary complexity.
Start with the problem, not the technology
It's tempting to create flashy features powered by artificial intelligence, but without a clear user need, they don't work; thoughtful design ensures that the technology serves the user, not the other way around.
Test
AI can be unpredictable, which makes testing extremely important
Competitive analysis
Together with the product manager, we conducted a competitor analysis that included both direct and indirect competitors
Early concepts
Current user experience
Together with the product manager, we conducted a competitor analysis that included both direct and indirect competitors
After the discovery process, I sketched out several rough concepts for the AI assistant to present to the C-level executives.
Main flows
Once the concept was approved, the product manager and I began developing the main workflows for the AI assistant. The main workflows include: proactive recommendations from the AI assistant to users; analysis of the user's sales from the previous day; and answers to questions based on the knowledge base.
What the final design looks like
I have prepared design mockups and prototypes of the AI assistant’s core features to review them with C-level executives and test them with users
Proactively initiating a dialogue with the user
Together with the product manager, we identified the main triggers that will prompt a message from the AI assistant
User Sales Analysis
The LLM analyzes a user's sales based on several parameters, and each one provides sales statistics for the previous day.
AI Assistant's Responses Based on a Knowledge Base
Since the AI assistant has a built-in knowledge base, users can ask it for the information they need
Interaction with support and reviews
I've also kept the option for users to contact technical support and leave feedback about their personal account
Test
I prepared a guide and a set of questions for usability testing to iron out any issues before rolling out the feature to production. Eight users participated in the study.
A/B test & launch
The AI assistant was rolled out gradually, in small waves: a few loyal users, 10% of the platform's users, 30% of the platform's users, 50% of the platform's users, and finally 100% of the platform's users.
What didn't work
One thing that didn't work out based on the release results was the proactive AI bubble, which initiated a conversation with the user. The click-through rate for this bubble was only 1.5%. Through in-depth interviews with users, we realized the reason is that they simply don't look in that corner because of the large number of pop-up notifications.
A/B test of entry points
Since there was already an entry point to the help window in the interface, we decided to conduct an A/B test of entry points to the AI assistant, because I hypothesized that the conversion rate would be higher in the header.
A/B test results
Since there was already an entry point to the help window in the interface, we decided to conduct an A/B test of entry points to the AI assistant, because I hypothesized that the conversion rate would be higher in the header. The A/B test lasted 35 days, and based on the results, the floating button option won.
Results
After several months, the AI assistant produced the following results