Case Study
A conversational learning experience
I defined and designed a learning experience aimed at introducing its users to AI and teach them about its capabilities and how they could apply them in their day-to-day work. I defined requirements in collaboration with a product manager based on user experience principles related specifically to conversational interactions and designed various concepts including flows and interaction mechanisms to support learning goals. My product partner and I worked with a group of talented software engineers to develop an internal facing live proof of concept for testing.
Designing a conversational learning experience that teaches AI by using AI.
O’Reilly set out to help enterprise customers build practical Generative AI skills through an interactive learning experience rather than traditional, passive instruction. I led product design for a new conversational learning experience that combined guided instruction with hands-on practice, helping non-technical professionals develop confidence using AI tools in realistic scenarios.
Executive stakeholders identified growing demand from enterprise customers who wanted to upskill their workforce on Generative AI.
This initiative began as a stakeholder-driven opportunity. Enterprise customers were looking for practical ways to upskill their workforce on Generative AI, creating demand for an experience that moved beyond theory and enabled learners to build confidence through hands-on practice.
The experience needed to support professionals across Human Resources, Legal, Finance, Operations, Marketing, Customer Service, Design, Manufacturing, Project Management, and executive leadership. Most had little or no prior experience using Generative AI, requiring the product to feel approachable while still teaching transferable skills.
Existing learning formats explained concepts well but offered limited opportunities to practice. I believed learners would build stronger mental models by interacting directly with an AI assistant in a structured environment that mirrored real-world usage.
Why a conversational learning experience?
Instructional design literature shows that an effective learning experience is engaging, relevant to the learner, interactive, provides opportunities for application and promotes active participation.
I believed that a learning experience which mimics the real world interaction users have with GenAI tools (like ChatGPT) would make for a compelling learning experience and it would enable users to build accurate mental models which I believed they could draw from when putting their knowledge into practice;
Working with informed assumptions before user research
Like many early-stage initiatives, the project began before direct user research could be conducted. I worked with Product to synthesize stakeholder knowledge, market context, and domain expertise into a proto-persona that captured our best understanding of the target audience.
Rather than treating these assumptions as facts, they served as design inputs, aligning us around learner motivations, anticipated concerns, and opportunities to reduce barriers to adoption until empirical research could validate or challenge them.
Defining requirements for a chat-based AI-powered conversational learning experience
To identify requirements for conversational interaction, I held a design live exercise (workshop) with my product partner where we collaboratively mapped out the desired conversational flow between an artificially intelligent chatbot and learners.
Together we devised the supporting system behaviors that would create the desired user experience and support our users’ goals.
Uncovering requirements through conversation design
I adapted a design exercise from Strategic UX writing by Torrey Podmajersky. Although this was our first-time defining and designing for a conversational learning experience, the exercise helped us uncover key requirements, dependencies and design implications.
In the exercise, we each took turns assuming the role of learners and the system, then alternated while enacting various conversation scenarios to identify and capture key considerations and experience requirements.
Exploring three distinct interaction models for AI-assisted learning.
Concept 1 treated the AI as both instructor and practice partner.
Learners progressed entirely through conversation while the AI explained concepts, corrected mistakes, and coached better prompting techniques.
The following screen shows the bot responding to an off-topic question; We ensure that it does not respond to or act on requests that are not aligned with the main objective of the learning experience.
This is meant to reinforce the experience’s educational purpose and ensure learners follow and progress through our predetermined conversation flow.
The bot characterizes its own responses, provides explanations and describes ways to phrase (or rephrase) instructions to modify and adjust its output.
In this concept, the AI is both the tool and the instructor. It responds to the messages and provides useful tips to help learners understand how their messages influence its output.
Concept 2 introduced both an AI coach and the course author inside a shared conversation.
This explored whether separating instructional voices could improve learner confidence while maintaining an engaging conversational experience.
More stakeholders responded positively and favorably to this concept and any other concept where an author or a guide was explicitly represented. Stakeholders seemed to resonate better with the method of instruction delivery implied by this concept.
Concept 3 separated instructional guidance from AI interaction using a split-view layout.
The course author delivered structured guidance while the AI became an interactive practice tool learners engaged with at appropriate moments.
Unlike the previous concept, where both the author and AI participated within the conversation, this approach positioned the AI as an on-demand assistant that responded only when prompted by the learner. The author’s guidance was delivered through the surrounding interface, providing instructional context before learners engaged with the AI.
Why I developed this approach
I developed this concept in response to stakeholder feedback regarding the perceived ambiguity of the relationship between the author and the AI coach in the previous design.
Stakeholders also wanted learners to have clear moments of pause and reflection between activities (inflection points) that encouraged them to absorb the material before applying it in practice. Separating the instructional guidance from the conversational experience created a more structured learning flow while preserving the AI’s role as an interactive practice tool.
Stakeholders ultimately preferred this concept over the others, largely due to its split-view layout. Its familiar structure helped distinguish instructional guidance from AI interaction, making the experience feel more intuitive and easier to navigate.
Why do I keep mentioning Stakeholders?
My preferred approach when developing design concepts is to test them directly with its target users. However, in the organizational and cultural environment context of this project, early concept evaluation was primarily driven by stakeholder feedback and alignment.
As a result, concepts needed to resonate with stakeholder expectations as well as satisfy product requirements. For example, stakeholders consistently favored split-view layouts and information-dense interfaces, believing that users felt more confident and could work more efficiently when presented with a greater amount of information on screen. These preferences influenced the direction of the concept that was ultimately selected for further development.
It was challenging to overcome especially as this organization had more of a top-down decision making framework. While I’ve always advocated for user-centric design methods, stakeholders support and buy-in for those approaches weren’t always forthcoming.
A disabled text input field shows a message when learners hover over it, the message states that they must complete the walkthrough before they can begin writing their own prompts.
The intention behind this is to ensure that all learners review the instructional guidance that we’ll prepare to equip them with a foundational understanding of Generative AI and its concepts.
Defining tasks, usage and context scenarios
I defined user and bot conversation flows to support a continuous learning feedback loop. The learning experience was developed iteratively, with concepts continuously refined as new dependencies, user flows, and data requirements emerged throughout the design process.
The conversation flow was mapped visually as a diagram which captures relevant considerations for learner:bot interaction. The following image shows a snapshot of the high-level flow, which my product partner and I closely examined to determine key requirements.
The learning experience was successfully launched and is now actively used by learners as part of O’Reilly’s AI learning offerings.
Following several rounds of stakeholder feedback and iterative refinement, the experience evolved from an MVP concept into a production-ready learning solution.
Throughout development, I partnered closely with Product and Engineering to clarify requirements, refine the interaction model, and provide supporting design deliverables to ensure the experience aligned with both learner needs and technical constraints.
The final product became a key component of O’Reilly’s AI learning strategy, providing non-technical professionals with an interactive, conversational way to build practical generative AI skills through guided practice and iterative feedback 👍
