Case Study

Enhancing learning with AI-powered answers

tl;dr

Improving the usability of an AI-powered question-and-answer experience embedded within O'Reilly's online book reader.

I Introduced AI-powered answers without disrupting the reading experience.

I joined the project after the AI team had proven the technical feasibility of delivering real-time answers inside the book reader. My role was to remove launch blockers by improving the layout, interaction design, readability, and overall usability before release.

The pre-redesign state of the book reader with the answer’s panel open.

Evaluated the prototype against usability heuristics.

Given the project’s stage, a heuristic evaluation was the most practical way to identify design improvements. I focused on the principles most relevant to the experience and used them to prioritize changes.

Annotated screenshot of book UI with answers panel capturing key issues.

Improved readability and visual hierarchy.

I reduced the answer panel from one-half to one-third of the screen, replaced the typeface with one optimized for reading, improved spacing, and restructured long responses into shorter paragraphs so the book remained the primary focus.

The AI assistant appears as a familiar side panel alongside the platform’s other learning tools. The feature is introduced through a task-oriented label (“Ask a question”) that communicates what learners can do rather than what the underlying technology is.

Simplified system feedback.

Instead of exposing technical processing steps while answers were generated, I replaced the loading experience with an indeterminate skeleton state that clearly communicated progress while reducing cognitive load.

I replaced traditional loading indicators with a lightweight skeleton state that immediately communicates progress while reducing perceived wait times and setting expectations as AI-generated responses are retrieved.

Aligned the experience with the design system.

I updated the answers panel, referenced sources, and follow-up questions to match the platform’s current design system, creating a more cohesive reading experience.

Related books and follow-up questions are organized into a clearer hierarchy that supports exploration without distracting from the primary task.

I refined the visual hierarchy of referenced sources by reducing unnecessary emphasis, improving alignment, and prioritizing the information learners use to evaluate a source at a glance. The result is a cleaner, more balanced presentation that better supports quick scanning.

I refined the suggested question cards to improve readability and scannability. Increasing the type size, reducing font weight, and introducing more generous spacing made questions easier to read at a glance while encouraging exploration.

Focusing the first release on stakeholder goals

After discussing the project’s history with the Product Manager, we agreed that addressing stakeholder concerns while protecting the user experience was the best strategy for a successful initial release.

When stakeholders described the affordance as ‘too subtle,’ I investigated the underlying concern rather than immediately producing alternatives. The real objective was increasing discoverability and adoption.

During design discussions, we explored placing additional functionality in the unused space beside the reading content. I ultimately recommended against it, recognizing that introducing competing elements would interrupt reading flow and reduce immersion.

Balancing discoverability with reading flow

Rather than introducing a visually dominant button, I proposed displaying the panel by default for targeted users while allowing it to be dismissed and restored intuitively. This achieved stakeholder goals without disrupting immersion.

The speech bubble icon was chosen because it already carries a strong association with asking questions and receiving answers, reducing the cognitive effort required to interpret its purpose.

I shared the design with the Product Design team to gather feedback, ensure consistency, and create awareness of how the new capability fit within the broader platform.

Peer design critiques helped validate assumptions and refine the experience. Each piece of feedback was evaluated against user goals, discoverability, and consistency before influencing the final design.

Supported engineering through implementation

After approval, I created a high-fidelity prototype that served as engineering’s source of truth. I attended stand-ups, answered implementation questions, clarified design intent, and performed quality assurance throughout development.

Every element follows a consistent spacing system that establishes a predictable visual rhythm throughout the interface. Standardized spacing improves readability, strengthens hierarchy, and creates a more cohesive learning experience.

The project gradually evolved beyond individual design decisions into a set of product principles. These principles helped align product, design, and engineering around a shared understanding of what constituted a high-quality learning experience.

The final design differed only slightly from the initial concept because aligning stakeholders on goals proved more valuable than repeatedly redesigning the interface.

Identified future research opportunities

The feature demonstrated technical feasibility, but future research should validate assumptions about learner behavior, AI usefulness, answer quality, and whether users prefer embedded assistance over external resources.

Assumptions on its utility and desirability

There’s the overall assumption that learners often have questions while reading a book, indicating that there’s a natural inclination for users to seek additional information or clarification during their reading.

We assume that users would find it convenient or would prefer to access answers within the same environment they are reading, rather than using an external source (like Google, Quora or Stack exchange – sources which many users already use and trust)

We also assume that learners are motivated to seek answers to their questions as part of their learning process while reading a book, indicating a strong motivation for self-directed learning.

Assumptions on technological feasibility

There was an assumption that the 2024 state of Artificial intelligence and Machine learning technology made it feasible to provide real-time answers to a wide range of questions related to various books and topics.

There’s also the presupposition that real-time answers provided by such a system would be accurate, reliable, and relevant to users‘ questions and that there exists a source of information that the system can reference (or be trained with) to generate real-time answers.

Next Case