Module Overview
Conversation testing methodologies, logging and analytics (completion rate, escalation rate), user testing and iteration loops. Students learn to measure and improve bots based on real usage signals.
Learning Objectives
- Design conversation tests and implement analytics tracking for key metrics.
- Run small pilots and analyze conversation logs to prioritize improvements.
- Create iteration cycles based on measured data and user feedback.
Lesson-by-Lesson Breakdown
1
Conversation testing frameworks and synthetic tests.
2
Logging heuristics: what to capture and how to anonymize.
3
Analytics dashboards: essential metrics and KPIs.
4
Running pilots and qualitative user testing.
5
Iteration planning and change rollout.
Hands-on Activities & Deliverables
Activities
Launch a 10-user pilot, collect logs & feedback, produce a prioritized improvement backlog and analytics summary.
📦 Deliverable
Pilot report + prioritized roadmap.
Required Tools & Readings
Analytics templates and test plan examples.
Assessment & Rubric
- Pilot execution & data quality40%
- Analysis insightfulness40%
- Proposed roadmap20%
Prerequisites
Modules 1–5.
👨👩👧
Parent-Friendly Value
Shows parents that bots are tested with real users and improved based on data.