The Conversation Is the Assessment

Think about how a great tutor works. They ask a question. The student answers. Based on that answer, the tutor adjusts — asks a follow-up, introduces a new concept, backs up and tries a different angle, or pushes deeper. There’s no moment where the tutor says “OK, now stop learning — it’s time for a test.” The conversation is the assessment. Every exchange is simultaneously teaching, evaluating, and guiding.

Nobody who has ever learned something 1:1 experienced a separation between learning and assessment. It would be bizarre. Imagine hiring a piano teacher who spent 45 minutes teaching, then handed your kid a written exam about piano. You’d fire that teacher. Yet that’s exactly how school works.

We teach for a few weeks, then stop everything for a test. The test produces a number. The number goes in a gradebook. Then we start teaching again. This isn’t how anyone actually learns. So why do we do it?

Because we had to. When you have one teacher and 25 students, you can’t have 25 simultaneous conversations. You can’t assess each student in real-time while also teaching. The logistics don’t work. So we invented a workaround: batch the teaching, batch the assessment, compare the results. Standardize the questions so grading is efficient. Separate the processes because one person physically cannot do both at once for that many kids.

The entire structure of “teach, then test” is an artifact of scale—an engineering compromise. And like most engineering compromises, we eventually forgot it was a compromise. We started treating it as the correct way to do things. Assessment became its own discipline, with its own theories and frameworks and billion-dollar testing industries — all built on the assumption that learning and evaluation are separate activities. They’re not. They never were.

What changes when you remove the constraint

At Tradewinds, our class sizes are small enough that teachers can actually have ongoing conversations with each student. But even in a class of 8, one teacher can’t be inside every student’s head simultaneously.

That’s where AI changes the equation.

We’re building an open source platform called Rabbithole that wraps around hands-on, real-world learning. The structure is a sandwich: a short AI conversation before an activity, the activity itself with no screens, and a short AI conversation after.

Before an ecosystem field study, a student spends about ten minutes with Rabbithole. The AI is Socratic — it asks questions, builds curiosity, activates prior knowledge. “You said animals depend on plants for food. But what do plants depend on? And what would happen if you removed one piece?” The AI isn’t delivering a lesson. It’s priming the student to notice things.

Then the screens close. The student goes outside, runs the experiment, builds the thing, observes the system firsthand. The real-world activity is the centerpiece.

The activity: no screens, real-world discovery

Afterward, another ten minutes with Rabbithole. This time the conversation is reflective: what happened? What surprised you? What would you change? Because the AI asks questions rather than gives answers, students have to construct meaning from their own experience. And every student processes simultaneously — no waiting for one kid to share while the rest tune out.

After the field study, Rabbithole opens with: “What surprised you most?”

Oliver shares a photo from the field. The AI pushes him to develop a theory.

Running silently behind these conversations is a separate AI process — the observer — that passively analyzes each transcript and records what it sees about the student’s understanding. Not grades. Not scores. Just: what concepts did this student engage with, how deeply did they understand them, and where should we go next? Students reveal what they understood, what confused them, and what connections they’re making — all without a formal test.

The student shares microscope data. The AI finds the pattern. The observer (right sidebar) records mastery in real time.

The information problem

Traditional tests are low-resolution. A student gets 78% on a math test. What does that tell you? They missed some questions. Which ones? Why? Did they not understand the concept, make a careless error, run out of time, or freeze up from anxiety? The number doesn’t say.

A conversation tells you everything. When a student explains their reasoning, the gaps become visible immediately. Better yet — misconceptions become visible. “Student believes heavier objects fall faster” tells a teacher exactly where to intervene and exactly what mental model needs revision. “Got 3 out of 5 wrong on the gravity section” tells them nothing.

The observer is specifically designed to treat misconceptions as gold. A well-articulated wrong answer is more informative than a correct one.

Teacher as conductor

The teacher is amplified — designing the learning experience, choosing which AI persona the student works with, which thinking lens to apply, what topic to focus on. While students work, the teacher monitors every conversation from Rabbithole’s teacher dashboard and can inject what we call “whispers” — private guidance to the AI that the student never sees. Something like “she’s getting frustrated, try a different angle” or “he’s ready for the challenge problem.” The AI weaves that guidance into its next response naturally.

It’s exactly what a lead teacher does when they lean over to an assistant teacher and quietly say “try asking her why.” Except now one teacher can do it for every student simultaneously.

Rabbithole’s teacher dashboard: monitoring every student’s conversation in real time

After each session, the observer suggests “seeds” — things this student should explore next, based on what actually happened in the conversation. A kid fascinated by garden math might get a seed for Fibonacci sequences in sunflower spirals. A kid analyzing game strategies might be ready for the prisoner’s dilemma. The teacher reviews and approves these before they become active. The system observes and suggests; the humans decide where to go.

The standards question

![[rabbithole-teacher-mastery.png]]
The observer tracks mastery by concept — not by grade or standard

One choice we made early: the observer records mastery by concept, not by standard. It writes “Alana demonstrated Analyze-level understanding of Nash equilibria in a game theory context” — not “Alana met standard 4.OA.3.” Nash equilibria don’t appear in any K-8 standard. They’re exactly what Alana was thinking about. We can still generate a standards compliance view when we need one — for parent conferences, accreditation, grant applications. But standards are a secondary lens, not the organizing principle. Common Core is one small, politically-negotiated slice of human knowledge. The real question is: what does this child actually know and care about?

Why this matters for gifted kids especially

Gifted students are often the ones most poorly served by the test-and-move-on model. They ace the test, so the system assumes they’re fine. But “getting the right answer” and “being intellectually challenged” are completely different things. A gifted student can score 100% on a fractions test while being bored out of their mind, and the test will never reveal that.

A conversation provides higher resolution. An AI and a teacher who are engaging with a student’s actual thinking will discover what that student is ready for, what excites them, where they can go deeper. And they can act on it immediately.

There’s a phenomenon in gifted development that’s important here: a kid might be able to Create — invent their own notation, design an original solution — before they can reliably Remember standard terminology. They can Analyze a system they can’t yet Apply procedurally. In traditional assessment, those gaps look like problems. In a continuous conversation, they look like interesting data, not errors.

The point

Education should feel like one long conversation — a tutor who is constantly teaching, constantly assessing, and constantly nudging toward discovery. The fact that we separated these things was always a concession to logistics, not a reflection of how people actually learn.

Small class sizes and AI make it possible to put them back together. Twenty minutes of purposeful AI conversation bookending real-world activity captures more about a student’s understanding than any test ever could. That’s what we’re building with Rabbithole. And we’re making it open source so that any school or any family can deploy it for free.

Rabbithole is a project of the Tradewinds Center for Advanced Education. Tradewinds School in Honolulu is one of our laboratory schools where we’re testing these ideas first — small classes, real kids, real teachers, real feedback loops.

Interested? Here’s how you can help:

• Parents and tutors: If you don’t mind some rough edges and paper cuts, test Rabbithole with your child and give us feedback!
• Teachers, curriculum designers: We’d love to onboard you to try out building lessons on the Rabbithole platform! No coding required, just clear thinking and the ability to articulate your pedagogical methods
• Engineers, product designers, and builders: We’d love for you to join the community of builders working on open source education. Check out our Rabbithole Github repo, run your own copy of Rabbithole, and start hacking! And get in touch if you have ideas!
• Families in Hawaii: We’re currently accepting applications for grades K–5 at Tradewinds School in Kaimukī, Honolulu.

Get in touch: andy@tradewinds.school