Learning Portfolio, LDT 2025-2026
Stanford University. Updated August 2026.
I came to LDT able to build things that helped people and unable to say why. This is the record of the year I spent finding out, told mostly through the times I turned out to be wrong.
I arrived at LDT from software engineering and teaching in roughly equal parts. I completed a B.Eng. with Honors in Computer Engineering at the National University of Singapore, where I taught computer science through much of my degree and then for a year afterward as paid teaching staff, working with more than a thousand undergraduates across seven courses and receiving a Best Teaching Assistant award. Before Stanford I built production systems at Apple, TikTok, Alibaba, and AMD, and worked as a founding engineer at early stage startups.
None of that told me why any of it worked. I could see that certain approaches improved retention and could not articulate the mechanism. I knew spaced repetition mattered and could not say what it was doing. I recognized when a scaffold had created dependency, but only afterward, never during the design review when it would have been useful. I came to LDT to stop operating on intuition, and the specific thing I wanted was the ability to tell productive difficulty apart from bad design before I built the thing rather than after.
Every quarter I thought I had located the real problem. Every quarter the next one showed me the problem was a level below where I had been looking.
| Quarter | The question I thought I was answering | What the quarter actually established |
|---|---|---|
| Fall | How do I help computer science students debug better? | The problem was never about programmers. What looks like a student learning to debug is a system debugging with the student inside it, and the same structure appears wherever a tool does the thinking on a learner's behalf. |
| Winter | How do I make one student's gaps visible to that student? | Calibration is not something that happens inside one head. It happens between people, and it happens through the defaults a platform chooses on behalf of thousands. |
| Spring | Does the design principle survive real learners? | It survives, and it works least well for exactly the students who need it most. |
| Summer | Is the report finished? | Precision about a system is a design instrument. The claim I could not defend was pointing at a feature I had never actually built. |
learnica.ai. Advised by Professor Hari Subramonyam. Supported by three Stanford grants. In live use by about one hundred students.
Students who rely on passive study strategies feel productive while they work and do not build durable understanding. They reread slides, they highlight, and they treat having read an AI summary as evidence of having understood what it says. Generative AI sharpens this because it performs the selecting, organizing, and rephrasing on the student's behalf, so what arrives on the screen is the artifact of someone else's thinking. Bjork and Bjork call the underlying issue the gap between performance and learning. Koriat and Bjork call it the illusion of competence.
I interviewed seventy six learners across four quarters. The same pattern appeared in almost all of them, and one participant put it more precisely than any of my own writing has since.
ChatGPT lets me feel smart without being smart. This makes me feel dumb but then I actually get it.
Computer science undergraduate, November 2025, immediately after scoring two out of five on a quiz covering material he had told me he already knew.
Note taking remains one of the better vehicles for learning, and its value spans before, during, and after class. Most AI note takers flatten that entire lifecycle into a single polished summary delivered at the end. Learnica keeps the lifecycle and moves the AI to second position within it. The system removes the effort that only costs time, such as racing to transcribe, and protects the effort that builds understanding, such as deciding what matters and finding out what you cannot yet explain.
The rule I have used on every decision since spring came from my advisor. Be certain about which struggles in note taking are productive, preserve exactly those, and use technology to remove the rest. That sentence is a filter rather than a slogan. When a student told me the system felt slow, it forced the question of whether the slowness was unnecessary friction or the thinking the system exists to protect. Sometimes it was the first and I removed it. Sometimes it was the second and I left it alone.
The argument underneath all of this was worked out in my first quarter, on a completely different problem. For my final paper in EDUC 333A I analyzed why students who debug successfully alongside an instructor cannot debug alone, and the answer that held up was that the cognitive work of debugging is distributed across a system containing the instructor, the environment, and the student. The system debugs. The student participates. We then observe the system succeeding and credit the student, which is a mistake about where the work happened rather than a mistake about how much the student practiced.
The criterion that turns this from an observation into a design rule comes from Pea.
If the support does not fade, then one should consider the activity to be distributed intelligence, not scaffolded achievement.
Pea (2004), pages 431 and 432. The full analysis is in the artifact below.
Apply that criterion to an AI note taker that writes the notes every time and the verdict is immediate. The support never fades, so it is not a scaffold. It is a permanent component of a system that takes notes, and the student is not learning to take notes but learning to participate in something that does. That is the same structure as the debugging case with the instructor replaced by a model, which is why the design question stopped being how good the notes are and became when the support is allowed to arrive and what the student has to have done first.
In the first pilot the system was supposed to require a student to attempt her own synthesis before the AI summary became visible. One participant typed a single period, pressed generate, and waited. The interface accepted it. One character counted as having written something. I had spent months protecting a moment of thinking and had left a three second path around it.
The gate that replaced it judges whether an attempt was made and never whether the attempt is correct. A short, messy, or plainly mistaken sentence passes in exactly the same way as a good one, because the requirement is effort rather than accuracy. Its shape matters more than its thresholds.
attempt_is_real(entry) :=
time_on_task >= floor
AND own_characters >= minimum beyond the provided skeleton
AND judge(entry) == "this reads like a thought, not a placeholder"
judge() is deliberately lenient, is bounded by a short timeout,
and fails OPEN. A student is never locked out by a slow model.
Only a passing verdict is cached. A refusal is never remembered,
so the next attempt is judged on its own.Failing open is a real concession and I would rather state it than hide it. It means the gate can be defeated by a patient student, and it means the gate is a design for the ordinary case rather than an adversarial one. The alternative, which is locking a student out of their own lecture notes because a model was slow, is worse than the failure it prevents.
When a student asks the assistant to explain something, it replies once with a question asking for the student's current understanding in a sentence, and it says that a guess is fine. Only then does it explain, and it begins by naming what the student got right. It asks at most once for a given topic and it never refuses an explanation. The design is an application of the assistance dilemma described by Koedinger and Aleven, which is that help given too early removes the work that produces the learning, and help withheld too long produces nothing but frustration.
Nine students used the rebuilt system in the first pilot. The behavioral data contained one finding I could not have predicted from the literature. The students who engaged most deeply with the review panel were not the ones who struggled with the material. They were the ones who already suspected they might be fooling themselves. The genuinely overconfident students glanced at the panel and moved on.
Koriat and Bjork assume that making a gap visible will change behavior. Visibility turns out not to be enough when a student does not believe the gap applies to them. That is the open problem this project leaves me with and it is a harder one than the one I started with.
The honest boundary of the current evidence is worth stating plainly, because the easy version of this project would present instrumentation as proof of learning. It is not. The system logs what each student wrote at each point where the design asks for effort, and whether each prompt was declined. A failed write is silent by design, so the log understates activity rather than overstating it. No claim about learning is made from the instrumentation on its own. The study that would support such a claim is the work of the coming quarter.
The challenge. Learners systematically misjudge what they know, relying on surface fluency rather than understanding. A word in large font feels more memorable. A concept just reviewed feels mastered. I wanted to test whether a simple and scalable intervention could move students from passive re-exposure toward targeted, confusion driven review.
What I made. A between subjects experiment. Ten participants watched the same video on octopus neurobiology. The template group received a document with sections headed Important, Confused, and Other Thoughts. The control group received a blank document. Everyone had five minutes to review, then took a seven question quiz.
What I learned. The quiz scores barely separated, at 5.8 against 5.2, and with ten participants that difference carries almost no weight. The behavior separated sharply. Three of five template users returned to the specific segment that had confused them. Four of five control users rewatched the video from the beginning, not because they were confused about everything but because they could not remember what had confused them. The template did not make anyone smarter. It gave them a map of their own confusion, and that distinction between knowing you are confused and knowing what you are confused about became the thread of my whole winter.
The challenge. A student explains a division strategy and the teacher responds that this is where you are going off again. In three seconds the reasoning disappears. We found this pattern hundreds of times across sixteen hundred elementary mathematics lessons in the NCTE corpus. The problem is not awareness, because teachers know abrupt dismissals shut students down. The problem is that knowing a pattern is harmful and knowing what to do instead are two separate things, and teachers almost never get to rehearse alternative moves outside live instruction.
What I made. I led the generation pipeline, the model evaluation, and the build and deployment of the web application. The pipeline filters lessons by observation score, narrows candidates through keyword extraction and a heuristic shutdown scorer, and produces 413 high precision exchanges at 86 percent precision. For each exchange the system produces four alternative teacher responses grounded in Dialogic Teaching, Productive Failure, Uptake, and Pressing for Reasoning. I evaluated three models against 46 human annotated gold examples using cosine similarity and Maximum Mean Discrepancy. The tool implements a contrasting cases practice cycle and runs its semantic similarity entirely in the browser.
What I learned. The same three words mean entirely different things depending on what follows them. The word nope followed by calling on another student is a shutdown. The word nope followed by asking where that seventy two came from is an invitation. The lexical analysis made this concrete, because shutdown language is dominated by terminal evaluative terms while the alternatives replace them with process verbs such as walk, explain, try, and tell. Our teacher collaborator in Colombia taught me the other lesson by reading our similarity score as a grade. Any number on a screen will be read as judgment whatever the label says.
Best Generative Research Prize and Best Overall Winner Prize.
The challenge. Motivated language learners often know the grammar and freeze when they have to speak, because speaking is public, time pressured, and emotionally risky. A learner can read a sentence in a workbook and still panic while checking into a hotel. We wanted to know whether an AI conversation partner creates a rehearsal space safe enough to practice in, and what would bring a learner back to it a second time.
What we made. A mobile application, built by our team over the quarter and carried through the entire arc rather than any single stage of it. Competitive and literature review, then generative research with real learners, then affinity analysis, then paper prototypes, then usability sessions, then a working build, then instrumentation, and finally a live fourteen day field study with people who were not us. Sixty two distinct people used it inside the study window.
What I did. I led the quantitative half. I built the event dashboards, computed the onboarding funnel at 40.32 percent completion, ran the retention comparison between deeply engaged and light use groups, and produced the feature usage breakdowns. Alongside that the team ran fifteen contextual inquiry participants across nine in person sessions, with follow up interviews and a check in survey, and grouped the observations through affinity analysis.
What I learned. The most useful lesson was about the limits of my own instruments, on a product I had helped build and therefore could not blame. Deeply engaged users returned at 61.1 percent on day one against 34.2 percent for light use users, and that advantage did not survive to day five or day ten. With groups that small, one person returning moves the number substantially, which taught me some humility about reading instrumentation at all. The larger finding was the gap between reaching a feature and committing to it. People arrived at our learning surfaces and stopped, and no amount of event data explained why. The interviews did, and the reason was that they wanted clearer signals about cost and trust before they committed. That pattern maps directly onto my own capstone, where students reach the review panel and do not always act on it, and it is a design problem rather than a motivation problem.
Final paper, individual. Full marks. The instructor described it as one of the standouts of the class and as an exemplary use of distributed cognition.
The challenge. When I taught data structures, students followed my live coding attentively, typed what I typed, answered correctly when I asked what the stopping condition was, and left with working code. A week later many of them could not begin a similar problem alone. One of the strongest students in the room said the thing that became the paper.
During your live coding, everything makes perfect sense. I can predict what you will type next. But when I am alone, I realize I was following your thinking, not developing my own.
The standard explanations for this are individual deficits, meaning insufficient practice or attention or aptitude, and the standard interventions follow from them. More practice. More worked examples. Explicit transfer strategies. None of them close the gap, and I had tried all three.
What I made. A five page analysis arguing that the gap is not a transfer failure at all. Following Hutchins, I took the unit of analysis to be the whole system rather than the student, and asked which component performs each cognitive function during a guided debugging session. Error detection, error interpretation, hypothesis generation, strategy selection, and progress monitoring are all performed by the instructor. The student types and observes. The system debugs successfully, and we then attribute that success to the student. I called this a learning attribution error, borrowing from Weiner on the locus of attribution. Hutchins makes the parallel exact in the cockpit case, where speed bugs do not help pilots remember speeds but are part of how the cockpit system remembers speeds. Pea supplies the design criterion, which is that support which never fades is distributed intelligence rather than scaffolding, and he raises the Clever Hans problem, where a correct answer is produced by cue following rather than by competence.
What I learned. Removing the instructor does not give a student the same task without help. It gives them a different task, one whose constituent functions they have never performed. That is why more practice cannot work. The student is practicing followership rather than debugging. The consequence for design is that learning outcomes have to name the individual cognitive functions rather than the holistic behavior, and instruction has to transfer them one at a time with deliberate fading.
I wrote this about debugging in December. Every claim in it survives replacing the instructor with a language model and the debugging session with a set of notes, which is what my capstone has been about ever since.
Group learning environment analysis.
The challenge. To analyze how virtual social platforms support language learning, and specifically why some designs sustain engagement while others with sound pedagogy do not.
What I made. A comparative analysis of three environments through Communities of Practice, Distributed Cognition, and Situated Learning. A static vocabulary room, a conversation based exchange, and a game based practice space.
What I learned. What looked like an early evolutionary stage turned out to be an unavoidable trade off. The social space creates authentic motivation and lacks structured progression. The vocabulary room provides cognitive artifacts and never adapts, despite having the infrastructure to do so. Adding tracking to the social space could diminish the authenticity that makes it work at all. This reframed how I think about design from resolving trade offs to helping learners move intelligently between environments optimized for different things, which is a posture I have used in my own project ever since.
| Quarter | Courses |
|---|---|
| Fall Finding the problem |
EDUC 229A LDT Seminar EDUC 333A Introduction to Learning Sciences EDUC 391 Engineering Education and Online Learning EDUC 200A Introduction to Data Analysis and Interpretation EDUC 215 Education Internship Workshop |
| Winter Testing the principle |
EDUC 229B LDT Seminar EDUC 398 Core Mechanics for Learning EDUC 315A Introduction to Computer Supported Collaborative Learning EDUC 473 Empowering Educators via Language Technology EDUC 475 Entrepreneurship in Education CS 224N Natural Language Processing with Deep Learning EDUC 215 Education Internship Workshop |
| Spring Building and testing |
EDUC 229C LDT Seminar CS 377U Understanding Users CS 153 Frontier Systems CS 25 Transformers United EE 292I Insanely Great Products |
| Summer Sharing out |
EDUC 229D LDT Seminar EDUC 215 Education Internship Workshop |
October 2025 to present, with Professor Chris Piech. codeinplace.stanford.edu
Code in Place teaches introductory programming to tens of thousands of learners through thousands of volunteer section leaders. I built a progress summarization system that gives instructors daily per student and per section summaries, an evaluation pipeline for volunteer instructor applications, and a time decay ranking for the course forum so that genuinely good posts stay visible rather than being buried by newer ones. I also helped prepare section leader training and reviewed hundreds of applications.
The shift that mattered was from asking whether a feature works to asking whether it helps people teach better. Reviewing applications forced me to articulate what signals predict a good volunteer teacher. Writing training materials meant encoding assumptions about good teaching into documents that many people would follow without ever meeting me. Every default setting and every ranking function encodes a claim about what good teaching looks like, which is another way of saying that design is pedagogy.
February to July 2026, with Professor Diyi Yang.
I built a video to training data pipeline for social norm discovery, using two stage annotation to prevent label leakage together with automated prompt optimization, scaling the dataset from 193 examples to more than two thousand, with enforced output schemas and per source stratified evaluation.
Under MIT CSAIL and the App Inventor Foundation.
Building integrations between App Inventor and learning management platforms such as Google Classroom. Worth noting for any international student reading this. Open source contribution programs count as valid experience in this field, and they are not on the standard internship pipeline where most people look.
Zikun (Zayden) Zhu. Learning Design and Technology, Stanford University.