Get stories like this delivered straight to your inbox. Sign up for The 74 Newsletter
Class Disrupted is an education podcast featuring author Michael Horn and Futre’s Diane Tavenner in conversation with educators, school leaders, students and other members of school communities as they investigate the challenges facing the education system in the aftermath of the pandemic — and where we should go from here. Find every episode by bookmarking our Class Disrupted page or subscribing on Apple Podcasts, Google Play or Stitcher.
MagicSchool has emerged as a breakout artificial intelligence tool in education, with millions of teachers rapidly adopting it. But what’s behind that growth? How exactly does it use AI? And what does its adoption mean for teaching and learning? Its founder, Adeel Khan, joined Class Disrupted hosts Michael Horn and Diane Tavenner to go beyond the hype. Tavenner and Horn pushed for answers on the MagicSchool’s quality, privacy, and whether tools like this actually improve outcomes for students.
Listen to the episode below. A full transcript follows.
Diane Tavenner: Hey, Michael.
Michael Horn: Hey, Diane. It is good to see you. As always, it’s been fun because we’ve had this arc of really digging deep on AI with school models. And then we’ve been moving into the edtech tools that are starting to define a lot of current models, the new models we might see and so forth. And we’ve got another big one. Dare I say, for today, this is going to be a good conversation.
Diane Tavenner: I am very much looking forward to it and looking forward to digging in. But before we do that, we have a quick ask for all of our listeners. Will you all rate or review Class Disrupted, wherever you’re listening to it? And of course, please subscribe. We’ve never asked anyone to do this in seven seasons. And you know, this is — it turns out, this matters. So we’d really appreciate it if you would do that.
Michael Horn: Yes, we would appreciate it. And we know we get a lot of feedback from listeners. We certainly get a lot of emails, and texts and things of that nature. So we know y’all are listening, but we need to see it in the ratings, reviews and subscriptions as well. It helps other people find the information, too. As folks know, this is a passion project for us, and we want it to actually be influencing the conversation more broadly. So, look, we know we got a lot of folks tuning in from our great partners with The 74, Substack, Apple, Spotify, YouTube, you name it, we know you’re out there.
But subscribe, rate us, leave us a comment, and we’ll just be appreciative for that.
Diane Tavenner: We will. And we will continue to be grateful for the direct feedback as well, because we use that. We take that in. And in fact, this conversation today is something folks have asked for. So we will not waste any more time.
Michael Horn: Yeah, no, it’s true. Well, I’m thrilled because we get to welcome Adeel Khan to the show, known obviously for MagicSchool. But before that — we’ll talk about MagicSchool in a moment — but Adeel worked as a teacher, assistant principal, founding principal for DSST, the Conservatory Green High School in Denver in 2017, and then left a few years — I think it was like three or four years in or something like that — to coach school heads. And my sense of — Adeel can correct me if I’m wrong on this — but my sense is, this wasn’t the calling.
A year later, I think it was, ChatGPT burst on the scene into people’s consciousness in November of 2022. And Adeel says, there’s something here. This can revolutionize the teaching experience and you know, quickly sees like, hey, teachers aren’t going to figure out how to use this productively. Let’s help them do that. Builds MagicSchool and it takes off like a rock rocket. No exaggeration. I think over seven million teachers now use it. That number’s probably out of date.
As I say. It partnered with more than 10,000 schools and districts, I believe over 160 countries. So we’ve got a lot to dig in here. The use cases are broad. But Adeel first, welcome. It is great to see you again.
Adeel Khan: Thanks Michael. It’s great to see you too. And Diane, pumped to be on the podcast today, and thanks for the kind introduction. Mostly accurate.
Michael Horn: What did I miss? Yeah, what did I miss?
Adeel Khan: You got, you got it mostly there. I think certainly when I got to coach principals, I didn’t, it wasn’t my favorite thing in the world. Not because it wasn’t a great opportunity. I worked on a great team, and principals need support. But rather, I think, when I was founding my school and building it, I thought, you know, if you’d have asked me if I had the most important job in the world, I would have answered yes. And I just missed that feeling. I didn’t have that kind of builder-chasing, really ambitious feeling when I was in that new role. And it’s sometimes good to know that about yourself, right.
I think that like sometimes trying something and knowing that you know where you are because, you know, in some sense at the end of my principal career and seeing the graduates in the first class of seniors, as I kind of built the full school out, I was exhausted. Right. And sometimes you think about, you know, what’s the other side like, you know, and you know, being an advisor sounds less intense than being in a school building every day and you know, chase this big dream and, and then, you know, knowing what the other side was like, I was like, oh no, some people are wired for this. Right? Being in the builder seat and being… I learned that about myself. And even when things get hard today, and they do, we have a lot of challenges. I know that this is what I want to be doing and, and I would be, you know, I wouldn’t be as… there’s no grasses greener thoughts anymore. Even when those seep in, I’m like, nope. I know this is the thing.
Even when it is, I’m in my most trustful habit.
Michael Horn: So I’m really, I love it because you’re willing to make the trade offs for the thing you really want. And obviously we hope all our young people start to grow that same muscle, right, in their lives.
Diane Tavenner: That’s the work I’m doing right now. So it’s an inspiring story.
Michael Horn: Well, I was going to say. Right, so like we have two, you know, Diane, Adeel, you know, founding school heads become founding ed tech company CEOs. I think the story of the founding story of MagicSchool is somewhat well known, but maybe what’s less known, Adeel, is how you think about the problem that you’re actually solving both originally and today because you all have expanded your scope quite a bit. But let’s start at the outset itself because I think your framing tells us a lot about why you grew so quickly. So, you know, how do you describe what MagicSchool first solved for teachers and why that approach was so important at the time it came out?
11 AI Prompts Every Teacher Should Know
Challenges using general AI tools in education
Adeel Khan: Yeah, so I think that there’s a part where there’s maybe a little revision on the way you describe the problem that I found, which was not that I didn’t think teachers would be capable of figuring out how to use this and use it productively, it was that the tool itself was a general purpose tool. It wasn’t built specifically for the domain and therefore it was clunky to use. And you know, at that time even really good prompting didn’t yield a really good result for teachers. It feels like, you know, ancient history that you had to have massive prompts for like an LLM to understand what you really meant in the prompt. But at that time it really was, so if you were prompting ChatGPT, it was like you could get something quality, but your prompt had to be a page long for it to really understand your context, really understand the format of your output, just all those different things. And if teachers were going to use it right away, we had to do some of that work to get them to a magic moment with the technology. A lot of teachers would report to me when I was training them on ChatGPT, which is kind of the way the product idea started was, you know, I was prompting it so much, it wasn’t saving me time, it was costing me time.
So there was like kind of a really acute prompt of, you know, promise that this has an opportunity to assist you, save you time and also make you more efficient, augment your abilities. But it’s not working for you right now or the vast majority of teachers in that moment. But I think of it as, you know, when I started thinking about building the product, it started with I actually thought I might be training teachers on ChatGPT, and that just didn’t work. I like, experimented with it. I went to my school and teachers didn’t really adhere to the product. So I was like, there’s got to be a better answer to this. I was savvy enough to kind of be looking around to see, like, you know, products that were coming out that were generative AI products. And they were like this first set of products that were hitting the market.
There was like Harvey, which was like AI for lawyers. I remember that really vividly. And I don’t think they even had a live product, but it was like announced as a product, and they’d gotten some funding, and there were news stories about it. And I was like, oh yeah, there’s going to be an AI that is made for each kind of profession. And that makes sense because each profession has its own quirks. It has its own domain knowledge, it has its own workflows, it has all these things that are specific to that industry. And a generalized alternative language model isn’t going to be the answer for every single one. So that was kind of the clear vision from the start, is we make a really domain-specific, intimate product for teachers.
And the products evolved a ton since then, but that core of it was like, it’s going to be really domain specific. Even in those early days, I thought to myself, we’re going to make some tools in this platform that if you’ve never worked in a K12 school, you would have no idea what that even means. Like, that was kind of my bar for myself is like, I want it to be so intimate, so clearly made for teachers that they scroll through that initial dashboard of tools and they see a couple things and they say, you know what? Somebody who’s worked in schools like, built this. I still hear that today. And it’s one of the things that brings me the most pride. Teachers don’t necessarily get built products that are so clearly intimate for them and know their work. And that’s what we hope to do and continue to do with the platform.
Exploring MagicSchool’s AI features
Diane Tavenner: This is such a good place to jump in because one of the things Michael and I are doing this season is trying to help people see how AI is truly being used in schools like today. And so building off that origin story and where you started and where it is today, I mean, let’s get into like how MagicSchool actually works and what, where the AI is in it. And, you know, this is the moment we get to kind of nerd out, which is fun. It sounds like you’re, you’re in that space too. Like, just to give people some context: When you enter as a teacher, it seems like there’s basically three big categories of support, teacher tools, writing feedback, and Raina the chat bot. And so, and then when I look at the list of teacher tools, you’re right, I open that up, I’m like, oh my gosh, there’s like 75ish tools. And, we’re getting everything from like a worksheet generator to report card comments, unit plan generator, standards unpacker, IEP generator, quote of the day, classroom management plans and even tongue twister, you know, so like help us unpack.
Like literally what’s going on here in this tools section? How’s AI? What is the role of AI and what, how are teachers using this?
Adeel Khan: So one thing that I think MagicSchool does exceptionally well as a product is, while there’s a breadth of tools and sometimes we get the feedback that it’s overwhelming, it’s like this tension we feel between it being overwhelming and also really familiar for teachers. Like, I’ve gotten that feedback since the very start and I always hesitate to like change the UI or make some meaningful shifts there because it has kind of become our calling card that we’re this like list of teacher terms almost. Right. The UI itself has been replicated by 20 other products at this point because of work. Right. It’s like, you know when you say AI to maybe a skeptical teacher or a teacher who’s like, technology’s not actually been a meaningful part of their story in the classroom, or a teacher who’s maybe been sold on technology being a really big opportunity for them, and they’ve been burned. And there’s all kinds of reasons why teachers and technology have not gotten along particularly well.
I will be clear. I was not somebody who like, as a principal, saw incredible value in technology. My school didn’t subscribe to any boxed curriculum-type technology tools or platforms. We built all of our own, internal. We built all of our own curriculum. We didn’t subscribe to a single ed tech product outside of the LMS that we used. And that was a kind of a requirement. And the reason was mostly because I just didn’t know that they were going to drive outcomes for our kids. And it was an outcomes-oriented principle.
I wanted to make sure our kids’ literacy and MAD scores and their SAT scores were like, that was, kind of our focus was preparing them for college. And I wasn’t sure that any of the tools out there would make a difference. And I might have been just wrong. Like, you know, maybe they just weren’t in my world. And there’s some great tools out there, I’ve learned since joining the ed tech world. But nonetheless, I think there is that skepticism amongst folks, and there are tools that have come to schools and have done just about nothing. I mean, everyone who’s worked in a school district or building will tell you, like, remember two years ago, that initiative we had? What happened?
That is just like a common story, like a new curriculum that’s going to revolutionize the way that you teach and can completely, all those kids that are behind your class, they’re going to catch up in a year. Or here’s a new tech tool that is promising to do this. So there’s all this baggage around, like products that come into schools. And the baggage often is, I invested so much time and energy in implementing this thing, and I don’t know how much value I actually got out of it, and then the district gave it up in two years. Like, you know, like, there’s like frustration here. So if you think about MagicSchool coming in, like really novice, you know, founder — me, I was just like, let’s get people to value really quickly, right? Like, let’s just get them into what this thing can do so it’s believable.
Using generative AI in education
Adeel Khan: So you jump into MagicSchool, you see the rubric generator, you click on it, it asks you some pretty simple like form fill up type questions like, you know, what we’re going to do attach the document maybe of the assignment you’re going to be assessing with this rubric. All very simple, super easy to use. And then you click the generate button, and you get a rubric that looks like a rubric. It’s what you expect it to be. It works the way you want it to work. It meets your expectations. Whereas like at the, at the, you know, in early days, ChatGPT and even still today, like it’s even the simple friction of if you went into a regular chatbot and you tried to build a rubric, of course you could, but like, you’d have to type in like four sentences.
I want a rubric. It’s gotta be six columns. It’s gotta be, you know, like making sure a boxed format. And then like, you know, you do it and then it, then the LLM will respond something like, oh, did you, you forgot this detail? Can you give me this detail? Wait, wait, what? Like I just gave you, like, it’s a frustrating experience for somebody who was told this is gonna save them time. And I have to know funky things about prompting nonetheless, like, that’s delight. A rubric generator that gives you a rubric is like a zero time-to-value, incredible experience for the actual. I see how this is going to be really helpful to me in my daily work. And the flywheel it creates is also a secondary value because if generative AI can do this for me, then what else can it do for me? And we have a chatbot on the platform Raina that is just like a ChatGPT but built for education with a couple, you know, bells and whistles and education-focused context. And if you use that, like, if you know that, you know, MagicSchool could create a rubric for you.
No, we’re not hiding the ball here. Like, you know, what’s happening is there’s a prompt, there’s inputs, it’s coming out with something. You know, you can probably prompt Reina and do something completely original or new, but you need to be able to see the value first for you to be really sucked into it. And like, I think an amazing job of getting people in the door with something really low stakes that should like, turns out to be really high quality, saves you a bunch of time and makes you want to use the platform more. It makes you want to think about, okay, what are the other tools? If it did this thing for me, what else can it do for me? And I think that’s the way that we see the flywheel start. We think about the user.
They start with these really simple tools. They might graduate into their own free form prompts and Raina and trying things. And then they might decide to use an agentic workflow like our class writing feedback tool, connect their Google Drive and have writing feedback dropped on each of their documents. Then they might want to think about, well, if I’m using this in my work and it’s really valuable, it’s helping be a thought partner to me. I wonder if it would be helpful for my students typically. So then we —
Diane Tavenner: Yeah, super helpful to understand your thinking and your flow, and it seems like that is working. Let’s go back and nerd out a little bit on the rubric generator. So where…? What happens when they type that simple thing in? Like, how are you using AI literally? What’s going on behind the scenes there? And as a person who’s written a lot of rubrics in my life, used a lot of rubrics in my life, like, how do I know that’s a good rubric and, like, what’s going on behind the scenes and under the hood?
Adeel Khan: Yeah. So great question. When MagicSchool started, it was just a prompt. It was like me with a prompt in the background. I had created rubrics as an English teacher in my life, and the best instructions I could give, and the judge was me. Like I was whether or not the quality was high enough or, you know, would meet my standards. And then it was a group of users who were doing the same things upon using it.
Diane Tavenner: And you’re prompting one of the big LLMs, essentially.
Improving model quality and selection
Adeel Khan: That’s two and a half years ago. So just know that’s like, it’s radically, radically different. So today we have a trust, safety and quality team, running our own evaluations on the platform. We have pedagogical experts on the team who are reviewing poor quality. So there’s a human part of it. Then there’s a thing called LLMs as a judge. Some of our evaluations are using models like Claude to judge the quality of the rubric. I was actually on a panel yesterday with one of the, a teammate, one of the teammates at Anthropic, his name is Nirob, and he was, he was naming that, you know, Claude itself at this point is probably the level of like a PhD in domain.
So, you know, you can have Claude, a PhD in a domain, almost, nearly, and getting better every day, judge the quality of the rubric too, and then iterate on the prompt model we’re using and change those things out. So we actually have, we have a wildly more complex version of judging for quality internally now. And it’s a combination of like LLM as a judge, human in the loop, user feedback on the platform. And then when new models come out, we are, on a regular basis, once every week or two, we’re saying, is this new model better performing against our criteria? And so each one of those essentially roll up to a score, and we have a score that determines whether or not this model can exist in the platform. And sometimes, you know, if it’s a 94 from one model and a 95 from one model, but one model is like one tenth the cost, we will choose the one that’s a little bit cheaper because we have to keep our platform affordable for schools. The vast majority of them, we’re able to say the absolute best output is the one that we go with, but they’re judged across a lot of different quality markers now to ensure that the output is something that we can stand behind.
Diane Tavenner: So just so I make sure I’m understanding. When I go in there and I pick a teacher tool, like, I think it’s report card feedback or something like that, you all have kind of established what good report card — the elements of good report card feedback. And then you’ve created prompts that are prompting the LLMs to do that. And then you’ve gone through and tested those responses to make sure that their quality. Because literally the humans aren’t testing while I’m the teacher here asking for that. You know, it comes back like that. Right? So.
So you just have confidence in those responses I’m getting?
Adeel Khan: Yeah. So, yeah, we’re essentially running out, like, hundreds of queries against them and then judging queries and then orienting around what the best kind of set of all of those variables. What’s the best prompt? What’s the best model? What’s the best output?
Diane Tavenner: Okay, got it. Now help me understand, like, that rubric. So now, like, I’m a teacher in my classroom, it’s often been standard that the teacher, sort of in their classroom, doing their thing, using their own rubric. How do you think about it from, like, your principal seat, like, the whole school? So, I mean, you know, for example, when I was leading schools, we built a longitudinal rubric that, you know, over time, the kids were — and so one of the things, as I play with MagicSchool, I’m like, oh, what’s the bigger picture here? You know, what’s the — what’s the high school arc? Or what’s the whole arc? And is this kind of more of an activity in my classroom base? Or is it the full big picture, the backward planned approach? How do you think about those things?
Adeel Khan: Yeah, it’s a really great question right now. I think that MagicSchool’s teacher side is best judged as kind of an assistant that helps you in the moment that you need it in your daily classroom activities. You can use Reina as a thought partner when you’re struggling with something or a Chatbot, you could, you know, really build out, like, a full, like, unit, then the subsets of lessons and like a generative thread, all the tools are threaded. So, for example, if you started with the unit plane generator, you could then say, build me the first lesson in this unit. It keeps context of that entire thread, and you could give it, like, input about, hey, my students performed this way on it. Can you generate the next lesson with that context? And you could do it that way.
But transparently, few people do. Right? Like, it’s really like, they come in, they kind of get what they need, and they keep going, we want to move toward more of what you’re describing, which is like, hey, the entire cycle is brought through in the platform. You give the platform your intent, and you keep these really rich threads with memory, context and knowledge of your classroom. So, we’re moving toward where you can kind of have the entire context of the classroom experience built in the platform. One of the key components of that is assessment.
Building personalized learning assessments
Adeel Khan: Like if the platform understands how your students did on their assessment, say for example today you could build an assignment in MagicSchool and you could, you could actually, you could build an assessment in MagicSchool. It could be taken by your students in MagicSchool. The results of that assessment could produce some really interesting insights about how they performed against the standard. And you could then create a material based on that assessment. So that’s where we think it’s going, is that hey, you’re going to build things based on the assessment results that take into context your students areas of strength and areas of growth based on the insights that are in aggregate across those folks who took the assessment. And you can imagine that there’s even more you can do, right? Like there’s on the student side, on an individual level, if you had assessment data about a child and you also had them taking assessments in the platform, you had them doing activities in the platform, you could trace a student’s personalized learning profile that understood kind of their academic needs. It could be continually updated with memory around the way that they interact with a platform. Not just from their academic strengths and how they’re growing, but also their stylistic preferences, their post secondary desires.
And you could really make these incredible persistent learning experiences for students that they always can tap into that’s associated with a really rich profile to serve them right in their zoning proximal development and in a style and in a way that they will engage and learn more.
Michael Horn: I’m curious off that just because that, that like becomes a very big vision, right, thwt shapes around the student how starting from like a teacher workflow, right to like the student life cycle almost, if you will, how much do you need to know and collect about individual students and how much do you need to know about the specific? Like take the assessment question, right, like, so I’m going to give feedback on you’re grappling with a particular poem or a book or something like that. How much does the model then need to get trained not just on the teacher rubric, but actually on the content itself as opposed to just the standard, which is probably a higher level statement of ability to do X. But ability to do X in one context might be very different from another. So like, just help us think through the scope of that, of getting to that vision you just painted.
Adeel Khan: Yeah, it’s a really good question. I mean, we are at the early stages of understanding how this will work without student data. But as we build so one of the things that we are doing or in the planning process of getting some of these ideas off the ground, there is a really interesting process where you can kind of use LLM agents as sample students and then you can also assess the quality of their interactions with the platform based on specific profiles that you create for the AI agents. And then you can kind of get a really good data set on what you’re describing, Michael, is like, is it working? Is it actually meeting your needs? Or is that student who’s an agent who has the stylistic preference around visuals for their needs. That’s the way that they learn best. Or audio is the way they learn best. Their reading level or their reading test score diagnostic was this.
And this is what that means. You can kind of create all these profiles individually and then you can run them against the interactions they have with the platform and then you can judge how high quality the interaction is on the platform with like an external observer. So, like, almost similar to where I described that LLM as a judge, the judge can actually judge the experience.
So you can think of it as almost like a principal judge. Right? There’s a principal who’s an agent who’s watching the AI interacting with the student and determining is that a high quality interaction or not based on the child’s profile, based on what I know about what high quality instruction looks like. So you can start building these kinds of recursive loops and running them hundreds of times and get to some pretty, pretty impressive verifiable outcomes pretty quickly. And of course, you know, AI is not kids. And they’re not going to be perfect.
Michael Horn: AI is not going to act out for —
Adeel Khan: Yeah, yeah, yeah. So I don’t want to, you know, kind of anthropomorphize that this actually is the answer, that we can simulate everything through generative AI. But it’s a great way to like in a, in a basic sense feel like, okay, it works in a simulation. And then when we bring it to students and we work through the pilots and we see schools and districts who embrace this, then we can see it in action. We can combine those insights with the insights we got from our simulated experiences and make something that we think is really powerful. Some of those generative experiences, we’re going to be starting this summer in summer school with some partner districts to see how it works in practice and is it actually the needs of the kids. And we’re going to use that data as data to inform the product as it gets into more live cadence.
Diane Tavenner: You just said so many interesting things in there. Like, I am of like five minds right now. Where what thread do I want to pull? But let me pull on the one, because you started by saying, like, we’re playing with what we can do without student data in there. And best I can tell, based on my experience in MagicSchool you’re not connected to an SIS or anything like that. So you’re not pulling in any data as the teacher around your students. That said, I did create, I did use the IEP generator and I generated an IEP and I, you know, I, look, I didn’t do it on a real student, obviously. I’ve got like 25 years of experience. So I sort of built a proposed persona in my head and like input the information.
10 Useful Tech Tools for Educators in 2026: A Practical Guide
Discussing AI-generated IEPs and privacy
Diane Tavenner: And I will say I was like, kind of blown away that a fully formed, detailed, and dare I say, very confident IEP like, came out. You know, and at first I was like, oh my gosh. And it looked like an IEP that you would kind of read in a school sort of thing as I was skimming it. Then I started like really digging it, and I was like, oh, as someone with 25 years of experience, like, this is not the IEP I would have written necessarily because I, in my mind, you know, the kids I’m thinking about, like, I have intimate details and I’m like, wait, that feels a little, it felt a little AI-ish, right? Like sometimes AI gives you really, like, seems like compelling results, but they’re not very specific or personalized or whatnot. And so that was one thing I was curious about. Like, how do you think about some of the tools that are like that and, and how they get used? And like, I was thinking I’m a first year teacher, and I use that, and I don’t have 25 years of experience. Like, can I? How do I do that? And then the second piece is you’re out with teachers, like, do they just pour a lot of information about kids into the platform? And I’m sure you get a lot of questions about privacy and security and, like, what’s going on there?
So I’m curious about those two angles.
Adeel Khan: Yeah. So I mean, we do a lot of professional development. I’ll start with the second one. We do a lot of professional development around making sure that teachers do not share information that could be sensitive into the platform. So there’s upfront training in that tool you used, actually, you’d see that like it actually actively reminds you in the tool itself to not, because that’s a tool that’s more susceptible for you to maybe submit accidentally that data. Of course, if any data is brought to our attention that was submitted to the platform, those PII, we remove it immediately.
And we have incredible data handling practices, and safe things are all available on our website. So we’ve done a lot of things to make sure that schools and districts can trust us. And so that’s certainly something that we consider. And we just think training and enabling teachers is the best way to prevent that stuff. Because even if MagicSchool has really great data handling practices and is kept safe, like they might bring it to another platform and you know, we want them to know how to use our AI, but also any generative AI and keeping student data safe is incredibly important when you’re using these tools.
Diane Tavenner: Yeah.
Adeel Khan: Second part, Diane, I think is really interesting. So I was a special education teacher too. So for me, I can think back to when I was a novice teacher, and MagicSchool would have been a godsend for me. The IEP would have prevented me from being anxious. I probably would have helped me save a lot of late nights, and it would have made me feel like I had real strategies to support kids because, you know, it’s a good point. So, so your question, I think I might challenge this. Like you presented it almost as like a fear that they’re not 25 years, so maybe they’re just taking this robotic IEP that’s not as good as the high-quality, 25-year IEP. I want you to know that like I’ve been a principal reviewing IEPs and I have seen teacher wonderful, hard-working teachers Submit IDPs with the wrong names in them because they were just copying goals and pasting goals from student to another because they were just trying to get it in by the deadline.
Right? Wonderful, hard-working, incredible teachers. So do I believe the world has gotten better because of MagicSchool’s IEP generator? Yes, it has. The floor has been raised because there are actual, because now the barrier is you understanding the student and getting some, some high quality things and you know, even a novice teacher will see the goals and at least they have, they know how to write goals now.
I didn’t know how to write right, like, and I didn’t necessarily have someone to go to to help me write those goals. So I would say net MagicSchools raised the board dramatically for the way an IEP is written. I think a family would be much happier to see a MagicSchool-written IEP than the ones that I was editing, if I’m being totally honest. And it’s again, not because teachers aren’t wildly hardworking and talented. It’s because there is no time. And so I think that’s kind of the reality now. In an ideal circumstance, you know, they have a really great draft and they have an instructional coach like you, who they can go to and say, what do you think about these things? And like, you know, they can question and challenge them and push them and make them even higher quality.
But I do think that, like, sometimes we miss the reality of what happens in schools when we were critical of the tools that teachers are using to better their practice. And sometimes we just need to trust teachers. Like, in that case, I’m like, actually, I even trust the first and second year teacher to use this appropriately. And especially if they’re educated and they’re told, like, the way to use it, not to submit appropriately private information, things like that. But I always challenge people is like, yes, you know, people, we have schools and districts that hide the IEP generator because they’re so scared of it. And you know, we respect anyone’s decision on what they want to do and they’re allowed to do that and like, you know, our enterprise product and then we support them in doing that. But I, on a personal level, I always challenge them. I say, like, look, like, ask your assistant principals who are reviewing IEPs.
Yeah, they see, are they, are they higher quality because you took this tool away or are they higher quality when MagicSchool is in the loop? I think that’s like the question to ask is what’s the before and the after rather than like, what’s the ideal? Right. You want the ideal. But I think that there is like a, this go between. I think it is super strong. And I think that the tools are getting even better over time. And I think that the better the context that you share with it, the better it’ll do of course.
Diane Tavenner: Yeah, that makes sense. And I think I understand the perspective that you’re coming from and certainly your lived experience. And I do think sometimes people, you know, have an idea of what is happening in school versus a reality of what is happening in school. And so I appreciate that. I was also really tuned into you saying that as a school leader, you all developed your own curriculum. That was my reality too. You know, we ended up building curriculum, we ended up building Summit Learning, which was like a whole massive curriculum. And so I’m wondering how.
But a lot of schools have adopted curriculum, as you know, how do you recommend that teachers sort of deal with the world of like, I have an HQIM that I’m given by my school and I want to use MagicSchool. How do those two things play together? Or do they, or what does that look like?
Adeel Khan: Yeah, I think that, you know, again, lived reality of an HQIM in a district. I’ve lived in that reality as well. We didn’t box any curriculum, but sometimes we would get for a certain subject we would have like, you know, free prep lesson plans or whatever it might be. And I think the lived reality of those things are teachers are always modifying, changing, supplementing, making those things work for their classes. And that’s good. So I think that that’s what we’ve encouraged them to do is like, yeah, keep the spirit of what your school wants you to do. And obviously there’s a research base behind the curriculum that you’re using, hopefully, and we want to be a great supplement to that. We want to make sure that you’re able to build the supplementary materials that you need to make sure your class functions and works.
Integrating curriculum with MagicSchool
Adeel Khan: And MagicSchool could be used alongside those things. There’s a world where we build knowledge into our platform so the schools and districts can upload things like standards, curriculum they’ve built internally that are not like, you know, copyright by the publisher. And there’s a world where we partner with publishers too, and we bring those knowledge bases into our platform and call them as well. And as you’d imagine, those publishers aren’t super excited to work with large language models because this is like their proprietary IP. And nonetheless, I think that like, you know, we think that there’s a future where there’s kind of a win-win for both of us in a world where teachers are finding that they’re starting their day with MagicSchool, and it’d be really convenient if they can pull in some of that information and make those curricula more flexible with generative language models. But yeah, I support that. I’m also, I think a lot about curriculum and I make spicy posts on curriculum on my LinkedIn, if you haven’t seen them. There are a couple curriculums that are incredible and obviously they’ve driven meaningful outcomes for kids.
And I don’t think they’re all that way. To be clear. I don’t think they’re all super research based. I think that a lot of them are not particularly valuable and I think, I don’t think. I know a lot of teachers hate being put in a box. They hate having to be told that you can’t be autonomous in your classroom. You must follow the script because the script is better than anything you would create.
Which is like the, not the intended message, but the unintended message that a teacher might receive when receiving certain curriculum. And of course they’re the teacher’s love. Right? Like they tell you, no, this thing’s amazing. It’s changed my classroom. So not painting with a broad brush. There’s also like really amazing ones. I will say that my lived experience has shared that like if you trust teachers to go find the right resources and give them the tools to do that and know their kids and coach them really well, you can get really extraordinary outcomes. Mind you, most of my experience is secondary, but we have incredible results for our kids.
We served a highness population and had dramatic growth. And so that’s my own experience. So I don’t know. I think that there’s a little bit of like, I would not call myself as a personal. Like on a personal level. I do not ascribe to the Church of Box curriculum. That is not my ministry as we used to call it in Atlanta.
So do I think there’s really good stuff out there? Absolutely. It’s not my ministry. I just teach it.
Diane Tavenner: That makes sense. Clearly you are outcomes driven. I know that based on the school that you started and all the language you’re using here. How do you think about — how do you now at MagicSchool think about what outcomes your — how do you hold — what’s your bar? What are your goals? Like what are your outcomes that you want MagicSchool to drive to and how do you measure them?
Adeel Khan: So there are a couple ways that we’re thinking about this. So outcomes are at the heart of our mission. We have named goals in our company about how we are going to drive student outcomes in classrooms. Right now, the way that looks in our platform is the amount of what we would call feedback delivered to students. So generally what we’ve seen in the first two years of the product is that like the things that we, that teachers have said have driven growth, the experiments we’ve set up or have not set up we’ve just been reported are that when students get things like feedback on their writing that they’ve done on their own aligned with rubric in the platform, that is powerful. That is something that drives an alchemy classroom.
Measuring feedback and student outcomes
Adeel Khan: We have actual classrooms that have shown state exam scores change over those things. We have enough data to say that like when MagicSchool gives student feedback that teachers is controlling and aligning to a rubric or the way that they’ll assess students, that’s a great thing. So we measure right now how many instances of feedback is MagicSchool giving to a child under the supervision of their teacher through either our assessment platform in the product which gives students just in time feedback after they take an assessment, or in their feedback portion of the platform where they tune in. So those to us are pretty hard. Like we don’t kind of look at it as like hey, you talk to a chatbot, so you learned. Like, that’s not enough for us to kind of count and like our something that we think is definitely going to drive an outcome. It certainly might drive AI literacy and we certainly think about that measure as well. But in terms of outcomes, that’s the way we think about it now.
Well, where we want to go is we want to be able to probably say that students have learned and judge it in the platform itself. So one of the ways we might do that is through having a diagnostic assessment students taking the beginning of the year and then at the end of the year or tracking timing platform as we have more persistent student profiles in the platform and simply asking the district to themselves compare their users data which students spent the most time in platform and how much did they grow and then just give us a report back. There’s ways that we can kind of say it’s not a perfect correlation, but if students are spending more time in the platform and they’re getting better results at the end of the year, they’re growing more than their peers. That’s usually a pretty good indicator to us. There’s an experiment that was run unbeknownst to us in our first year. I actually shared this on a panel yesterday. I was at South by Southwest.
But I think it’s a powerful example and I think what a lot of organizations are doing around generative AI around the world and I actually think this is fun because a school district was ahead of enterprises, they’re ahead of companies in the way that they’re thinking about generative AI and our first year that we had a real enterprise product was 2024 to 2025 or 2023. Our first partner, which you imagine our very first partner at MagicSchool would be pretty innovative. And they certainly were as Aurora Public Schools, one of our very first ones. At the end of their first year using MagicSchool, they got a printout of their MGP scores at the district level. MGP in Colorado is basically like a growth score that is associated with each teacher in the district based on their state exam scores. Their grade has a high stakes exam. So the district quite literally gets a list of all the teachers who had the highest growth in order of like this calculated growth score. So they kind of have like which teachers are having the best results in their classroom.
It’s pretty sophisticated calculation. It tracks like based what was their expected growth based on their prior year’s performance to this year’s performance. It’s a pretty solid number. Like at the end of the day it’s like the kids actually grew and, and it’s based on some pretty hard metrics. What they did was that they liked the way the story is told, is the academic, one of the assistant superintendents called the technology director and said, I have all the teachers who have the highest scores or the highest MGP scores in the district. Can you pull up MagicSchools our user dashboard which shows which teachers are using it the most? And they said like one for one. It was like best growth scores were in the top 20 users, best growth scores, best users.
Using AI to boost engineering productivity
Adeel Khan: So the way industry is doing this now, what we’re doing at MagicSchool is we, we have an enterprise version of Claude that our employees use and where we have a real big focus on like agent decoding for our engineers to move faster and ship with more velocity, build a lot of really amazing things for our users. And one way we’re measuring the efficacy of generative AI right now is we’re looking at our leaderboard, like, which engineers are using the most tokens in our version of that enterprise dashboard? And then we’re asking the managers, would you call that your highest performer? Like, is that engineer who also is using the most tokens in Claude performing better? Are they shipping more? Are they meeting the goals that you’ve set for your team in a better way because they’re using AI more? And if the answer is no, then we need to rethink about is generative AI actually helpful? But if the answer is yes, then we need to spotlight that engineer and we need to have that engineer teach the other engineers how they’re using it and how it’s making them more productive. And Aurora was doing the exact same thing two years ago. So I think it’s a really, really cool way to think about how this is actually amplifying productivity in a meaningful way.
Michael Horn: Adeel, I think that’s a good place to leave the conversation for now. I appreciate how much you’ve geeked out with us, and also I appreciate that you’ve told us where it is now and where your sort of vision for it is. As you know, a lot of entrepreneurs, they sell the future/present as one package as opposed to distinguishing the two. So I appreciate that in this conversation as well. Before we move to our last segment, as listeners know…
And with that we’ll move to our last conversation, which is it’s just a fun segment we’ve had and people track this and so forth, Adeel on LinkedIn and believe it or not, about things that we’ve been watching, listening or viewing and would love to hear something that you’ve been tuning into what’s either on your bedside table or on your playlist.
Adeel Khan: I think I’ll go with a Netflix show. So it took me quite a long time to get to it, but I finally watched just finished the last season of Stranger Things, which I was late to the party yard in the first place but kind of binged it a few years ago. And so I was eagerly anticipating the final season. None of it got spoiled for me and I got to watch the whole thing and it was delightful. So I felt like I wrapped the bow on a really special I feel Stranger Things is so awesome if you guys have seen it, but I feel like it’s such a special cultural phenomenon that everyone kind of experienced together and watched together. So that was mine. Speaking of a new show. So excited to hear from you guys.
Diane Tavenner: Well, I’m not going to give you a show today. I apologize. I’m gonna give you a book. It’s a little bit ironic to have AI books, I think, but this one feels special to me. So it’s called Co-Intelligence Living and Working With AI by Ethan Mollick. And I will say, when it arrived, my kiddo who works on the models said, oh, that’s perfect for people like you. I was like, what do you mean? He’s like, you know, for, for regular people who don’t understand what we’re actually doing, but, like, have some sense of it. And it’s really useful for kind of who are really trying to make sense of AI and world and what that looks like.
And that’s what it feels like to me. And so it’s, you know, I’m not telling you anything new by sharing this book with you, you know, a bestseller. But it is, it’s short and it’s thoughtful and I think useful for anyone who’s really trying to grapple in this space. I would recommend it.
Michael Horn: The one question Diane, I have, I love Ethan’s Substack as well. And when the book came out, I bought it as well and read it. But I’m just curious, like, does it still feel current given, like, the race?
Diane Tavenner: You know… yes?
Michael Horn: Interesting.
Diane Tavenner: Yeah, I think so.
Michael Horn: OK.
Diane Tavenner: I think so.
Michael Horn: Cool.
All right. I’ve got a book as well. So, Adeel, I’m also striking out on the show, watching, I think, at the moment, but I still haven’t done Stranger Things. Diane, I think this was on her list. I can’t remember how many episodes ago
Diane Tavenner: I started early, but I haven’t finished off the season. So you’ve been —
Michael Horn: I was about to say. You just inspired her to finish it. Yeah. The book I’ve been reading is A Heart of a Stranger by Angela Buchdahl. She’s a rabbi at the Central Synagogue in New York City, which I think is the largest synagogue in the U.S. or, or top two, I guess. And it’s terrific, she’s a Korean-American rabbi. And so it’s like a very interesting story about where she, the circle she has not belonged in, and then making sort of this, I guess, momentous movement into being a rabbi and sort of what that’s been like and her life story through it.
So it’s been a very good read. As Diane knows, my wife’s Korean-American, so, like, and I’m Jewish, so it’s like hitting on multiple levels in our household. TBD if anyone else in the household reads it beside me. But I’m enjoying it quite a bit. And we’ll leave it there. Adeel, huge thanks for coming on, joining us, geeking out with us, and for all of you, keep the questions, comments, notes coming after this episode and in general, and we’ll see you all next time on Class Disrupted.
This episode is sponsored by LearnerStudio.
Did you use this article in your work?
We’d love to hear how The 74’s reporting is helping educators, researchers, and policymakers. Tell us how
