Like most of us, Emily Freitag hears a lot about generative artificial intelligence in the classroom. Much of it seems like the typical ed-tech hype.¶
Co-founder and CEO of Instruction Partners, a Nashville-based nonprofit that works with teachers to provide high-quality resources and instructional support, she got curious about AI after seeing the tool Coursemojo in action.¶
It’s designed to help students read texts closely, and it did it in a way that was “very different than any tech I had seen,” Freitag recalled. It made her think that AI “may be a horse of a different color. It may be really, actually a step forward in capabilities.” But she wanted to look past the hype.¶
A former assistant commissioner of curriculum and instruction in Tennessee, Freitag got the idea to take a closer look at the full spectrum of AI tools. “I wanted to see it as a mom. I wanted to see it as an educator, and I wanted to see it to be a good partner to our schools.”¶
So she and a colleague last January began a six-month investigation that would eventually take them to 16 school systems in seven states. They sat with students as they worked through AI-enabled lessons, watched teachers teach with AI, and interviewed more than 100 teachers, school system leaders and ed-tech CEOs.¶
The results of their first foray are out this week: Instruction Partners on Monday released the initial findings in what it is calling its AI in Action Learning Tour — a year-long effort to observe how actual students and teachers use AI.¶
The report looks at 20 student-facing AI products already in classrooms, broken down into three categories: general-purpose chatbots such as ChatGPT, Claude and Gemini; education-focused platforms such as Brisk Teaching, Google Classroom, MagicSchool and Playlab, and targeted instructional tools built around a specific instructional jobs — they include Amira Learning, Khanmigo and Quill.¶
“AI is definitely not a single category,” Freitag said. “We saw a huge range of products designed to do a huge range of things.”¶
They framed their inquiry around two questions:¶
- How does it compare to paper-pencil instruction?
- How does it compare to the best possible learning experience in that subject?
Freitag and her colleague found a decidedly mixed bag, with a few key surprises.¶
“I saw tools that held a higher bar for learning than we see in paper pencil instruction and did more to tailor the questions and the feedback for kids to the specific sort of responses they were giving than a teacher could possibly do all at the same time,” she said. “So I really walk away thinking there’s a lot of good here that we need to leverage — and a lot of bad here we need to mitigate.”¶
They found that general-purpose chatbots like Claude and ChatGPT pose the biggest threat to student thinking, while the platforms — Brisk, MagicSchool and the like — produced different results, depending on how teachers used them.¶
The targeted tools, such as Khan Academy’s Khanmigo and Quill, a writing app, produced the most consistent and strongest learning experiences, they found.¶
And teachers told them they’re frustrated by apps kids can easily game — in one case, students found that by simply hitting the “Refresh” button on a computer, they could complete an assignment by answering the same question repeatedly instead of the lesson’s full list of questions. Many students also learned that they could simply Google an answer and paste it into their lessons.¶
In another case, an app gave students detailed feedback on their writing — “I thought the feedback was really good and useful and actionable,” Freitag said. But she watched in horror as students simply ignored it and began rewriting without consulting it.¶
In another instance, she sat with a student as he worked on writing a book based on characters he liked, focused on letter sounds he needed to practice. “He created the book, and he said, ‘Now it’s your turn. Let’s create a book for you.’ And I was like, ‘But are we going to read your book?’ And I realized he was just making books without reading them.”¶
The boy, she said, simply liked making the books.¶
“As someone trying to make sense of where AI has a role to play or not, I think seeing how kids are actually using products matters much more than understanding what they’re designed to do,” Freitag said. “And there are many cases where what I saw in action was quite different than what I expected to see based on the product demo.”¶
In fact, many of the apps that give students the ability to work together shone in Freitag’s review. By contrast, several apps designed for solo use turned out to be “boring” to many students, who spent perhaps 10 minutes engaged. “By minute 42 of the lesson, they are just tuned out.”¶
In many cases, she said, the same product produced dramatically different results — and gave kids different experiences — depending on whether teachers built in clear routines and actively monitored dashboards in real time.¶
In one instance, a teacher taught a lesson with a math app that allowed her to display four students’ solutions to a problem side-by-side on a Smartboard, allowing students to evaluate the solutions. “I was stunned by how much kids learned from looking at each other’s work in that way,” Freitag said. Even with just a little feedback, it was effective. “When it was used socially, it really created a ton of learning opportunity for kids.”
The apps that worked did something that teachers couldn’t do alone, such as offering instant feedback or a quick analysis of an entire group’s work. Those, she said, “were the products where I was like, ‘Wow, this is extending what a teacher could do.’”¶
But the report emphasizes that AI shouldn’t cut teachers out of the equation. They still have a vital role to play and AI should never be seen as replacing teachers. “My take is that the future is hybrid,” Freitag said. “Teachers have a key and lead role, and there’s tech that gives them an assist.”¶
In fact, one of the most promising AI innovations is its ability to identify where students are struggling with foundational skills — and where they need more practice. But forcing students to practice those skills in isolation could be counterproductive. “If we help kids practice a bunch of reading skills without actually paying attention to the text or the knowledge that they need to access that text,” Freitag said, “we’re not going to see reading scores improve.”¶
She also worries about AI’s near-miraculous ability to re-level reading passages with a single click, one of the most popular features of most reading tools. While it can help slower readers understand a text, that releveling can become “a permanent lower setting” rather than a bridge back to grade level.¶
Arman Jaffer, CEO of Brisk Teaching, one of the apps reviewed, said his philosophy is that releveling “should be used to increase entry points to high-quality instructional material,” but not to limit how far students can go.¶
“AI is something that’s incredibly powerful that can help you do things like relevel,” he said. “You can make the entire internet accessible to your students, but at the same time, a teacher who might not have the best instructional practices might be using it to hold back students.”¶
Jaffer said the findings underscore the need for coaching and professional development for teachers as AI emerges in the classroom. “I think every AI company, every technology company, every vendor, truly needs to be a partner to their district or their school or their organization.”¶
Peter Gault, CEO of Quill, an AI-powered writing app that was also reviewed, said Freitag and her partner watched students using the app in New Jersey and worked hard to unpack how it worked. “I was really impressed with how much due diligence they did. They really went under the hood to understand what works and why it works.”¶
The duo, he said “did the most thorough deep dive of anyone who has looked at our AI.”¶
Alex Sarlin, co-host of the EdTech Insiders podcast, called the new report “amazing work,” noting that Freitag spoke not just with teachers and students but with tech CEOs. “She’s talking to all sides of the equation,” he said.¶
The time she and her colleague spent in classrooms is key, Sarlin said. “That gives a much richer picture of how AI is actually being used in the classroom than we usually see, and certainly more than we see in mainstream coverage, which tends to be very surface level and a little panicky.”¶
This fall, Freitag’s team has grown to three people — they’ll soon look at 30 more products and report on them this winter. And Instruction Partners plans to work with Stanford’s AI in Education Lab to independently vet impact claims by winter 2027. The new report notes that just seven of the 20 products have independent studies examining outcomes across student subgroups.¶
For her part, Freitag said teachers are hungry for clear guidance around AI. “We work with instructional leaders across the country, and they said, ‘We are getting inundated with sales pitches, and we are getting inundated with calls now to “Ban it all,” and we don’t really understand where there’s value in this.’ So that helped me see that the debate about AI in schools is moving faster than real information.”¶
This story was produced by The 74, a non-profit, independent news organization focused on education in America.¶
