What really is ‘adaptive learning’?
What do you think about when you hear the phrase “adaptive learning”?
The answer may vary based on which platform you currently have in mind. The edtech industry is bursting with platforms that attempt to tailor lessons and materials to individual students. There’s Summit Learning, supported by the Chan Zuckerberg Initiative, which offers a digital teacher-student learning platform to K-12 students. Khan Academy provides instructional videos and drills on thousands of topics, while Duolingo targets world-language students, funneling streams of new vocabulary words to them. Newsela, as another example, claims to adapt reading material to the student’s level; and entrepreneurs around the world are developing still more educational platforms, including 17zuoye and Yixue Group’s Squirrel AI in China and BYJU’S-The Learning App, developed by Think & Learn Pvt. Ltd. in India.
All these platforms claim to be adaptive learning systems at one level or another, yet their differences are more numerous than their similarities. In order to speak intelligently about them, it helps to break them down into their own categories. One can’t treat Summit Learning and Newsela as the same learning products.
A team of students and researchers at the Stanford Graduate School of Business set out to assess how effective adaptive learning systems are. They wanted to test the belief that adaptive learning can transform education and allow more kids to reach their individual potential. At first, the team planned to interview leaders of edtech companies, sort systems into categories, and evaluate the effectiveness of products against their stated purposes. Perhaps they would learn that the more adaptive a product was, the more effective it was. Or perhaps it was the opposite!
The team was immediately struck by the lack of consistent definitions in the field. They couldn’t even start to think about evaluating products because there was no sensible way to categorize them. People in the field treated Duolingo and Khan Academy, for example, as the same thing — they were all “adaptive learning products.” Any frameworks that might have existed were elusive.
So the team pivoted and decided to propose a common language for talking about adaptive learning and a framework for thinking systematically about it. After numerous interviews with thought leaders in this space, they found something useful.
To start, let’s clarify what the team means by the term “adaptive learning system.” Slightly adjusting Lou Pugliese’s definition, the team defined it as “a learning system that is designed to dynamically adjust the level or type of course content based on an individual student’s abilities or skill attainment in ways that aim to accelerate a learner’s performance with automated technology-based interventions” (modified from Pugliese, L. (2016)). What’s more important than this definition is how to categorize various adaptive platforms.
TAXONOMY — TECHNOLOGY
After talking to edtech leaders, the team came up with two taxonomies, the first of which sorts products by the technology they use. This is important because everyone wants to know which products are using “AI,” but what does AI really mean? This taxonomy helps answer that question.
Rules-based adaptive systems
These systems do not use an algorithmic approach. Rather, the learning path is pre-determined by rule sets and decision trees that do not change for individual students.
Examples: Pearson TestGen and Khan Academy (as of April 2019)
Item response theory and Bayesian knowledge tracing systems
Systems based on these two technologies use statistical models to measure mastery of concepts and the probability of a student’s answering each question or problem correctly. These platforms require knowledge maps or other structures detailing academic concepts and the difficulty level of each question. The systems use their knowledge of mastery to determine the series of questions to give the student.
Examples: the Graduate Record Examinations (GRE) and Graduate Management Admission Test (GMAT)
Machine learning-based adaptive systems
These systems predict the probability of a student’s answering all items by using machine learning algorithms, such as logistic regression or neural networks. To maintain student motivation, these platforms give problems or lessons of an appropriate level of difficulty. Machine learning algorithms require large amounts of data to improve the accuracy of predictions, so they’re best suited to teaching subjects for which large amounts of structured data can be collected and processed computationally in real time.
Examples: Yixue Squirrel AI and McGraw Hill’s ALEKS
Note that the above definitions are not mutually exclusive. Companies and even specific products can exist across multiple categories.
TAXONOMY — ADAPTIVITY
In addition to the technology taxonomy, the team also found a need for a more functional look at adaptive learning products. In other words, how granular is the adaptivity? Does it consider every student’s answer, route them to the right problem, and give adaptive hints? Or does it just suggest which reading level they’re at?
Adaptivity is independent of the underlying technology — you could have a system that’s very adaptive but uses complex heuristics, or you could have state-of-the-art machine learning for suggesting a few very coarse recommendations, like what course to study next.
According to Yixue’s chief U.S. data scientist Dan Bindman, one can think of adaptivity with the following equation:
Adaptivity = Diagnosis + Prescription (crediting Dan Bindman)
A system, Bindman adds, is adaptive only if its “prescription” of study materials or other interventions is based on its “diagnosis,” or its assessment of the student via his or her initial performance, mouse clicks, or other behavior. Thus, one way to consider the adaptivity of a system is to consider the granularity of the prescriptions.¹
“Micro prescriptions” are hints, practice problems, and instructional material that are adaptively given to a student to help with a particular concept. A system offering micro prescriptions, such as Khan Academy, can adjust the difficulty level of material to match a student’s needs or provide an intervention such as a video or other readings. By contrast, “macro prescriptions” recommend what comes after learning a particular concept — this could be yet another concept, a new unit, or even an entirely different course. LinkedIn Learning, for example, recommends courses, while Fulcrum Labs and Knewton suggest concepts. It could also be a suggested concept or unit to review, as is common in many platforms that leverage the science of spaced repetition.
A fully adaptive platform would offer both micro prescriptions and macro prescriptions. Current systems are skewed toward one or the other, and few offer the most extreme of micro prescriptions — such as highly specific problem hints — or the most extreme of macro ones, such as a recommendation for which math course a student should be taking in the first place.
THE TAKEAWAY
Adaptive learning systems can mean a lot of different things, so it’s important to be more explicit about the type of adaptive learning we’re talking about. Otherwise, the variety in edtech products can make the definition so broad as to be meaningless.
When talking about and assessing products, it’s useful to consider both their technology and their adaptivity. One of the key takeaways here is although technology might get most of the attention, the actual adaptivity of a product is somewhat independent from the technology. A system could use the most advanced state-of-the-art machine learning, yet still be designed to make only a few generic recommendations on, say, what course to study next. Meanwhile, a platform with very simple algorithms could be designed to sort learners into numerous categories and give them specific hints, problems, and course recommendations accordingly.
All of this only begins to scratch the surface. At the end of the day, the success of adaptive learning platforms may depend as much on the outside world as the platforms themselves. In a real-life classroom, the teacher needs to believe in adaptive learning systems, or at least agree to use them. Teachers welcome products that they understand and control, rejecting features that they see as unhelpful. One former researcher at ALEKS noted that many teachers disabled ALEKS’s AI-based content recommendations and instead chose assignments and quizzes themselves. Indeed, the feature that turned off ALEKS’s adaptivity was one of the most popular features! To build something adaptive for the sake of being adaptive is to forget about the “learning” in adaptive learning and all the other factors around it. First and foremost, adaptive learning should be about the “learning.”
— —
Learn more about the Golub Capital Social Impact Lab at Stanford Graduate School of Business.
Follow us @GSBsiLab.
The research team included Elena Chen, Drew Bent, and Andrew Yeo.
With writing help from Louise Lee.
Footnotes
¹ Note that this framework has some similarities to the Pearson Decoding Adaptive framework, although that framework breaks up adaptivity across “adaptive content, adaptive assessments, or adaptive sequences” as opposed to micro and macro prescriptions as we do here.
