How do you evaluate digital learning in different formats?

1 Learning objectives

Before you start evaluating any digital learning format, you need to define your learning objectives. What are the specific skills, knowledge, or attitudes that you want your learners to achieve or improve by the end of the digital learning experience? How will you measure their progress and performance? Learning objectives should be SMART: specific, measurable, achievable, relevant, and time-bound. They should also align with the format and content of the digital learning, and the needs and preferences of your target audience.

2 Evaluation models

There are many evaluation models that you can use to assess the effectiveness and impact of digital learning in different formats. One of the most widely used models is Kirkpatrick's four-level model, which evaluates digital learning based on four criteria: reaction, learning, behavior, and results. Reaction refers to how learners feel about the digital learning experience, such as their satisfaction, engagement, and motivation. Learning refers to how much learners have acquired or improved their skills, knowledge, or attitudes as a result of the digital learning. Behavior refers to how learners apply what they have learned to their work or life situations, and how they change their actions or habits accordingly. Results refer to how the digital learning affects the organizational or societal outcomes, such as productivity, performance, quality, or revenue.

3 Evaluation methods

When evaluating digital learning, various methods can be used to collect and analyze data. Surveys and questionnaires are a great way to get feedback from learners regarding their reaction to the digital learning and self-reported learning outcomes. Tests and quizzes are useful for measuring the learning outcomes of learners, such as their comprehension or application of the digital learning content. Interviews and focus groups can be used to gain in-depth insights from learners about their behavior and results after the digital learning, as well as their suggestions for improvement. Additionally, observations and performance reviews are helpful for observing and evaluating the behavior and results of learners in their work or life contexts, as well as how they apply what they have learned from the digital learning. Online tools such as Google Forms, Quizlet, Zoom, Google Docs, Sheets, or Slides can be utilized for these methods.

4 Evaluation tools

Evaluating digital learning in different formats can be made more efficient and effective with the use of online tools. A Learning Management System (LMS) is a software platform that allows for the creation, delivery, management, and tracking of digital learning in various formats, such as e-learning courses, webinars, podcasts, or videos. It also has analytics, reports, dashboards, and feedback features to evaluate the reaction, learning behavior, and results of learners. Examples of LMS include Moodle, Canvas, or Blackboard. A Learning Record Store (LRS) is a database that stores and retrieves data about the learning activities and experiences of learners across different digital learning formats and platforms. It also has xAPI (Experience API) or Tin Can API features to collect and analyze data about the interaction, engagement, completion, and achievement of learners. Examples of LRS include Learning Locker, Watershed, or Grassblade. Lastly, a Learning Experience Platform (LXP) is a software platform that provides personalized and adaptive digital learning experiences for learners based on their preferences, goals, and needs. It also has features such as recommendations, gamification, social learning, or microlearning to enhance the reaction, learning behavior, and results of learners. Examples of LXP are Degreed, EdCast or Docebo.

5 Evaluation tips

When evaluating digital learning in different formats, it is important to plan your evaluation strategy beforehand. Consider your objectives, methods, tools, and indicators and make sure they are aligned with the learning objectives, format, and audience. Utilize both quantitative and qualitative data to measure the extent and magnitude of the learning outcomes as well as understand the reasons and context behind them. Additionally, use a feedback loop to collect feedback from learners and stakeholders, analyze it, and use it to make changes and improvements to the digital learning design, delivery, and evaluation.

6 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?