Introducing a new educational technological tool can cause the classroom to erupt in excitement, but excitement does not necessarily mean that the tool is enhancing learning. A new tool can be fun to use, convenient, or flashy, but what really matters is that students learn about and retain the information that is being taught. A difference can only be measured if there are figures to prove it.
The growth of the Education Technology Market is driven by the increasing use of digital tools in educational institutions. From personal learning assistants and online assessments to artificial intelligence tutoring and interactive resources, teachers and professors find themselves with an overwhelming number of options to choose from. But just because new learning technologies are available doesn’t mean that their use actually enhances learning. The most valuable tools are effective in solving a pressing educational need.
Start With the Learning Goal
First, it’s essential to identify how exactly that technology is supposed to boost things. Suppose a school adopting a math platform wishes that students will do a better job of solving multi-step problems. Suppose a university, after implementing a piece of digital assessment technology, wishes that the instructors will spot misconceptions earlier.
A language learning group could adopt speech recognition so students could have more practice in speaking.
We would look at different benchmarks of progress in these examples, none of which consist of students telling us whether they had fun on the technology.
Read: The Benefits of Advanced Education
Establish a Baseline Before Implementation
It’s impossible to prove a difference in a skill if the original ability isn’t understood.
A baseline is what students or participants currently ‘know’ or ‘can do’, and it forms the basis of all measures of progress. The choice of assessment will depend on the skill and aim; examples range from diagnostic testing, exams, pieces of work, skill-related activity, to skills tests, for example.
A baseline should be as close as possible to the stated learning aim to allow comparisons. A measure of frequency logged onto a reading page does not indicate where a student begins regarding their comprehension and does not enable you to see differences as regards reading comprehension unless the baseline was measuring comprehension based on the interpretation of argument, evaluation of evidence, or ability to identify evidence and make inferences from unseen text.
After you have a baseline,e it will be appropriate for that assessment to be used, or a comparable one, as a point of comparison for a difference measure later in the process.
Measure Learning, Not Just Engagement
Most computer-based learning systems collect a lot of data about student behavior. It is often possible for educators to observe data such as frequency of log-in, length of time spent with a lesson, number of tasks completed, or frequency of use of a particular feature. This data has the potential to be very useful, but it cannot be treated as directly indicative of learning.
The amount of time a student spends interacting with a system, or the number of exercises they complete on it, are not necessarily direct measures of the amount of learning that occurred during that interaction.
Data like this has the potential to become far more interesting when it can be correlated with external learning results. For example, educators may be able to measure whether students who actively use a learning resource also perform better on tasks external to that resource. In this case, interpretation must be cautious. Perhaps more engaged or academically self-confident students make greater use of learning resources, explaining performance outcomes rather than technology itself.
Compare Results Where Possible
Identifying the true impact of one issue with evaluating education technology is being able to distinguish the technology’s effect on students from other influencing factors. Students may get better due to changes in how teachers teach, extra support, or simply due to having more time to practice. If there’s no comparison group, it can be tough to know why results improve.
Ideally, educators compare students learning the same subject and using the new technology against the standard educational practice.
Evaluating outcomes before and after use of the technology with comparable populations gives a stronger indication of whether technologies really matter. Researchers use controlled trials where they randomly assign students to use a new technology or not, or a control group. Unfortunately, in classroom settings it’s usually not practical. Creating and measuring a comparison group in a well-structured study can yield valuable insight even when true experimentation isn’t possible.
Look at Retention and Transfer
Instant recall just after the lesson is just one aspect of what students will learn. If they score well during that lesson with a software, it might make no difference what students learn two or three weeks later. Delayed learning checks can be a better assessment for when students might have lasting learning.
And what we learn at school has often been taught to us before, so there are times when knowledge is about far more than software skills (there may be learning beyond it that’s also important.
Transfer is yet another thing we will want to look for in software. It’s good for a maths program to help students accurately get one type of equation correct when they see it, for example, but a better sign of deeper learning may be if that same underlying math principle could be applied to a different problem type. The gap between the two shows students simply know their way around software, as opposed to actually getting a grasp of the underlying principles that form knowledge.
Consider the Teacher’s Role
Technology is not neutral as it relates to teaching. Whether and how educators incorporate the use of technology could determine whether the results achieved were the ones they sought. A digital platform may, for example, ascertain that a cohort of students is not grasping fractions.
The insights provided are potentially very valuable; the educational impact, however, may turn upon what the educator does to complement or remediate.
Providing more instruction, opportunities to practice particular problems, or altering classroom pedagogy could make the difference between real achievement for students and the failure to resolve an isolated misconception. Therefore, whether or not technology was actually used will not be informative enough. The actual usage within the classroom will determine the outcome. In this light, two students might experience drastically different educational experiences with the same tool if two teachers choose to utilize it differently (through instruction and implementation, support and scaffolding, etc..
Put Industry Growth Into Perspective
The growing popularity of digital education has in turn driven demand for research into emerging technologies and sector trends. Organisations like Expert Market Research and Informes De Expertos provide market research which can offer more context to trends in educational technology as well as changing patterns of demand.
Such research can be helpful in understanding why certain technologies are getting attention and how the general field is developing. However, the question of market growth and educational effectiveness are two different ones.
A technology can become popular due to its accessibility, ease of implementation, or corresponding to overall tendencies in the field. None of these circumstances, however, proves its ability to make students learn better.
For educators, the difference between industry tendencies and actual results in learning outcomes remains crucial.
Check Who Benefits
Differences within individual users might also be overlooked. For example, a tool might cause an overall improvement and, while this would be represented in the average results, its effect might actually be higher for students who are already strong. Other possibilities are that it might support already weaker learners more in the form of more immediate feedback or additional practice.
Evaluations should then report whether this is the case depending on a learner’s level, needs, or technological support.
It is also imperative to take accessibility into account: a tool which is promising in a well-equipped classroom with good Internet will not be particularly good in a context where learners face internet connectivity issues, are deprived of computers,s and find certain interface designs difficult to use.
Pay Attention to Unintended Effects
Bringing technology into use will have effects that aren’t immediately evident in assessment. It can lead to students becoming overly reliant on automated hints and feedback. Teachers might find they are spending more time dealing with logins or technical issues, and that the use of digital activities may displace discussion, collaboration, or practical experience in the classroom.
There can also be unexpected benefits – such as that the use of the platform encourages independent learning, early diagnosis of student misconceptions, or improved access to certain materials.
The final evaluation needs to look at both (good and bad) unplanned impacts to appreciate the technology’s overall impact, not just to verify that it has worked.
Evaluate AI Tools With Particular Care
But along with the potential for personalized learning, automated feedback, and digital tutoring, AI systems raise further assessment concerns. In some cases, the responses provided by AI can vary depending on how a student structures a query; in others, the quality or usefulness of its feedback can differ. In both instances, therefore, instructors should consider whether not only assessment outcomes, but also student supports, were adequate and reliable.
A particularly sensitive concern regarding independent performance: if a student uses AI to get to a robust answer and cannot tell you why that is, have they actually learned something?
Assessments using no technological aid can often best determine if learning has truly occurred.
Allow Enough Time to See Meaningful Change
In some instances, results are seen quite rapidly; in others it takes much longer. If you’re implementing a tool to enhance fundamental memory recall, then significant differences could be expected within a couple of weeks. But an evaluation of a system intended to enhance critical thinking, problem-solving, or retention over the longer-term could need to be significantly longer.
Consequently, for these reasons, it can be advantageous to look at measurement over several times. It may include a measurement at the time of implementation/first use and further measurements at spaced intervals after implementation, looking at retention or transfer. This may help identify if initial gains are actually sustained over time.
Build a Simple Evaluation Framework
A good practical evaluation does not necessarily mean a huge research project. It can start with answering five simple questions: What learning problem does the tool target? What success would show that it has helped? Where are the students starting from? What would they be like without the technology? Are there other factors that might explain the results?
Answering these questions will help educators move from lofty assumptions to hard evidence.
And also make the difference between a tool used and an effective tool.
What Meaningful Success Looks Like
The value of an EdTech tool can only be assessed by what pupils know and are able to do at the end; it cannot be assessed by its own visual impressiveness.
Trustworthy evaluation takes baseline data, produces meaningful assessment, possibly accompanied by a comparison, the influence of retention, the transfer of learning and variation between learners, the teacher’s input, and the potential for unintended results and problems as well as successes.
Accessible, personalized, and measurable learning is possible with technology, only as far as any of those potentials manifest into enhanced educational learning.
Hence, the real question becomes not ‘Are students using the technology?’, but ‘Are students learning more as a result of this technology?’ By framing the question as ‘is it helping my students learn more effectively?’ instead of the ease and amount they use it, we will create a truer form of selecting the technologies deserving a place in learning.
Author’s Bio:
Roshan Kumar writes about education, technology,y and emerging trends shaping modern learning. His interests also include developments across higher education and the broader learning ecosystem, including research on the higher education market.
