Revolutionizing e-Assessment: The Transformative Role of AI Agents

From automating mundane tasks to creating entirely new paradigms for evaluation, AI agents are poised to revolutionise how assessments are designed, delivered, and experienced. Below, Pat Coates, eAA Chief Executive who believes AI Agents will revolutionise assessment content creation and test development (see our Predictions for 2025 article here) explains how AI agents apply to e-assessment, exploring opportunities, challenges, and the transformative potential of this technology.

I recently read an article by Doug Delaney around the 7 levels of AI Agents. The concept of AI agents is the most exciting development in AI as the models can’t get any bigger, surely not! However, they can get more sophisticated in how they work. In its simplest form, an AI agent is a mini program that undertakes tasks based around a series of rules. Generally, I would say that an AI agent is more sophisticated and would check and improve itself to perform better.  In addition, an AI agent could use multiple models from different AI tools to create something more robust and reliable. Here are some examples of different types of AI agents and their practical applications in e-assessment.

Rule-Based: Basic AI agents that follow predefined rules to perform specific tasks without learning or adapting. They operate reactively based on inputs.

E-Assessment example:

• A tool for writing multiple choice exam items
• A chatbot answering standard FAQs for test-takers, like “How do I reschedule my exam?”

Opportunities:

• Streamline repetitive tasks like grading straightforward questions or handling simple queries.
• Reduce administrative burden for testing organizations.

Task-Specific: Agents that excel at a specific activity by leveraging narrow AI capabilities, often outperforming humans in defined tasks.

E-Assessment Example:

• AI-powered plagiarism detection tools that analyze written responses for originality.
• Automated essay scoring systems trained to assess writing based on rubric-defined criteria.

Opportunities:

• Enhance consistency and speed in grading.
• Provide insights into candidate performance for specific skills like writing.

Contextual Analysis: AI agents that can process and synthesize complex data from multiple sources to adapt to dynamic scenarios.

E-Assessment Example:

• Proctoring systems that monitor test-takers using video feeds, keystrokes, and mouse movements to identify potential misconduct.
• AI recommending alternative assessment methods for students with disabilities, based on their historical data and preferences.
• Creating a test form based on a blueprint and item performance statistics

Opportunities:

• Improve test security by detecting nuanced cheating behaviors
• Adapt assessments to accommodate diverse needs, fostering inclusivity.

Empathetic AI: gents that interpret human emotions and intentions, enabling empathetic and personalized interactions.

E-Assessment Example:

• Virtual proctors that detect test-taker stress levels through facial expressions offering calming suggestions or flagging potential issues.
• AI tutors that adjust their feedback levels based on the emotional state of students, ensuring better engagement.

Opportunities:

• Enhance the test-taking experience by reducing anxiety and improving comfort.
• Build trust and rapport with candidates during assessments.

Self-Improving Systems: Agents capable of introspection and self-optimization, continuously improving their own performance without external intervention.

E-Assessment Example:

• An AI grading system that refines its scoring algorithms based on feedback from educators (e.g. If essay samples were also being human graded), adapting to align better with human evaluators, although whether the humans are better or not may be open to question.
• Proctoring systems that learn from past ‘mistakes’ to improve the accuracy of cheating detection.
• Creating equated test forms based on a blueprint and updating forms as more item data is made available as the test is taken.

Opportunities:

• Reduce manual calibration of AI tools, saving time and resources.
• Continuously enhance the fairness and accuracy of assessments.

Multi-Domain: General-purpose AI systems capable of undertaking multiple tasks across multiple domains, resembling human adaptability.

E-Assessment Example:

• An AI system that manages the entire assessment lifecycle, from test creation and candidate scheduling to scoring and reporting.
• Adaptive assessment and learning platforms that create personalized exams by analyzing a candidate’s performance in real-time.

Opportunities:

• Centralize assessment processes, reducing complexity for organizations.
• Provide a seamless and personalized assessment experience for candidates.

Blue Sky: Systems that go beyond perhaps what we think of as the norm now and are capable of creating new ways of doing things.

E-Assessment Example:

• An AI agent that redesigns assessment models to eliminate biases, optimize learning outcomes, and adapt to future educational needs and provides a meaningful measure of future success for a candidate
• AI systems that predict industry trends and create assessments that prepare students for emerging fields, we all know that most of jobs the next generation are going to do don’t exist yet.

Opportunities:

• Transform education and employment by redefining how knowledge and skills are assessed.
• Foster innovation in e-assessment, preparing candidates for the future.

Conclusion

AI Agents provide an opportunity to not only do the way we do things now better, but also to change the way we do things in the future. So rather than just making the horse go faster, we have the internal combustion engine which will change the way that we do things. It is not easy to do, as the education industry is very conservative, and any change is often seen as dumbing down by the media.

This also needs to be done in the context of ensuring fair valid and reliable assessments and taking into consideration any current and future legislation like the EU AI Act which means that some developments, such as the use of Empathetic AI, may simply not be an option.

The most important thing though is not being frightened to try and do things differently and working on the basis that what we do now is perfect. I mean we have driverless cars? Are we saying we can’t do exams in different ways than we do now?

About the eAA 

The e-Assessment Association plays a vital role in bringing together a global community of awarding organisations, assessment providers, technology experts, academics, and educational institutions dedicated to advancing the field of digital assessment. By fostering collaboration across these diverse stakeholders, we create a unique platform to share expertise, insights, and best practices that drive innovation and address the evolving needs of the assessment landscape. 

This extensive network of professionals equips the eAA with a deep understanding of assessment technologies and their applications, enabling us to address complex challenges such as ethical considerations, accessibility, and the rapid pace of technological evolution. As digital assessment becomes more central to education and professional qualifications, our role in shaping this transformation ensures that it remains inclusive, equitable, and aligned with the highest standards of integrity. 

Central to our mission is the eAA’s unwavering commitment to fostering confidence and trust in digital assessments. This includes addressing critical issues like data security, ensuring assessments are accessible to all learners, and building frameworks that inspire trust among stakeholders. By creating spaces for dialogue and collaboration, we help guide the global assessment community toward solutions that not only meet today’s demands but also anticipate future challenges, ensuring that digital assessments are secure, fair, and impactful for learners worldwide. 

Individuals can join the eAA for free – find out more here.