Best Summative Assessment Project Award

The English for Academic Purposes (EAP} module is compulsory for all Pathways students taken by over 5000 students each year, across nine geographically dispersed Kaplan Pathway Colleges in the UK. Its assessment is incredibly high-stakes, as it is a pre-requisite to securing progression to university. Prior to the transformation, the assessment was paper-based and posed challenges with maintenance and management due to its high-stakes nature and the continuous need for resource-heavy production of multiple new assessment versions every year. The decision to transition to a digital, adaptive assessment platform was driven by the desire for greater sustainability of assessment processes for Kaplan Pathways, increased results precision and improved personalization of the assessment experience for Pathways students. The new approach retained the existing four discrete skills testing model used in the EAP module but replaced paper-based examinations with online adaptive KTE assessments for listening, reading, and writing, while maintaining face-to-face speaking examinations. The adaptive nature of KTE, powered by Item Response Theory (IRT), resulted in personalized testing experiences for students, reduced error rates and efficiency gains through instant, automatic scoring. Moreover, the new approach improved operational sustainability, increased flexibility in test administration and enhanced overall assessment security. Addressing potential barriers to adoption, the Kaplan International Pathways and KTS engaged staff and students in the validation process. Staff validated the alignment of the new assessments with the EAP module learning outcomes through careful content reviews and mapping of test constructs with the EAP curriculum. Students were consulted on their experience with the new assessment strategy, and their feedback revealed that they overwhelmingly endorsed the online delivery, remote accessibility and adaptive nature of the assessments. After an initial rollout and scaling across all colleges, the institution observed incremental improvements in student outcomes and satisfaction levels, attributing these to reduced stress and increased personalization of the testing experience. The positive impact on student performance and module feedback led to enhanced confidence from university partners and widespread adoption of the KTE adaptive test for additional various purposes within Kaplan International Pathways, including admissions and progress testing. Overall, the case study highlights the successful implementation of computer-adaptive assessments in high-stakes English language modules, resulting in improved sustainability, precision, personalization, and student satisfaction. It underscores the effectiveness of adaptive testing in enhancing learning measurement and test-taking experiences, ultimately contributing to the institution's educational goals and partnerships.

Kasetsart University (KU) stands as a global leader in academia, offering an extensive range of undergraduate, postgraduate, and research programs spanning various fields, including Business Administration, Agriculture, Humanities, Economics, Engineering, and Agro-tech. KU aspires to be a world-class institution dedicated to education, research, and innovation aimed at achieving sustainable societal development rooted in indigenous knowledge. The management sought a unified, centrally deployed, cost-effective solution capable of catering to diverse school needs. They aimed for a secure test delivery mechanism to safeguard their departmental item banks and tests, scalability to accommodate a large number of students taking tests simultaneously and timely results with comprehensive visibility for top management. Considering that certain programs were offered in both Thai and English and some exclusively in Thai, the university required a solution that would enable tests and metadata in both languages, ensuring seamless administration across language preferences. Conducting thorough market analysis, KU explored both open-source and proprietary software options. Teaming up with Excelsoft, they implemented a new platform enabling nationwide access for students to take exams online, even in areas with unreliable internet connectivity. Leveraging the fact that over 90% of students already owned iPads, KU adopted a Bring Your Own Device (BYOD) model, facilitating greater participation in exams while reducing costs associated with providing computers. The implementation of this solution led to a swift adoption of digital summative exams, with over 72,000 items and 3,000 exams efficiently generated, resulting in the submission of over 733,000 summative exams within three years. Presently, more than 2 million students can take the university entrance exam using the digital platform.

EQAO transformed Ontario’s provincial large-scale assessments in response to two major catalysts: the urgent need for reliable data on student learning created by the COVID-19 pandemic, and the release of new curriculum over the same period. The transformation leveraged relevant, digital assessment models and provided an engaging, accessible assessment experience for all students. The assessments continue to be aligned to the Ontario Curriculum and follow best practices for psychometric measurement and data quality. The modernization also included substantial enhancements to assessment reporting by leveraging interactive dashboards that improve access to timely and meaningful data to inform practice and improve student achievement. EQAO ensured the success of the end-to-end transformation through active collaboration with and support to partners in the education system; a steadfast focus on the needs of all students; and an agile and evidence-informed approach to implementation. An example of how EQAO’s transformation is impacting the education system in Ontario is the ongoing efforts of educators and administrators to improve math learning in the province. Following the pandemic, and the launch of a revised mathematics curriculum, year-over-year tracking of mathematics data was necessary to guide decision making at a school, school board and provincial level. EQAO provided the education sector with sub-score reports that give educators information on how students do on items mapped to specific strands and skills of the revised mathematics curriculum. For example, schools can see how students performed on geometry, algebra and data sections of the assessment. This information allows them to make data-informed decisions around instruction and professional development as educators become accustomed to the new curriculum. The reports were highlighted in 2023 as part of the Ministry of Education Mathematics Achievement Action Plan as an important way to determine areas where students may be struggling in Mathematics. EQAO’s School Support Team has been supporting school in accessing and using the reports to better understand math results.