ACTFL and Language Testing International (LTI) introduce an automated scoring system for the ACTFL Assessment of Performance toward Proficiency in Languages (AAPPL) Spanish Interpersonal Speaking and Listening (ILS) component. This pioneering automated scoring system was first introduced in 2023 for the AAPPL Spanish Presentation Writing (PW) component and was the culmination of nearly ten years of dedicated research and development. This system represents a significant advancement in language assessment rating systems and enhances the accuracy and efficiency of the AAPPL evaluation process.
The AAPPL ILS component evaluates the spontaneous listening and speaking abilities of learners in grades 3-12, without access to revisions and editing tools. It serves as a crucial measurement of listening and speaking language performance, which can be used to more effectively guide goal setting, curriculum development, and instruction.
The automated scoring system, also known as machine scoring, will be launched January 20 for live Spanish ILS tests, in conjunction with ACTFL-certified human raters. This integration of machine scoring alongside human ratings means that all Spanish ILS tests will now be double-rated. This approach will benefit AAPPL test takers and users by providing an additional rating while also enhancing our quality assurance processes. In addition, the rating process will be more streamlined by identifying tests that require additional human rating. The machine scoring system assesses the AAPPL ILS responses based on several parameters, as described in the ACTFL Performance Descriptors, including functions (global tasks the learner can perform in the language), context/content (situations and topics the learner can comprehend and discuss), text type (comprehension and production capabilities for performing functions), and comprehensibility (accuracy and vocabulary use).
LTI's machine scoring system has undergone iterations of rigorous training using rating data from ACTFL-certified AAPPL raters. Extensive research, conducted by Dr. Erik Voss of Teachers College Columbia University, has validated the accuracy of machine scoring compared to human raters over multiple years of administration.
ACTFL and LTI remain committed to maintaining currency and relevance in language testing and contributing to the advancement of research in non-English, second language machine scoring procedures.