HSE Researchers Benchmark First-grade Performance of Russian and British Schoolers
The HSE Institute of Education researchers have updated the iPIPS school-entry ability evaluation tools to allow for equated benchmarking of primary-schooler progress across countries. The first results have been obtained from a representative sample of Russian and British first-graders. The study is published in Assessment in Education: Principles, Policy & Practice.
It was back in 2014 when a team of researchers from the HSE Institute of Education and Durham University (UK) came up with an idea of assessing school first-grader performance on an international basis. At that pilot point, the study primarily sought to validate whether iPIPS (The International Performance Indicators in Primary Schools) could be applied for such internationalized evaluation, and also to try and benchmark the progress of English, Scottish, and Russian first-year schoolers under a uniform cross-country framework.
The iPIPS is a unique international study of children school-entry ability in a number of learning and development areas. It serves as a baseline evaluation tool and provides policy-relevant information about schooler progress during the first year, helping to contextualise PISA data and giving informed answers to questions of learning outcomes and achievement that children make on starting school.
Led by the Centre for Evaluation and Monitoring at Britain’s Durham University, the iPIPS is based on an assessment system which has been developed over the last 20 years and has been used with more than two million children in various countries. The iPIPS assessment system now comprises several testing modules covering a diverse set of skills, including math, reading, phonemic, and vocabulary proficiency.
In the course of the project, the HSE Institute of Education (IOE) research team has succeeded modifying the original iPIPS evaluation toolset in a way to make international schooler performance data correlate and compare, allowing an equated and fair analysis within a uniform reference system offsetting variances in the sociocultural context, including education curriculum. The updated toolkit has been tested in benchmarking Russian and the UK first-graders’ progress in math.
Numerous factors have been considered for such a benchmarking system to become practical and deliver valid and representative scorings across nations and educational paradigms, the researchers state. Thus, dealing with the iPIPS methodology, apart from the said major factors, age deviances also need to be accounted for, as children in the UK normally start school at the age of 4–5, while the respective average in Russia is 7.
“To allow for relevant comparison of schooler entry ability and progress, we had to cater for all the factors that could have made Russian or British schoolers look stronger performers in certain tests just because of the age/curriculum variance alone. Otherwise, the resulting scores would have been misleading. For example, since Russian children start school at a later age, they are typically better at multi-digit numerals than their British peers. Or, by contrast, British first-graders are better at solving logical tasks. Further, British children tend to progress faster over the first school year since, as a rule, younger kids are quicker learners, so, while Russians generally display stronger entry ability, their British peers basically catch up as they complete the first grade,” the researchers say.
As such, to ensure adequate benchmarking, a number of ‘neutralizing’ adjustments have been introduced by altering test complexity upwards or downwards where the original iPIPS evaluation framework was found most sensitive to the above-mentioned school starting age, curriculum, and other similar ‘extra-personal’ factors. At the same time, the remaining ‘core’ tests, making some 30% in the total amount, were applied as unchanged to all participating children and served to derive ‘net’ progress by benchmarking entry and end-of-grade scores.
“This has been truly the one-of-a-kind academic endeavour enabling straightforward, realistic, and convincing evaluation of first-graders’ performance across international communities, and particularly of their basic math skills at school entry. Sure, the results arrived at by such equated benchmarking place us better to provide compelling answers to the question of the most apt age for a child to start primary school. Going forward, we will keep tracking the sample’s performance as the children are done with their third and fourth grades in the spring of 2017 and 2018, respectively. This will enable even a more representative and all-embracing analysis, and will ensure more solid grounds to derive meaningful conclusions from assessing longer-term learning outcomes,” the study co-author and Director of the IOE Centre for Education Quality Monitoring Elena Kardanova comments.