OECD mapping of longitudinal information systems: preliminary results

Size: px

Start display at page:

Download "OECD mapping of longitudinal information systems: preliminary results"

Albert Roberts
6 years ago
Views:

1 «Educational information systems for innovation and improvement», New York City, 14 Oct 2010 OECD mapping of longitudinal information systems: preliminary results Stéphan Vincent-Lancrin OECD Centre for Educational Research and Innovation (CERI)

2 What did we ask for? 1. Goals of the system 2. Data model (data collected for students, schools, teachers, enrolments, assessment, etc.) 3. Coverage and frequency of collection 4. Data linkages 5. Quality processes 6. Access and Privacy 7. Comparison possibilities 8. Accountability usage 9. Instructional support, networking facilities and professional development 10. Other features (extendable platform, etc.)

3 General 33 answers from 19 countries (so far ) Most systems are recent: 6 before 2000, 10 after 2005 Jurisdictional and public for most of them (national, regional, district), but also for school networks Big diversity: methodology, coverage, access, frequency of data collection, speed of feedback, use, etc. Systems: Australia, Belgium (Fl.), France, Hungary, Italy, Japan, Korea, Mexico, New- Zealand, Slovak Rep., Spain, Sweden, Netherlands, United Kingdom, United States No system: Finland, Germany, Luxembourg, Switzerland

4 Average ranking of system goals (respondents asked to rank goals from 1st to 4th)

5 Unique identifier Systems Student School Teacher Course Australia My School Y Y N N Australia - NSW SMART Y Y N N Australia - Victoria Ultranet Y Y Y P Belgium (Fl.) FNAP Y Y N N Belgium (Fl.) No name Y Y Y N France No name Y Y Y Y Hungary NABC Y Y N N Italy Anagrafe Y Y N N Japan NAAA N Y N N Korea Schoolinfo N Y N N Korea NEIS Y Y Y Y Mexico Mako V. 2.0 Y Y Y Y New Zealand e-asttle Y Y Y N Slovak Republic No name Y Y P P Slovak Republic No name N N N N Spain HE P Y N P Spain Schooling Y Y Y N Sweden ETF Y Y Y N Sweden No name Y Y Y Y The Netherlands COOL Y Y N N The Netherlands CITO Y Y Y Y The Netherlands BRON Y Y N Y United Kingdom RAISEonline Y Y N Y US - Charlotte Charlotte NC Y Y P Y US - NYC ARIS Y Y Y Y

6 Data coverage & frequency (green: daily collection; yellow: yearly or at specific intervals (age, grade, etc.)) School Teacher Course and class Schoollevel Student enrolment Attendance Schoollevel National or assessmen regional t and assessment grades End of educational cycle/stage grades Australia FT NC NC FT Y Y NC FT FT Australia - NSW FT NC Y Y Y NC NC Y NC Australia - Victoria D D D D D D D NC NC Belgium - FL. NC NC NC NC NC NC NC Y NC Belgium - FL. D D NC D FT FT Y NC Y France Y M Y Y Y FT Y Y Y Hungary Y NC NC Y NC NC NC Y NC Italy FT NC FT FT FT FT FT Y Y Japan Y NC SG SG SG NC NC NC NC Korea FT FT FT FT NC NC FT FT NC Korea D D D D D D D D D Mexico D D D D D D FT FT FT New Zealand D D D D NC NC NC NC NC Slovak Republic Y Y Y Y Y Y Y Y Y Slovak Republic NC NC NC NC NC NC NC Y NC Spain Y Y Y Y Y NC Y Y Y Spain NC Y NC Y Y Y Y FT Y Sweden Y SG SG Y Y NC SG FT SG Sweden Y Y Y Y Y NC NC Y Y The Netherlands FT NC FT FT NC NC FT FT FT The Netherlands FT NC NC FT NC NC FT NC FT The Netherlands M NC NC Y M NC NC Y Y United Kingdom FT NC FT FT FT FT FT FT FT US - Charlotte D D D D D D D D D US - NYC D D D D D D D D D

7 Student assessment data Systems Course grade Course credit Course qualitative assessment Any soft or generic skills assessment Formative assessme nt Student percepti on National / Regional assessme nt Australia My School Y N Y Y N Y Y N Australia - NSW SMART P N N N Y Y Y N Australia - Victoria Ultranet Y N Y Y Y N N N Belgium (Fl.) FNAP N N N N N Y N Y Belgium (Fl.) No name Y Y N N N N N N France No name Y N N Y Y N Y N Hungary NABC Y N N N N N Y N Italy Anagrafe Y N N N N N Y N Japan NAAA N N N N N Y na N Korea Schoolinfo Y N N N N N P N Korea NEIS Y Y Y N N P Y na Mexico Mako V. 2.0 Y N P Y P Y P Y New Zealand e-asttle N N N Y Y N na na Slovak Republic No name Y P N P Y N Y P Slovak Republic No name P P P P Y Y Y P Spain HE N P P N N N na na Spain Schooling Y N N N Y N P Y Sweden ETF Y Y N Y N Y Y Y Sweden No name Y N N N N N Y N The Netherlands COOL Y N N Y N N na na The Netherlands CITO Y N N N Y N N Y The Netherlands BRON Y N N N N N Y N United Kingdom RAISEonline Y N N N Y P Y N United States - Charlotte Charlotte NC Y Y N N P P Y Y United States - NY ARIS Y Y N N Y N Y Y Large scale test

8 Speed of feedback (Green: quick feedback (< 1 month); Yellow: medium (1-3 months); white: slow) Systems Administrator School Teachers Parents Students Researchers Australia My School > 6 M > 6 M > 6 M 4-6 M 4-6 M 4-6 M Australia - NSW SMART 1-3 M 1-3 M 1-3 M > 6 M > 6 M 1-3 M Australia - Victoria Ultranet RT RT RT RT RT No Belgium (Fl.) FNAP > 6 M > 6 M > 6 M > 6 M > 6 M 48 H Belgium (Fl.) No name 48 H > 6 M > 6 M > 6 M > 6 M 1-3 M France No name 4-6 M 4-6 M 4-6 M 4-6 M 4-6 M > 6 M Hungary NABC > 6 M > 6 M > 6 M > 6 M > 6 M > 6 M Italy Anagrafe 48 H 48 H RT RT RT > 6 M Japan NAAA 4-6 M 4-6 M 4-6 M 4-6 M 4-6 M 4-6 M Korea Schoolinfo 1-3 M 1-3 M 1-3 M 1-3 M 1-3 M 1-3 M Korea NEIS RT RT RT RT RT 1 W Mexico Mako V. 2.0 RT RT RT RT RT RT New Zealand e-asttle RT RT RT RT RT > 6 M Slovak Republic No name 1 W - 1 M 1 W - 1 M 1 W - 1 M 1 W - 1 M 1-3 M 1 W - 1 M Slovak Republic No name RT RT RT RT RT RT Spain HE > 6 M RT > 6 M > 6 M > 6 M > 6 M Spain Schooling > 6 M > 6 M > 6 M > 6 M > 6 M > 6 M Sweden ETF 4-6 M 4-6 M 4-6 M 4-6 M 4-6 M 4-6 M Sweden No name 4-6 M 4-6 M 4-6 M 4-6 M 4-6 M 4-6 M The Netherlands COOL RT RT RT RT RT 4-6 M The Netherlands CITO 48 H RT RT > 6 M > 6 M 1-3 M The Netherlands BRON > 6 M > 6 M > 6 M > 6 M > 6 M > 6 M RAISEonlin e 4-6 M 4-6 M 4-6 M 4-6 M 1 W - 1 M 4-6 M Charlotte NC RT RT > 6 M RT > 6 M > 6 M United Kingdom US - Charlotte US - NYC ARIS RT RT RT RT RT RT

9 Common patterns in access and access restrictions Administrators: no access to individual data (not anonymised) School principals: their school and public data Teachers: their students, sometimes their school, and public data Students: their data and other public data Parents: their child and other public data Researchers: No access or strictly regulated access to anonymised data, on a case by case basis

10 Comparison tools Schools Students Predicted pattern Class Compar Same ison Others charact tool eristics Others Same Student chara past cterist perform ics ance Student statistics Others Same chara cterist ics Teachers Others Australia Y Y Y N N N N N N N Australia - NSW Y Y Y Y Y N Y Y N Y Australia - Victoria Y Y N Y Y N N Y Y N Belgium - FL. Y Y Y N N N N N N N Belgium - FL. P P P N N N N N N N France Y Y Y N N N Y N N N Hungary Y Y Y N Y P N Y N N Italy N Japan N Korea N Korea Y Y Y Y N N Y Y Y P Mexico Y Y Y P P P P Y Y Y New Zealand Y Y Y Y Y N N N N N Slovak Republic N Slovak Republic N Spain N Spain N Sweden Y Y Y Y Y Y Y Y Y N Sweden N The Netherlands P Y N N N N N N N N The Netherlands Y P P Y Y P P Y Y N The Netherlands Y Y Y Y Y N N N N N United Kingdom Y Y Y Y Y Y Y P P Y US - Charlotte P P P P P P P P P P US - NYC Y Y Y Y Y N N Y Y Y

11 Comparability (quality) of assessment data National or regional assessment School level assessment Systems MCQ Other technique MCQ Other technique Australia My School Y Y Y Y Australia - NSW SMART Y Y Y N Australia - Victoria Ultranet N N N N Belgium - FL. FNAP Y Y N N Belgium - FL. No name N N N N France No name N Y N Y Hungary NABC N N Y N Italy Anagrafe N N N N Japan NAAA N Y N N Korea Schoolinfo Y Y N N Korea NEIS Y Y N Y Mexico Mako V. 2.0 Y Y Y Y New Zealand e-asttle N Y N Y Slovak Republic No name Y Y P Y Slovak Republic No name Y Y P Y Spain HE N N N N Spain Schooling N N N N Sweden ETF N N N N Sweden No name N N N Y The Netherlands COOL N N N N The Netherlands CITO Y N Y Y The Netherlands BRON N N N Y United Kingdom RAISEonline N Y N Y US - Charlotte Charlotte NC Y N Y N US - NYC ARIS Y Y N Y

12 Quality process Systems All can report mistakes Enrolment data Ex-post quality check Ex-ante support and incentive s All can report mistakes Assessment data Ex-post quality check Automat ed Ex-ante support and incentives Australia My School N Y Y Y Y Y Y Australia - NSW SMART Y Y Y Y Y Y Y Australia - Victoria Ultranet Y Y N Y N N N Belgium - FL. FNAP N N N N N N Y Belgium - FL. No name Y Y Y Y Y Y Y France No name N Y Y N Y Y Y Hungary NABC Y Y Y Y N Y Y Italy Anagrafe N Y N N N Y Y Japan NAAA N N N N N N Y Korea Schoolinfo N N N N N N N Korea NEIS Y Y Y Y N Y Y Mexico Mako V. 2.0 P P P Y N N Y New Zealand e-asttle N Y Y Y N Y Y Slovak Republic No name Y P Y P Y Y Y Slovak Republic No name Y Y Y P Y Y Y Spain HE N P N N N P N Spain Schooling N P N N N P N Sweden ETF N Y Y N Y N Y Sweden No name N Y Y N N Y Y The Netherlands COOL N N N Y N N Y The Netherlands CITO N Y Y N N Y Y The Netherlands BRON N N N N N N N United Kingdom RAISEonline N N N Y Y N N US - Charlotte Charlotte NC N P Y N P N Y US - NYC ARIS Y Y Y Y Y Y Y

13 Use for accountability High stake assessment Use for Marking or ranking Student Teacher School School district Teacher School principal School Australia My School Y N Y Y N N Y Australia - NSW SMART Y N Y Y N N N Australia - Victoria Ultranet N N N N N N N Belgium - FL. FNAP N N N N N N N Belgium - FL. No name N N Y N N N N France No name N Y Y Y N N N Hungary NABC N N N N N N N Italy Anagrafe N N N N N N N Japan NAAA N N N N N N N Korea Schoolinfo N N N N N N N Korea NEIS N N N N N N N Mexico Mako V. 2.0 Y Y Y P Y Y Y New Zealand e-asttle N N N N N N N Slovak Republic No name P P Y Y N N N Slovak Republic No name N N N N N N N Spain HE N N N N N N N Spain Schooling N N N N N N N Sweden ETF N N N N N N N Sweden No name N N Y Y N N Y The Netherlands COOL N N N N N N N The Netherlands CITO N N N N N N Y The Netherlands BRON Y N Y N N N Y United Kingdom RAISEonline N N Y N N N Y US - Charlotte Charlotte NC N N Y N Y Y Y US - NYC ARIS N P Y N N N Y

14 Other features Teacher access to instruct. material Parent or student access to instruct. material Communities of practice Possibility of Integration with other classroom applications Third-party applications Extension of core data model with third-party applications Australia My School P Y Y P Y N Australia - NSW SMART Y P Y N N Y Australia - Victoria Ultranet Y Y Y P Y N Belgium (Fl.) FNAP N N Y N N N Belgium (Fl.) No name N N N N N N France No name Y N P N N Y Hungary NABC N N N Y Y Y Italy Anagrafe N N N N N N Japan NAAA Y Y Y N N N Korea Schoolinfo N N N N N N Korea NEIS P N N P N N Mexico Mako V. 2.0 Y Y Y Y Y N New Zealand e-asttle Y N Y Y Y N Slovak Republic No name N N N P P P Slovak Republic No name N N N N N Spain HE N N N N N N Spain Schooling N N N N N N Sweden ETF N N N N N N Sweden No name N N N N N N The Netherlands COOL N N N N N N The Netherlands CITO N N N N N N The Netherlands BRON N N N N N N United Kingdom RAISEonlin e N N N N N Y Charlotte NC N N N Y Y N US - Charlotte US - NYC ARIS P P Y P Y P

15 First impressions (Caution: more quality checks needed) Most systems could improve the speed of feedback to teachers and schools Comparison tools, especially at the student level, could be further developed Few systems integrate in a deep way data systems and learning management systems: room for using more the systems as platforms (for teachers and students instructional support and networking) Too little reporting of «non-traditional» learning outcomes (generic and soft skills)

16 THANK YOU

ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT

ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT Pursuant to Article 1 of the Convention signed in Paris on 14th December 1960, and which came into force on 30th September 1961, the Organisation