Using Cognitive Interviews to Validate an Interpretive Argument for the Ets Iskills Assessment

Evaluating the trustworthiness of Internet-based or other digital information has become an essential 21st century skill. The iSkills™ assessment, from Educational Testing Service (ETS), purports to measure such digital evaluation skills, along with other digital literacy skills. In this work, we use an argument-based approach to assessment validation to investigate the extent to which iSkills test scores can support inferences about the ability of college students to evaluate information in a digital environment. Eighty-eight undergraduates responded to iSkills assessment tasks and to more open-ended "naturalistic" tasks. These naturalistic tasks were designed to be like homework assignments that incorporate the critical evaluation of digital information. We observed weak-to-moderate correlations between scores, suggesting overlap in the skills assessed by the iSkills and the naturalistic tasks. Analyses of concurrent cognitive interviews (n=11 of 88) suggested distinctions between students' response processes to the assessment and naturalistic tasks. Although iSkills assessment tasks appear to elicit skills consistent with evaluation of digital information in the real world, students' responses to the naturalistic tasks demonstrated broader evaluation skills and less attention to the testing context. This study provides empirical validity evidence regarding ETS's iSkills assessment, as well as valuable insights into how undergraduates evaluate information in a digital environment.

Media Literacy Research Index

The purpose of this index is to bring together and promote international media literacy and media education research. The index is updated regularly, and you can also inform us about missing research via e-mail. Information presented here is all openly available in the Internet. Accessing full articles may require permission from the publisher.

Please, report to us if there is a malfunctioning link or if the information presented in the index is incorrect.

Medialukutaidon ja mediakasvatuksen viitetietokanta

Viitetietokannan tarkoituksena on tehdä näkyväksi medialukutaitoa ja mediakasvatusta käsittelevää tutkimusta. Tietokantaan on koottu yhteen kansainvälistä aiheita käsittelevää tutkimusta. Tietokantaa päivitetään säännöllisesti. Voit myös täydentää tietokantaa toimittamalla puuttuvan tutkimuksen tiedot sähköpostitse. Tietokannassa esitetyt tiedot ovat vapaasti saatavilla Internetissä. Kokonaisten artikkelien lukeminen saattaa edellyttää lupaa kustantajalta.

Ilmoitathan sähköpostitse, mikäli jokin linkki ei toimi tai esitetty tieto on väärin.