Grade Expectations

The idea of some “one size fits all” assessment for the nation is daunting when you consider the regional variations of this vast nation. English language learners (ELLs) pose a particular challenge to test developers, who not only have to consider that questions should be as culturally appropriate for a child in Des Moines as one in Santa Ana but also have to take into account the laws of certain states designed to level the assessment playing field.

Currently the Common Core State Standards (CCSS) only cover math and English. While it may be unrealistic to expect Spanish translations of the English assessments, unless we want to assess literacy and comprehension in any language, translations of math items and inclusion of cross-cultural contexts are crucial for ensuring that ELLs perform as well as possible on CCSS assessments.

The two major consortia of CCSS assessment, the Smarter Balanced Assessment Consortium (SBAC) and the Partnership for Assessment of Readiness for College and Careers (PARCC), claim to be working towards implementing accommodations for ELL students.

Smarter Balanced has set up a special advisory committee and published a series of reports on implementing accommodations. Magda Chia, the consortium’s director of support for under-represented students, is quick to point out that no modifications are being made to assessments to cater for ELLs: “From the very beginning of our item development, we’ve followed best practices to ensure that ELLs are not at a disadvantage.”

Such practices include ensuring that the context of its questions does not favor any particular group and the use of performance tasks based on classroom activity in every test. Although a classroom teacher does not have access to the performance task, the classroom activity will allow the teacher to review contextual information and vocabulary with students prior to testing. Another key tool is a language-complexity rubric (see below) to help make the vocabulary, grammar, and syntax in a test item appropriate for the intended grade, standard, claim, target, and difficulty so that the item is most accessible.

Language Complexity Rubric created by Gary Cook and Rita MacDonald for the consortia
Jamal Abedi and Nancy Ewers helped SBAC develop a common accommodation system for ELLs to reduce or eliminate variations across states, including a system to help states choose which accommodations to use for ELLs based on validity and effectiveness.

Accommodations are sorted into the categories use, use/low evidence, not use, and unsure, according to the following definitions:

Use
An accommodation labeled under this category is supported by existing research as being effective in making assessments more accessible and/or valid (i.e., does not alter the focal construct) for ELLs/SWDs (students with difficulties). The literature shows multiple studies that provide consistent results supporting the effectiveness and validity of the accommodations used for ELLs and SWDs.

Use/Low Evidence
This label is used mainly for accommodations for students with disabilities that require additional research-based evidence on their effectiveness and validity.
However, given a minimal level of research supporting these accommodations combined with expert opinion and direction for use in federal disability laws, use of the accommodation is recommended when indicated in a student’s individualized education program (IEP).

Not Use
An accommodation is labeled as “not use” when there is enough consistent evidence suggesting the accommodation is not effective and alters the focal construct; thus, the validity of assessments under this accommodation is questionable.

Unsure
The research-based evidence is inconclusive or there is not enough evidence to make a judgment about the effectiveness and/or validity of this accommodation. However, neither is there enough evidence to completely reject this accommodation as ineffective or invalid. (Source: www.smarterbalanced.org/wordpress/wp-content/uploads/2012/08/Accomodations-for-under-represented-students.pdf)

Smarter Balanced is also planning to offer its member states a range of translation options for the math assessments. According to Chia, translations will be “stacked” so students can view an item in the their native languages first and then scroll down to view the item in English. There will also be an embedded glossary for construct-irrelevant terms in English and other languages which may even include an audio component. In addition to Spanish, Vietnamese, Arabic, and American Sign Language are slated to be available.

PARCC has published a manual of accessibility features and accommodations for the assessments that measure how well students master the CCSS. PARCC serves approximately 23 million students in 21 states. The manual offers guidance to districts and decision-making teams to ensure that PARCC mid-year, end-of-year, and performance-based assessments produce valid results.

The manual outlines accessibility features for all students that are embedded in the computer-based test platform and accommodations for English language learners (ELLs) and students with disabilities. PARCC guidelines call for a Personal Needs Profile (PNP) to be completed for every student to document which additional supports will enhance accessibility.

PARCC describes which special accommodations should be made for ELLs based on their proficiency levels, either beginner, intermediate, or advanced. ELL-specific accommodations include extended time for taking the test; general administration directions clarified, read aloud, and repeated if necessary in the student’s native language by a test administrator; scribe or speech-to-text accessibility so that students can dictate their responses to mathematic assessment in English; and access to a word-to-word English/native-language dictionary. The manual describes ideal test settings and allows for small-group or individual testing when appropriate.

PARCC’s manual also outlines the decision-making process for selecting, using, and evaluating accommodations for ELLs. The steps range from expecting all students to achieve academic grade-level content standards to evaluating and improving the use of accommodations. Ultimately, PARCC’s guidelines aim to ensure that all students have a fair shot at performing their best on the PARCC assessments.

Despite these efforts, there are a plethora of challenges involving accommodations for ELLs facing the assessment consortia.

Typical accommodations for ELLs include extra time allocated for completing items, repeating instructions, and translations of exam instructions and items. However, translating an exam is a complex process that demands many considerations. Most importantly, translations are designed to give the ELL students a fair opportunity to demonstrate their knowledge. If the translation is not accessible to the student due to language complexity or dialect, it will ultimately not be an effective accommodation. Therefore, after deciding that a translation should be offered to students, the preparation of the translation must be performed with significant care.

The heterogeneity of ELLs also proves to be a challenge. Even though there are a mere handful of languages that are most in demand, variations within these languages call for different translations within the same language. For example, the Spanish of a student from Argentina is quite distinct from the Spanish of a student from Puerto Rico. Furthermore, their cultural contexts could hardly be more distinct.

Aside from disparate language abilities due to dialect, student proficiency in his or her native language and history of schooling in English also heavily determine whether a translation will be a useful accommodation. As Guillermo Solano-Flores points out in his report for Smarter Balanced entitled Translation Accommodations Framework for Testing English Language Learners in Mathematics, “…because ELLs vary tremendously in their reading and writing proficiencies in English and have multiple schooling histories in English, many of them may not benefit at all from translation accommodations. Classifications of students according to a few levels of English proficiency are not sensitive to the ability of ELLs to read and write in English, especially in the context of academic English. Unduly assigning this form of accommodation to ELLs may be more harmful than beneficial.”
One political challenge is that each state has its own policy on ELL assessment. In some states, ELLs have the right to a translation of test and item instructions, and other states prohibit translations. Therefore, it is impossible to create a one-size-fits-all accommodation system. Each state has its own special considerations and regulations regarding ELLs, which in turn take the “common” out of Common Core. The consortia must create ELL accommodations state by state, which require time.

The consortia’s timelines for implementing their CCSS assessments do not allow extra time for the development of accommodations for ELLs, so the pressure is on. Preparing and implementing the accommodations requires first that the testing items be finalized. Expect a flurry of activity over the next few months.