Why some Spanish assessments aren’t what they appear

You’re probably familiar with the idiom “The emperor has no clothes,” which comes from a folktale popularized by Hans Christian Andersen. This idiom refers to logical fallacies, meaning things that are widely accepted as true and praiseworthy but are, in fact, lacking.

This idiom comes to mind as I survey K‒12 Spanish reading assessments, some of which are indeed complex and praiseworthy, while others are severely lacking.

I explore this point in my new Assessment Masterclass video. In this blog, I’ll take a deeper look at some important considerations, and I’ll share key questions to help you evaluate your current assessments in Spanish.

Understanding the validity of Spanish assessments

The two most essential requirements of any assessment are that it is reliable and valid. These ideas are covered in the first days of any assessment course and remain essential throughout. Despite the primacy of these ideas, many assessment tools are abjectly failing to meet the needs of Emerging Bilingual students whose home language is Spanish. This is because the tools lack full validity.

Let’s begin with a quick review. Essentially, validity is the degree to which an assessment measures what it purports to measure. Shouldn’t an assessment of reading in Spanish cover all the essential skills for progress in Spanish? Some do, but others don’t.

In recent years, educators have been asking providers for tools to assess literacy levels in Spanish, because they:

1. Understand the usefulness of knowledge of students’ abilities in their home language.
2. Want to support students in developing true biliteracy.

While many providers have responded, the thoroughness and thoughtfulness of their responses have varied widely. Some tools, like our own Star Reading in Spanish and Star CBM Lectura, reflect authentic—and therefore valid—assessments of reading in Spanish. Other tools, however, fall short.

Mapping the development of literacy in Spanish

In the same way that Renaissance’s assessments in English are based on empirically validated learning progressions, so too are our assessments in Spanish. But in this case, that’s much easier said than done. For the assessments in English, the ELA standards of each US state provide extensive guidance. But the only US states or agencies that have articulated Spanish reading standards are California, Texas, and the Common Core.

Through reviewing these standards, as well as the Spanish reading standards of Puerto Rico, Mexico, Peru, and Chile, our content teams created the Progresión de la lectura de Renaissance, an authentic learning progression of Spanish reading. This progression forms the backbone of our Star Assessments in Spanish.

This process of building an authentic learning progression ensures there’s a thorough identification of all skills necessary to become literate in Spanish. In this sense, the development process begins with a thoughtful consideration of the language before transitioning to a consideration of assessing reading skills related to that language.

And this is where some assessment providers take a shortcut—with negative effects for both educators and students.

What makes a Spanish assessment authentic?

Rather than undertaking the time-intensive work of developing a learning progression unique to Spanish, some providers have taken their reading assessments in English and translated or trans-adapted them into Spanish. Why is this problematic?

While we all understand the concept of translation, “trans-adaptation” likely requires some explanation. Under this approach, the assessment creator identifies skills that carry over across languages, such as the concept of idioms. But many idioms in English cannot be translated word-for-word into Spanish (or any other language). They simply wouldn’t make sense.

So, in addition to translating the question (the stem of the assessment item), the idioms themselves must be “swapped” from the English originals to the closest Spanish equivalents. This is an example of the sort of adaptation made in trans-adaptation.

Sadly, this is where some providers stop. They translate and trans-adapt, but that’s as far as they go. This results in assessments that measure some elements of reading in Spanish but not all, because elements unique to Spanish have not been considered. In other words, the assessments are not fully valid.

Identifying the skills essential for reading in Spanish

Here are a few examples of elements unique to—and essential for—reading in Spanish that are not addressed when an assessment in English is translated and trans-adapted:

• Accents: The presence or absence of accents can create different meanings in words that are written with the same letters (e.g., si means “if,” while sí means “yes”; other examples include te, té and tu, tú).
• Gender: In English, gender is mostly associated with people and animals. In Spanish, however, every noun has a gender. Table, for example, is feminine (la mesa), while book is masculine (el libro). There are rules for this and exceptions to those rules.
• Formal and informal pronouns: In English, we only have one second person pronoun: “you.” Spanish, however, has both formal and informal pronouns translated as “you:” tú, vos, and usted. And these varying pronouns result in different verb conjugations (e.g., tu tienes, vos tenéis, and usted tiene.)

Given the necessity of such skills to be fully literate in Spanish, consider how incomplete such an assessment would be!

Assessments in Spanish: The problem with translation

An assessment creation process solely based on translating and trans-adapting represents a “one-way flow” from English to Spanish. Without a deep understanding of the Spanish language, it’s hard for those of us who are not native Spanish speakers to grasp how incredibly lacking the end result would be.

So, let’s turn the tables and consider what would happen if we began with an assessment of reading in Spanish and then translated and trans-adapted its items into English.

The result would be an assessment that would not, for example, cover skills related to long and short vowels. The concept of varying vowel sounds is essential when learning to read in English. For example, an “e” can make more than one sound, depending on the other letters around it. This is not the case in Spanish, where each vowel sound is associated with a single letter of the alphabet.

Other examples of skills that are essential to literacy in English but that do not exist in Spanish include:

• R-controlled vowels
• Sight words
• The use of apostrophes to form possessives

Clearly, such an assessment of reading in English—which would fail to assess these essential skills—is not fully valid, which is where translation and trans-adaptation fall short. This does not mean that these practices can never be used, however. Instead, assessment providers must take the difficult, laborious, and costly but critical additional steps of:

1. Building an authentic Spanish learning progression.
2. Creating items to assess the reading skills that are unique to the Spanish language.

Spanish assessments and educational equity

The deep consideration of the dynamics of both English and Spanish that our work around learning progressions required also informed the creation of our assessments.  We learned, in detail, that while some skills are essential in English, they might not even exist in Spanish, and vice versa.

A lack of this cross-linguistic understanding results in the well-intentioned but misguided questions that we sometimes receive, such as “Why don’t you test long and short vowel sounds in Spanish?”  Some monolingual educators, understanding the importance of long and short vowels in English, incorrectly assume this skill would also be essential for learning to read in Spanish.

It comes as a shock when we respond that testing long and short vowels in Spanish wouldn’t be appropriate, because the concept does not exist in that language. In short, reading skills that transfer across languages are reflected in both assessments, while:

• Skills that are unique to English only appear in our assessments in English.
• Skills that are unique to Spanish only appear in our assessments in Spanish.

In this sense, parity across assessments does not mean that every Star curriculum-based measure in English has an identical measure in Spanish. Parity instead means that the essential reading skills in each language are appropriately covered in the assessments in that language.

The skills many Spanish assessments are missing

Our Spanish learning progression includes two more important resources for educators and students.

First, we’ve identified Spanish Focus Skills, which are the most critical skills for students to master at each grade level in order to read proficiently in Spanish. These Focus Skills are highlighted in our Star Assessments in Spanish, and we also make the full list of Spanish Focus Skills freely available to all schools and districts.

Because we developed our Spanish learning progression and our Spanish assessments to support students’ development of biliteracy, we’ve also flagged transferable skills within our Spanish assessments. These are reading skills that are relevant to both English and Spanish and transfer between the two languages, such as:

• Alphabetizing
• Picture/word association
• Identifying key ideas and details
• Making inferences while reading
• Making predictions while reading

Both Focus Skills and transferable skills have tremendous value for instructional planning. They also shed additional light on our discussion of authentic Spanish assessment.

How so?

As I explain in my Masterclass video, there are 52 Focus Skills that are unique to Spanish and are not transferable to English. They represent 17% of the total Spanish Focus Skills across pre-K–grade 12. Most of these skills (40 out of 52) are found in the domain area of Fundamental Skills, and the majority of these (27 out of 40) relate to Phonics and Word Recognition.

This means that Spanish assessments created via the translation/trans-adaptation method are missing nearly one-fifth of the skills that are essential for reading in Spanish. Worse, the majority of these missing skills are in the early grades, where students are building the foundations for success in reading.

I can state with confidence that no school or district administrator would adopt a reading assessment in English that failed to assess any essential skill, much less 52 of them. I firmly believe that administrators should hold their Spanish reading assessment to the same standard—and should ask themselves whether they’ve chosen assessment tools that are truly meeting the needs of their Emerging Bilingual learners.