Imagine a standardized test being used to measure the healing of a patient, and the effectiveness of the doctor.
It would look something like this.
A doctor sees a patient through treatment of a condition, and at the end of a prescribed length of time the patient completes a bubble test to determine progress. It is irrelevant what the patient’s condition was at the start of treatment, what other issues the patient is experiencing, how long the patient received treatment, or how well the patient followed medical advice.
The physician’s perception of the patient’s progress, or any additional insights he or she might have, is also irrelevant. It is the bubble test result that will determine whether the physician is an effective practitioner.
This scenario is readily recognized as absurd, and even potentially dangerous, when applied to medicine. Why do we accept it as appropriate for education?
Yet, high-stakes standardized testing is viewed as not just appropriate for education, it is viewed as essential. So essential, that even in the face of dissent from the majority of parents and educators, our politicians continue to reinforce the myth that standardized tests are a fundamental method for assessing student learning, and therefore, by extrapolation, a credible way to determine the effectiveness of teachers and schools.
This false narrative was initiated with the publication of A Nation At Risk in 1983 and reinforced and perpetuated through Goals 2000, No Child Left Behind, and most recently, The Every Student Succeeds Act.
Ohio’s implementation of the Every Student Succeeds Act is how I found myself standing at a podium providing testimony before the Joint Education Oversight Committee at the Ohio Statehouse.
“Ms. Taylor, do you believe that the state legislature can honestly check the required box indicating that stakeholder feedback was included in the Ohio plan?”
This was the final question I was asked during my testimony. I had been invited to the statehouse by the Ohio Federation of Teachers to serve as a voice for educators across the state, and to provide insight to the committee on whether the Ohio draft plan for implementation of the federal Every Student Succeeds Act (ESSA) accurately reflects the views of stakeholders and serves as a visionary document.
Being called for this task felt like a huge responsibility.
I walked into the room expecting something fairly familiar and comfortable. In my mind, I was anticipating a group of people sitting around a conference table. Instead there was a podium in front of a raised bench of legislators. This suddenly felt like an overwhelming responsibility. I was near certain that the entire room could hear my heart racing and my knees knocking.
I knew that my physiologic reaction was ridiculous. I have engaged in significant research and reflection on this topic. I know the salient points, and I know how to articulate them in a cohesive and powerful manner. And I am not afraid.
Except I was afraid.
This was more important than fear. To quote Dr. Seuss, “I am the Lorax. I speak for the trees.”
I speak for students, and for educators, and for our future society because high-stakes standardized testing is not innocuous. It is not just something we debate about at the dining room table. It is truly damaging.
However, we are up against a mighty foe – the testing industry and a social construct that school accountability measures are effective, necessary, and appropriate – and we must be willing to fight furiously against this.
Which is how I found myself testifying at the Ohio statehouse about ESSA.
The Every Student Succeeds Act (ESSA) was passed by the federal government in December of 2015. It replaces No Child Left Behind, and it allows states greater flexibility in teacher and school accountability measures.
One of the requirements of ESSA is the engagement of stakeholders in the process of developing state plans. It is this mandate that prompted the question asked by Representative Fedor. She is a state representative from Toledo who serves on this committee, and she is a friend to education.
The Ohio ESSA draft plan notes, “As part of the legislation, each state is required to conduct significant outreach to stakeholders to collect input for their state plan. Ohio takes this mandate very seriously and has already engaged 15,000 Ohioans in the development of this draft.”
In every section of the draft there is evidence of stakeholder feedback. However, startlingly, in several critical areas, this feedback has not been incorporated into the current plan.
Relative to required testing, the plan states, in the section titled Aligned Academic Assessments, that stakeholders emphasized the need to “strategically reduce tests where it makes sense to do so.” It goes on to claim that “while the state has reduced the amount of time students spend taking tests – down by approximately 50% from 2014 to 2016 – stakeholders expressed an interest in continuing to explore a further reduction in testing.”
And here is the statement regarding the testing requirements under the “new” plan. “As part of ESSA, Ohio will reexamine its testing requirements. The department is poised to work closely with the Governor, legislature, and education leaders to examine the pros and cons of adjusting the testing schedule.”
In other words: no change.
Additionally, while it is true that testing has been significantly reduced since the 2014-2015 test administration, this data is a red herring. It seems to imply that, over time, testing requirements have been reduced, and that is simply inaccurate. The 2014-2015 school year included the ill-fated implementation of the PARCC testing. Each of the PARCC tests included two administrations – one in February and one in April, thus doubling testing requirements. Thankfully, this double battery of tests was eliminated with the transition to the AIR tests the following school year. This change did reduce testing by nearly half; however if the 2014-2015 school year is removed from the data set as an outlier, then it becomes clear that over time, the number of mandatory state tests in Ohio has actually increased, not decreased.
The second area in the Ohio ESSA draft plan that I found concerning was the provisions regarding teacher evaluation. Currently, Ohio public educators are evaluated based on a combination of factors, and this varies based on the grade and subject area being taught, and that grade and/or subject area’s testing requirements.
Here is what Ohio’s ESSA draft plan says,
- “Strong support for local educators – they understand the critical roles teachers and leaders play in helping students learn and grow”
- “Educators do not believe that the current evaluation system is working as it should”
- “Concern on the part of educators related to the calculation of student growth and its inclusion in the evaluation system”
“Ohio’s state plan requires a description of our methods for ensuring that students have access to quality teachers and leaders. Our plan will be based on those elements currently in state law and our existing equity plan.”
In other words: no change.
Currently, the Ohio teacher evaluation system is designed on a combination of factors. This is a complex calculation where 50% of a teacher’s evaluation comes from observational data assessed using the rubric of the etpes system, which includes 10 areas of assessment, each of which can be scored as: Accomplished, Skilled, Developing, or Ineffective. The remaining 50% varies based on the grade and subject area being taught, and that grade and/or subject area’s testing requirements.
For some Cincinnati Public School teachers, this remaining 50% comes exclusively from the value-added measure of standardized test results. For other teachers, 26% comes from the teacher’s value-added standardized test results, 10% comes from shared attribution – or the standardized test data for growth across the building as a whole, and 14% comes from “Student Learning Objectives” (SLOs). For teachers in non-tested areas, 40% comes from SLOs and 10% comes from shared attribution.
Clear as mud, right?
To add to the complexity, no one knows how these growth measures – called “value-added scores” are calculated.
Ohio contracts with Battelle, a private company, to generate value-added data from standardized test results. They consider their formula “proprietary information,” and despite evidence that these scores are invalid, they remain in place. The only mathematical approximation I have seen as to what this formula might look like is this.
(Fortunately, because of the transition to new assessment tools, test data from the 2014-2015 and 2015-2016 school years have been under what is known as “safe harbor,” meaning that for the given years, standardized test data has not been included in teacher evaluations.)
An additional piece to consider is that when the state counts the required number of tests, they fail to mention the requirements of Student Learning Objective assessments (SLOs). SLOs are another type of growth-measure assessment. Depending on the district, these may be vendor-purchased or teacher-created tests. The majority of teachers must give two SLOs as a required component of their annual evaluation. Each SLO requires a pre-test and a post-test. So for every teacher, this is a minimum of 4 more mandated assessments. To be fair, these tests are far less burdensome than the state tests, but think about a high school student who may take seven classes. This student could take up to 28 SLO tests – two pre tests and two post tests for each of seven teachers. Add the state tests, and final exams, and, at some grade levels, the ACT, PSAT, or SAT as well.
Are you getting the picture yet?
Nevertheless, there remains more to the story. Currently, each of the state tests has 2 sections. Students with identified disabilities often receive an extended time testing accommodation; this allows them to have up to an entire school day to complete each portion of the test. I want to be very clear that I think this is an important provision.
As a special educator, I teach my students best-practice testing strategies. I teach them to read the questions before reading the passage. I teach them to read and annotate the text of the passage before beginning to answer the questions. I teach them to look back at the text. I teach them to use elimination. I teach them not to rush. I teach them to go back and check over all their answers – more than once. All of these things take time, and I have had several students who literally take the entire day to complete a section of the test. I do not want to restrict them in this.
However, many schools have high percentages of special education students. At Gamble, 36% of our students have identified disabilities. When this many students have the right to use the entire day for a section of the test, this provision drives the testing schedule. It is not fair, nor feasible, to give two sections of a test in a day to the general education population, while only scheduling one a day (as legally required) for the special education population. Doing so would mean that special education students would test for twice as many days as general education students, and would therefore miss the instruction being provided on the extra days of testing. This slower-paced scheduling increases the number of days relegated to testing.
At the high school level, there is yet another issue to consider. Passage of the high school state tests is required for graduation (unless a student is on the newly created vocational “pathway,” which has a whole different set of testing requirements.) Therefore, students who have not passed sections of the test are expected to retake these tests three times a year (one is a summer testing) until they achieve proficiency.
I don’t think that people who don’t work in schools really understand the true impact of what we are being told we must do. Governor Kasich has included in his current budget a requirement that teachers do “internships” with local businesses each time they renew their education licenses, so they understand the job requirements in the “real world.” Allow me to suggest that legislators (and executors) should be required to extern in schools, so they understand the actual “real world” implications of the mandates they are passing down.
Let me provide you with a real-world, worst-case scenario. I taught Bryce in junior high. He is a student with an identified learning disability. He struggles academically, but performs especially poorly in a testing situation.
Bryce is now a junior in high school, and he has not yet passed any of the required tests – ELA I, ELA II, Algebra I, Geometry (or Integrated Math I), Biology, or American History, and he is currently enrolled in American Government, which also has a required test. Each test has two sections. Extended time testing is written into Bryce’s IEP, so he must be provided with the option of using the entire day to complete each section of each test. He is a student who needs this extra time.
Were you counting? That’s 14 school days (or nearly three weeks) of testing.
These tests are scheduled by the state and district at the end of April and the beginning of May, as they should be since they are intended to assess the entire curriculum, and an earlier testing session would further truncate instructional time. However, in high school, students must also take final exams. In every class. Because of the timing of the school year, these final exams are administered immediately following the conclusion of the state testing. That is now 17 nearly-consecutive days of testing.
I have not yet mentioned that Bryce also had to take 6 of these state tests during the first round of retakes in December (Don’t forget – 2 sections for each test, so 12 days) and the ACT in April. Before SLOs are factored into the equation, Bryce will have spent 30 days – or nearly 6 weeks of the school year – taking tests.
This is not just a nightmare; this is Bryce’s current reality.
And it is madness. Ultimately, it’s not even about student learning. It’s about assessment of public teachers and of public schools.
The test results that we put so much stake in and spend so much time thinking about and preparing for, are of little use in instructing students.
Does this come as a surprise? Let me explain.
The preliminary test results are generally released over the summer, and final data is usually provided at some point in the fall. At this point, the students who took these tests have moved on – to a new grade, a new teacher, and a new curriculum. The tests they will take next will be focused on the expectations of the new curriculum, not the old one, so knowing a student’s scores from the prior year is only marginally beneficial for a teacher.
In addition, what does this data show? It may seem as if this question should have an obvious answer. They show what a student knows, and therefore, by extrapolation, they show how well a student has been taught. Right?
I question this assumption.
Any teacher will tell you that his or her test scores vary from year to year – often wildly. Are we really that erratic in our teaching practices?
The value-added measures can indicate huge gains – more than two years of academic growth in a year’s time. That sounds great, but, as an educator who has received scores like this, I am not convinced that this is realistic. In the same vein, value-added measures can indicate huge losses – more than 2 years of academic decline in a year’s time. How is this even remotely possible? How is it possible for a teacher to be so bad that she or he causes a student to LOSE two years of academic instruction, while simultaneously providing instruction for the entirety of a year?
This makes no sense.
Early this year, I learned that my teaching partner and I had the highest test scores in the building related to student comprehension of informational text. I was asked what we did to have such success – how could this be replicated throughout the building?
I had to laugh. What did we do? We heavily taught literary text. We focused less on informational text last year than we ever had before.
It wasn’t really intentional. We just didn’t have time for everything, and we had generally chosen literary text standards over informational ones that year. And yet our test scores for informational text standards were much higher than they were for literary text standards. Go figure.
So, I don’t have the greatest confidence in the reliability of testing data as an indicator of much of anything at all. Besides, if standardized tests tell us such important information, why aren’t private and parochial schools demanding these tests? Why aren’t our politicians demanding that the schools that many of their children attend be implementing these tools that measure student learning and teacher effectiveness? Don’t they want the best for their children? Don’t they want to be reassured that their child is learning? Don’t they want to know the quality of their children’s teachers?
No, they don’t. They don’t because standardized tests are not an effective tool for assessing these important things.
We put students in public schools through this wringer of testing for what? If it doesn’t tell us about kids, and it doesn’t tell us about instruction, and it doesn’t tell us about teachers, then why are we doing it? That remains unclear.
It seems as if nearly everyone has one or more teachers who had a profound influence on their growth and development. Who was yours? Think about this person – or these people. Try to identify what it was that made them so influential, so impactful on your life. What were the qualities they possessed that inspired or guided you?
Now consider the impossibility of designing a standardized-test based teacher or school evaluation system around fostering those qualities. Do we want to build educational institutions around test scores, or do we want to build them around nurturing children?
So to answer Representative Fedor’s question: Has the state effectively included stakeholder feedback in the development of Ohio’s ESSA draft plan?
In a word, No.
Stakeholders clearly said, “Fewer tests.” The draft plan indicates no change in the number of tests.
Stakeholders clearly said, “Amend the teacher evaluation system.” The draft plan indicates no change to the teacher evaluation system.
Despite more than a year to develop it, the draft plan doesn’t look much different from what Ohio’s educational legislation looked like under No Child Left Behind. To be fair, in both of the sections of the draft plan that I have critiqued, there is indication that changes could come in the future. However, Ohio has had more than a year to develop this plan, why isn’t change evidenced there already?
As I stated to Representative Fedor, and to the Committee as a whole, I was shocked to see the stakeholder feedback so blatantly ignored in the draft document. As an educator I feel devalued, disheartened, and unsupported by the state of Ohio.
The system is backwards. We have politicians telling educators what to do to prove themselves, rather than educators informing politicians about what it is we need in order to teach children.
What we don’t need are standardized tests. Politicians believe that these tests tell us important things about education. Teachers know that they do not.
Education is a service industry. Unlike manufacturing, service industries work with human capital. Our students are our raw material, and they are each unique individuals. They each come to us at a different place, they each have different external factors at play, and they each approach instruction in a different way.
Their growth and development is as complex as they each are as individuals. To try and measure this in a standardized manner is folly.
The Ohio state legislature wanted to know if the Draft Plan was visionary. Oxford defines the word visionary as, “Thinking about or planning the future with imagination or wisdom.”
Is the Ohio draft plan visionary? No. But then neither is ESSA. To be visionary, we must walk away from the folly of this testing madness.
There is precedent for this.
Just twenty years ago, we had a different system. There was no such thing as high-stakes testing.
Many schools gave standardized tests as a means to compare their students to students around the country. But not in every grade and not every year. It was one piece of the educational puzzle. It provided teachers and schools with some small amount of insight into student learning. But that is all. There was no school report card. There were no punitive measures for teachers.
We must walk back from the precipice on which we are standing. In just two decades, politicians and the testing industry have whipped us into a testing frenzy driven by the notion that these tests provide an accurate measure of school success, and that this is an appropriate way to hold schools and teachers accountable.
It is not.
To be truly visionary, it is not enough to simply demand fewer tests. We must change the paradigm. We must create a new narrative.
How to do this is, of course, the ultimate question. Teachers and parents must band together. We must arm ourselves with data and evidence. We must keep speaking truth to power. We must speak up again and again and again. We must have courage.