Steps"UURC Research - Student Impact
Sensitive standardized measures of reading and writing ability are used to evaluate student reading performance before and after instruction/intervention. Measures for 2018-19 will include:
Reading Level Assessment (RLA): a standardized, criterion-referenced assessment for instructional reading-level based on oral reading accuracy, rate, and reading comprehension (for reliability and validity information, see Craig, Brown, Fields, & Morris, 2009; Frye & Gosky, 2012; Morris et al., 2011; Morris, Trathen, Frye, Kucan, Ward, & Schlagel, 2013).
Word Recognition Automaticity (WRA): a standardized, criterion-referenced assessment for word recognition automaticity (for reliability and validity information, see immediately preceding citations).
Test of Silent Contextual Reading Fluency (TOSCRF-2): a standardized, norm-referenced assessment of reading fluency and comprehension (for reliability and validity information, see: http://www.proedinc.com/downloads/toscrfmeasurereadcompgroups.pdf, http://www.proedinc.com/downloads/toscrf_validity.pdf, and this 2024 study https://doi.org/10.1080/10573569.2023.2175340).
Dynamic Indicators of Basic Early Literacy Skills (DIBELS-ORF, DAZE & RETELL): a standardized, norm-referenced assessment of reading fluency and reading comprehension (for reliability and validity information, see http://dibels.org/pubs.html).
AimsWEB (AimsWEB ORF & MAZE): a standardized, norm-referenced assessment of reading fluency and reading comprehension (for reliability and validity information, see https://www.aimsweb.com/resources).
Student Assessment of Growth & Excellence (SAGE): a Utah State Office of Education standardized, criterion-referenced assessment of reading and language arts achievement based on Utah Core State Standards.
http://www.schools.utah.gov/sage/Facts/ItemWriting.aspx
https://datagateway.schools.utah.gov/Assessment/SAGE/2016
2003 - 2016 Student Baselines: Second through 12th grade regular and special education students selected to receive Next or Higher StepsSM intervention were identified as "significantly below grade level" on performance assessments administered in early September. At baseline, these students (n = 2,955) scored one to ten years below expectations. First grade and special education students selected to receive Early StepsSM intervention were identified as "seriously at risk" for reading difficulties based on performance assessments administered at the end of Kindergarten or early-1st grade. These students (n = 876) demonstrated incomplete mastery of the alphabet, weak phonemic awareness, weak concept of word, and/or inability to read Kindergarten-level text. Students in classrooms who received UURC Tier I Text intervention (n = 2,214) were identified by a district-specified percentage of low performance on DIBELS and eligibility for free or reduced lunch.
2004 - 2017 Student Post-Test Results: After approximately 45 Next or Higher StepsSM sessions, participating regular education students in grades 2 through 12 averaged 1.11 to 1.10 years of reading growth respectively. Students with documented learning disabilities averaged .75 years of growth. This growth is in marked contrast to students' pre-intervention performance when one-half year's growth or less was the norm. It is important to note that substantial improvement (1.5 - 3 years growth) was achieved by many older English Language Learners, specifically those in grades 5 through 10. Participating educators found most of these students eager to improve their reading abilities - despite initial concerns that students in this age-range lack motivation.
With regard to beginning readers, after an average of 85 Early StepsSM tutoring sessions, 46% of regular education first graders finished the school year reading at or above grade level. It is important to note that the project's criteria for grade level are rigorous. They require at least 90% accuracy and an oral reading rate of at least 40 words per minute on the end-1st grade passage (Dynamic Measurement Group, Inc., 2010; Hasbrouck & Tindal, 1992; Morris, 2004; Morris, Tyner, & Perney, 2000; Santa & Hoien, 1998). Additionally, 71% of the project's first graders came close to meeting those criteria by reading a slightly easier passage with at least 90% accuracy and an oral reading rate of at least 30 words per minute (Good, Harn, Kame'enui, Simmons, & Coyne, 2003; Morris, 2004; Morris et al., 2000; Santa & Hoien, 1998). This benchmark is significant because a beginning reader at that level has reached an important level of reading independence. Specifically, these readers are close to automaticity on 75 or so high frequency words - a corpus large enough to build on. With this foundation, students who exit first grade reading close to end-1st level are able to access second grade text and make progress - an achievement not likely for those who exit below this level.
Corroborating evidence for Early, Next, and Higher StepsSM student achievement comes from Word Recognition Automaticity (WRA) and Woodcock Word Attack (WRMT-WA) across project years 2006-2016. On the former measure, regular and special education students in grades 1-12 averaged a year's growth (1.13 for n = 769), and on the latter measure, students averaged a year or more growth (range = .8 - 3.7) in phonological recoding and yearly percentile rank growth (range = 3.4 - 10.5).
In July of 2018, the paper, "Longitudinal replication of a Tier II intervention model for struggling readers across grades 1-12" was presented at the Society for the Scientific Study of Reading (SSSRI) in Brighton, UK. The purpose of this study was to investigate the longitudinal efficacy of a Tier II intervention model (U Steps) for struggling readers in grades 1 through 12 in Title I schools.
Brown, K.J., Fields, M.K., Craig, G., & Robbins, K.P. (2018, July). Longitudinal replication of a Tier II intervention model for struggling readers across grades 1 - 12. Paper presented at the annual meeting of the Society for the Scientific Study of Reading, Brighton, UK.
Evidence for student achievement related to Tier I Text practica from 2014-17 comes from RLA, TOSCRF, DIBELS and SAGE scores from 2,214 students. Participating pilot educators saw their students' instructional reading levels grow by a full year (RLA) and their fluency and reading comprehension scores progress from an average standard score and percentile rank of 87 and 19, respectively, to end-of-year average scores of 99 and 47 (TOSCRF). Additional measures of oral reading fluency and comprehension score follow this positive trend for pilot partners. With regard to oral reading fluency (DIBELS) across the years, 74.8% of the students in grades 2 through 6 made expected or above expected progress and 68.5% reached the same benchmark for reading comprehension. It is important to note that these percentages are based on increases expected from the beginning of the school year to its end for each grade level; for example, in second grade, students are expected to gain 35 words per minute and 10 points in comprehension over time.
Utah's Student Assessment of Growth and Excellence in Language Arts (SAGE-ELA) is the instrument developed to measure the extent to which the state's third through sixth graders meet Utah Common Core Standards (UCCS, 2013). As a point of reference, at the end of 2014, 42.2% of Utah elementary students scored proficient or above on the SAGE-ELA, and at the end of 2015 and 2016, 44.1% met this benchmark. Data for 2016-17 were not yet available at this writing. State averages provide a background for UURC Tier I practica - two in urban settings, and a third in rural, central Utah. SAGE data from other Tier I practica follow a similar pattern and are available upon request.
- South Sanpete School District (Ephraim, Manti & Gunnison Valley Elementary Schools). After a Tier I practicum during 2015-16, 51% of South Sanpete students scored proficient or above on the SAGE-ELA - an increase from the previous year's 44%. A closer look at individual schools shows that Ephraim held at 57% proficient, 50% of Manti's students scored proficient or above in comparison to 42% the previous year, and Gunnison Valley, with the most rural and poorest student population in the district, saw an increase from 38% proficient to 46% on this measure.
- Murray School District (Grant, Horizon, Liberty, Longview, McMillan, & Viewmont Elementary Schools).The UURC conducted a Tier I Text practicum for all second grade teachers during 2014-15 and followed that with a Writing About Tier I Text practicum for the same group in 2015-16. Positive impact is evident in third grade SAGE-ELA scores that moved in a measurable positive direction over this time: at the end of 2014, 43% of Murray's third-graders were proficient or above on SAGE Language Arts, at the end of 2015 that percentage moved to 47%, and at the end of 2016 52% met or exceeded this benchmark.
- Ogden City School District (SPED students K-12). Ogden teachers of students with learning disabilities have been participating in UURC Tier II since 2006, and began completing Tier I practica in 2014. Student end-of-year SAGE-ELA scores for the end of 2015 showed that 4.2% of the district's SPED students in grades 3 through 12 were proficient or above. At the end of 2016, that percentage had moved to 6.9%. SPED administrators and educators in Ogden City School District attribute this improvement to sustained, intensive professional development.
