K-3 Literacy
The K-3 Literacy metric measures how well lagging readers "catch up" the next year.
In this post, we'll see how, surprisingly, high reading achievement might lead to a low K-3 Literacy score due to the small sample size effect.
Skyrocketing Statistics
If you've ever read news reports with statistics (and I know you all do!), you're probably familiar with Skyrocketing Statistics. For example:
-
Absenteeism skyrockets 100% at local chess club!
-
Crime rate soars over 80% this year!
-
Wild goose population up an astronomical 500%!
​
Naturally, your mind conjures up visions of sofas piled with truant chess club members, convenience stores stocked with criminals, and a goose every other footstep. This certainly grabs attention. How can it be true otherwise? In fact, due to small sample size, the truth may be much more... mundane.
​
Small Sample Size, Big Headlines
While statistics are technically always true, sometimes they aren't helpful. Small sample size is one of those unhelpful situations. It's especially unhelpful when examining change (increase or decrease) from a previous time point.
Let's see how small sample size can lead to big, misleading headlines.
Say we have a Chess Club with four members. Chess Club has five meetings per year, one hour each. So, in total, our members spend 20 hours (4 * 5) at Chess Club.
Last year, two students missed one meeting. So that's 18 hours total spent at Chess Club, and 2 hours absent.
This year, Billy and Susie catch pinkeye one week, Tyler had one doctor's appointment, and Kaitlyn was out of town once for debate club. That's a total of 4 hours absent. Not too bad. We can then calculate the percentage increase by figuring the total increase:
(4 hours absent this year - 2 hours last year) = 2 hour increase
and dividing by last year's absenteeism (2 hours), times 100 (since it's a percentage), for a result of
((4 hours - 2 hours) / 2 hours) * 100 = 100% increase
Whoa! Sounds enormous!
Likewise, if we go from one lonely goose to a party of six, our goose population has increased 500%. (An increase of 5, which is 500% of our original 1 goose). It sounds sensational, but the reality is much more commonplace.
K-3 Literacy Implications
For the K-3 literacy calculation, this boils down to some good news, and some bad news.
Good news: The vast majority of kids can read!
Bad news: Misleading statistics ensue!
The current ODE report card website shows this dichotomy.
Let's assume a school with 100 kindergarteners where 90 of them (90%) can read at level. That's pretty good! There's an ODE chart for this, but no metric.
At the end of the year, of the ten not reading at level, four have improved and six still lag. That's a 40% improvement rate, which sounds awfully low. This amounts to a "D" grade.
Statistics aren't helpful for a sample size this small. A single student? Anything is possible. Severe learning disabilities, sudden personal life tragedies, and other situations can greatly impact a child for a year. At best, this is demotivating for our special educators. At worst, students could be singled out and shunned for circumstances out of their control.
K-3 Literacy Clarification
Can we give credit for achievement while maintaining accountability for progress? Yes! In fact, we've done it before for grades 3 and up, with the Performance Index (achievement) and Value-Added (progress) metrics.
The central problem for K-3 literacy is that there's no indicator for achievement. Why not apply the same philosophy of having both achievement and progress indicators to all grades? This consistency will greatly improvement report card comprehension and allow a smooth transition from K-3 literacy evaluation to 3rd grade and higher standardized testing.
We can show achievement again with a bullet chart (similar to the current ODE chart), and state a goal (currently 100%).
Recommendations
1. Use both achievement and progress metrics for K-3 literacy to mitigate small sample size shortcomings.
Then show improvement with a second chart. For now, we'll leave this progress metric unchanged, acknowledging the small sample size limitations.
​
The improvement graph may still look low. But, when it's balanced by a strong achievement graph, people can accept the limitations of statistics.
