Every morning before the sun rises, I drive by the top ranked Colorado high school. It doesn’t standout. It’s just a nondescript medium sized K-12 charter school, located within the outer boundary of the wealthy Boulder Valley School District (BVSD). The plot is in a semi-rural area, near mobile home parks and farms. The buildings are new, modest, but tidy, and the lawns are trim. I don’t know much about the school itself, other than its legendary waiting lists. But there is one remarkable thing about this school that continues to surprise me, its consistently high U.S. News and World Reports (U.S. News) rankings.
I am familiar with other schools in the district. One public high school, nestled at the base of the foothills, gives off an aura of understated privilege. Rolling green hills crest into a worn out, windowless octagon structure that clashes with a parking lot full of luxury vehicles. Its proximity to a university and its prestigious International Baccalaureate (IB) program make it a top choice for the academically inclined. In fact, this school posted the top standardized test scores in every category for the BVSD in 2014. This school nurtures more student research scientists, more math, science, and history competition winners, a Presidential Awards for Excellence winner in 2015, more Ivy-bound graduates, more perfect ACT scorers, and more concert musicians, than any other school in the area. Yet, U.S. News ranked this school a mere 10th in the state and 2nd in the district behind the charter school in 2015. This didn’t make sense to me, so I decided to learn more about the U.S. News rankings.
I found that U.S. News rates high schools based on two objective metrics, standardized test scores, and Advanced Placement (AP) and/or IB classes (U.S. News Methodology).This looks impressive in an executive summary, but a deeper look will show this is a flawed approach. First, the focus on test scores is based on a minimum level of proficiency, a low bar. The second measure, called the College Readiness indicator looks specifically at the number of AP/IB classes taken by each student and the AP/IB test pass-rate. U.S. News does not consider any other advanced programming or look at college success rates. Instead, U.S. News applauds schools that have no measurable achievement gap between rich and poor students. While this is an admirable achievement, it is not the only indicator of academic rigor, so it is misleading.
I also discovered that the algorithm used by U.S. News is not published so it is impossible to verify that the process is fair. And, because the scope of the report is so broad, covering more than 29,000 schools, it is impossible to verify self-reported data by charter schools. As a result, the entire system is vulnerable to cheating and manipulation by some schools.
The First Step Eliminates Schools with Quality Programs
Standardized testing measures whether students are proficient or not. The U.S. News ranking process starts by eliminating schools that don’t meet several proficiency measures, based on state standardized test results. One is the Disadvantaged Student Performance indicator which measures the achievement gap. In the BVSD, I know of five quality high schools that were eliminated from the 2015 rankings because of disparities between the non-disadvantaged and disadvantaged students. One of these high schools has a popular project-based engineering program and also an IB program. Another offers more college classes taught by adjunct college professors than any other school in the area and the only business school. A third school has an excellent theater and writing program and the second highest reading scores in the district for 2014.
In contrast, many of the charter schools selected for consideration by U.S. News do not offer enriching programming. These schools pour resources into test preparation curricula to get every student passed that proficiency bar. The test preparation approach is limiting because it narrows subject matter content and influences instructional practices.
The Second AP and/or IB Step is Too Narrow
Once cleared of the proficiency threshold, U.S. News considers College Readiness. In this step, U.S. News looks first at the percentage of children who take and pass the AP and/or IB tests. This is an objective indicator of academic excellence, but not the only one.
As mentioned previously, there are other paths to college and career attainment than the test preparation approach. One of the unranked high schools in BVSD offers advanced mathematics through Calculus III and college-level physics and chemistry beyond AP. This school also offers more advanced computer science options than any other school in the district. U.S. News does not look beyond AP/IB.
One way some schools, such as the high-ranking charters, ensure high AP/IB scoring is by providing tutors that work on test preparation with students, sometimes one-on-one. Students do not need to score high on these exams. They only need to pass with the minimum score.
One unfortunate result of this single-minded emphasis on AP/IB is the dumbing down of these classes to make them available to more students. In Colorado, teacher development programs are encouraging teachers to steer more students into the AP program to help schools appear successful, even if their students aren’t college ready (Colorado A.P. Push).
As a result, AP classes are becoming so watered down, colleges are refusing to accept AP credit for full college credit. Dartmouth College was one of the first to recognize that a high school AP class is not the equivalent of a Dartmouth class. Dartmouth, Brown, and other prominent schools are scrutinizing these programs for what they are, difficult high school classes and nothing more (Dartmouth Ends A.P. Credit).
U.S. News Ranking System Favors Charter Schools
Charter schools by their nature are selective. Most charters have an additional enrollment step that may be confusing to low income parents. They often don’t provide transportation and afterschool care, adding more barriers. A report published from the National Education Policy Center in Boulder, Colorado shows how charter schools discourage low income and special needs students from applying to their schools (NEPC Report on Charter Schools).
Charter schools have been known to counsel out underperforming special education children because these schools are not legally required to provide resources to serve them, unless it is specified in their charters. Unfortunately, most parents do not know this before they enroll their children. Some of these students leave midyear and are not replaced. Disadvantaged children might leave midyear for lack of support. These schools keep the money and count them in their student body figures before passing them off to the public schools, which can do neither. The public school must absorb their costs and their test scores. After enrollment numbers are recorded at the end of September, many charter schools do not admit new students for the rest of the school year or any new students after the 9th grade. U.S. News does not factor these anomalies, even though they have the effect of punishing public schools in the rankings.
High-Stakes Testing Encourages Cheating
All publically funded schools obsess over standardized test scores, despite their pronouncements that they don’t. Test scores determine school rankings and reputation, which in turn influence enrollment, funding, and in some cases, justifications for sanctions against poor performing schools. Test scores determine where students are placed academically and even if they can graduate. Test scores are used to evaluate teachers, determine tenure, and are tied to bonus pay in some schools.
This is why I am not surprised by the alarming statistics that show widespread cheating by teachers and administrators in 39 states, including Colorado (Test Cheating). This makes sense. Standardized tests are administered by teachers and school officials, the very people the tests are intended to evaluate. Testing systems have algorithms built in to reveal statistical anomalies in extreme cases, but the subtle ways to cheat are not detectable: (60 Ways Schools Cheat). U.S. News, in effect, validates this high-stakes system that enables cheating by some schools.
The Data Cannot Be Verified
Large studies tend to be less accurate. U.S. News collects data from over 29,000 public high schools. This information comes from the Common Core Data (CCD) national database, which is a massive compilation of state data. Because of the volume, it is impossible to verify that the data is precise. Human errors are common. And because much of the information submitted by districts and charter schools is self-reported, such as student-to-teacher ratios and the racial makeup of the student body, there is a strong likelihood that many of the schools are misrepresenting the data used by U.S. News.
U.S. News Does Not Publish the Algorithm
Although the U.S. News methodology can be found on their website, the algorithms are not transparent (U.S. New Methodology). In BVSD, an unranked school shows an almost identical breakdown of test scores, student body makeup, student-to-teacher ratios, and achievement gap numbers as a comparable school from another district, ranked 32nd in the state. The discrepancies cannot be analyzed using the methodology provided to the public by U.S. News. This is extremely frustrating for parents and school administrators who expect accuracy and consistency in the rankings.
Conclusion – Lies, Damn Lies, and Statistics
U.S. News ranks public high schools by the number of students that score above the proficient cut score on standardized tests. This process looks at the whole student body and has merit for policy makers and district leaders who make decisions about the schools. These studies are important to ensure equity and funding to our most vulnerable children.
Unfortunately, some schools use the U.S. News rankings to mislead parents who may confuse proficiency with achievement. U.S. News feeds into the high-stakes madness of American education and, in my opinion, encourages, condones, and ignores the cheating, manipulation and lies by some schools.
Newsweek, a magazine competing with U.S. News for the rankings market, recently came out with a two-pronged approach to ranking high schools. The first approach, like U.S. News, looks at socio-economic factors and recognizes schools for their efforts to close the achievement gap. The second approach looks only at achievement and college readiness, providing a different perspective. Perhaps this trend will inspire schools to give up the gimmicks and get back to teaching children. I’m not counting on it.