• Article

Flawed! The Trouble with Crash-Test Ratings: Adding Injury to Insult

The federal system that designates a vehicle’s safety is flawed, because most vehicles earn top ratings. Federal regulators agree that consumers should be able to differentiate among vehicles, but they say changes to the star-ratings system won’t be in place until the 2019 model year.

Email to a Friend

Insurance Institute for Highway Safety

“More stars. Safer cars” is the catchphrase that National Highway Traffic Safety Administration (NHTSA) uses to describe its five-star rating system. Unfortunately, we found that the ratings system doesn’t allow consumers to tell much of any difference between vehicles.

NHTSA’s Five-Star Safety Ratings Program reports the safety of drivers and passengers in frontal crashes, side crashes and rollovers. The star ratings, which are posted at SaferCar.gov and listed on the window stickers on all new vehicles, were designed to allow consumers to compare vehicles when they shop for a new vehicle. The ratings are part of the government’s New Car Assessment Program (NCAP).

However, of the 369 models that were evaluated by NHTSA for an overall safety rating in 2015, only 3.0 percent received fewer than four stars on the agency’s five-star scale for overall ratings. Eleven vehicles received three-star, or “average,” ratings.

The star ratings are based on a statistical analysis of the probability that occupants would be injured seriously in a crash that’s similar to NHTSA’s tests. However, NHTSA doesn’t define specifically how different star ratings translate into how likely you are to be injured.

Every rating is relative to an average risk of injury that’s seen among all models, the agency says. For example, NHTSA considers the crash-injury risk for occupants who are in five-star-rated vehicles to be “much less” than what the occupants would experience in a vehicle that has three-star protection, which has average risk. (Of course, bigger SUVs and pickups will fare better overall than, say, small cars.) NHTSA considers the crash-injury risk to occupants of four-star-rated vehicles to fall somewhere between less than average and average.

NHTSA tells us that a direct comparison between overall star ratings and injury probability isn’t possible, because the overall rating is based on a combination of front, side and rollover ratings.

Critics say NHTSA’s star-rating system doesn’t provide consumers with a useful way to judge which vehicles are the safest. Furthermore, crash tests that are conducted by Insurance Institute for Highway Safety (IIHS) can confuse consumers similarly.

Click chart above to view full presentation

Click chart above to view full presentation

NHTSA says it will make the star-rating system more relevant, but it will be years before we see the results. Consequently, consumers should be wary about giving too much weight to how many stars a new vehicle earns, particularly when it comes to models that have the same rating.

STAR CARS? It’s too easy for automakers to earn high scores, says Jack Gillis of Consumer Federation of America. The ease of achieving four- and five-star ratings “is a serious problem that has, in effect, neutered the benefits of the program,” he says.

Gillis is a former NHTSA adviser. Since exiting that role, he developed a crash-test index by analyzing raw crash data. He publishes his index in The Car Book, which is produced annually by Center for Auto Safety (CAS). Gillis compares the differences between his ratings system and NHTSA’s star-ratings program thus. Let’s say a school has a grading system where all students who earn 90 percent or better receive an A and those who earn between 80 percent and 90 percent receive a B. “Imagine if you had 50 students and they all got test scores between 80 and 100—you’d have all A and B students,” Gillis says. NHTSA, in effect, hands out all A’s and B’s, he says. Gillis says he curves his scores: The top 10 percent get a score of “10” regardless of the raw data and the bottom 10 percent get only a “1.” NHTSA didn’t respond to questions about Gillis’ system.

The percentage of vehicles that earn four or five stars in NHTSA’s overall rating jumped to 97 percent for the 2015 model year from 81 percent in 2011. The lower 2011 rankings were the result of changes that NHTSA made in response to previous criticisms of high crash-test scores. For the 2011 model year, NHTSA tightened requirements by adding a side pole test, which measures the sideways crash of a vehicle into a tree or a post, and introduced test dummies that could gather more injury data. (In 2010, 99 percent of all vehicles earned four- or five-star ratings in a frontal-crash test, and 96 percent of all vehicles earned four- or five-star ratings in a side-crash test prior to NHTSA issuing an “overall” rating.)

Back to Article