Tesla has published its first voluntary “Vehicle Safety Report,” and the numbers seem to clearly back CEO Elon Musk’s assertion that drivers who use Tesla’s sort of self-driving Autopilot feature are involved in fewer crashes than those who turn it off, and far fewer crashes than the general driving population. But without more detail, the numbers mean little.

In the report, Tesla says that between July and September of this year, it “registered one accident or crash-like event for every 3.34 million miles driven in which drivers had Autopilot engaged.” Tesla drivers not using Autopilot went 1.92 million miles between incidents.

The company equates “crash-like events” with near misses but didn’t respond to a request for more detail on what that means. The report offers no insight into the severity of the crashes, whether anyone involved was injured, what may have caused the crashes, or where and when they happened.

Tesla’s Autopilot cleverly combines adaptive cruise control, which maintains a set distance from the car in front even if it slows down, and steering assistance, which keeps the car between painted lane markings. Tesla stresses both features are intended for use in limited circumstances. “Autosteer is intended for use only on highways and limited-access roads with a fully attentive driver,” the Model S’s manual says. Although the system can be engaged anywhere, that technically means drivers using Autopilot sticking to roads like freeways—routes free of intersections, pedestrians, cyclists, and other complicating factors. Drivers not using it might be on crowded city streets or twisty country roads, making a comparison useless without that extra information.

Since releasing Autopilot in 2014, Tesla has faced criticism that the system makes drivers overly confident in its abilities, lulling them into a dangerous sense of complacency. At least two people have died in crashes when Autopilot was engaged. Three have crashed into stopped fire trucks in 2018 alone (all survived without serious injury). Musk has tangled with the National Transportation Safety Board over its investigations into Autopilot crashes and attacked critics of the system during a May earnings call.

“It’s really incredibly irresponsible of any journalists with integrity to write an article that would lead people to believe that autonomy is less safe,” Musk said. “Because people might actually turn it off, and then die.” During that same call, he promised Tesla would start releasing these quarterly reports.

The safety report compares that 1.92 million miles per incident figure to data from the National Highway Traffic Safety Administration. It says NHTSA figures show “there is an automobile crash every 492,000 miles.” (Tesla apparently used the NHTSA’s public database to derive this number.) That indicates drivers in other manufacturers’ cars crash nearly seven times more often than drivers using Autopilot.

But again, a closer look raises questions. A broad comparison of Tesla with everyone else on the road doesn’t account for the type of car, or driver demographics, just for starters. A more rigorous statistical analysis could separate daytime versus nighttime crashes, drunk drivers versus sober, clear skies versus snow, new cars versus clunkers, and so on. More context, more insight.

“It’s silly to call it a vehicle safety report,” says David Friedman, a former NHTSA official who now directs advocacy for Consumer Reports. “It’s a couple of data points which are clearly being released in order to try to back up previous statements, but it’s missing all the context and detail that you need.”

Tesla’s one-page report comes the day after Consumer Reports published its comparison of “semiautonomous” systems that let drivers take their hands off the wheel but require them to keep their eyes on the road. That ranking put Cadillac’s Super Cruise in first place and Autopilot in second, followed by Nissan’s Pro Pilot Assist and Volvo’s Pilot Assist. It evaluated each on how it ensures the human is monitoring the car as well as its driving.

In response to those criticisms that Autopilot lulls users into trusting it too much, Tesla has recently used over-the-air software updates to ratchet up how often the human must touch the steering wheel to confirm they’re still alive and concentrating. Cadillac’s approach is more sophisticated: It uses an infrared camera to ensure the driver’s head is pointed at the road (instead of down at a phone), allowing for a truly hands-off system. (Audi’s Traffic Jam Pilot uses a gaze-tracking setup that allows a driver to look away in certain conditions, but it isn’t available in the US.)

Other safety-minded groups, including the IIHS and the UK’s Thatcham, are designing their own tests for these increasingly popular features, acknowledging they’re all flawed.

Tesla does have an excellent safety record when it comes to crash testing. In September the NHTSA awarded the Model 3, Tesla’s newest car, five stars in every category. The Model X SUV got the same commendation, and when the Model S sedan was tested in 2013, it proved so strong it broke the test equipment.

And it could be that its Autopilot system is making highway driving safer, perhaps by reducing driver fatigue or reducing rear-end collisions. But this report isn’t enough to show that. Friedman says he was hoping for more. He wants Tesla to give its data to an academic, who can do a rigorous, independent, statistical analysis. “If the data shows that Autopilot is delivering a safety benefit, then that’s great.”

Tesla is unique among automakers in releasing this type of data at all, and going forward it could expand on it to make it more useful. The company’s blog post with the latest statistics says it “introduced a completely new telemetry stream for our vehicles to facilitate these reports.”

And the size of its fleet is growing fast, as Tesla ramps up production of the Model 3. Its delivery numbers released on Tuesday show it put 83,500 cars in new driveways in the same quarter its safety figures cover. That means there’s going to be a lot more data to analyze in the future.

Tesla has always moved faster than the mainstream auto industry and deserves credit for acceleration the adoption of electric driving, software updates, and self-driving features. But if it wants to be congratulated for making roads safer, it has to cough up more data.

More Great WIRED Stories