On his One Foot Tsunami blog, friend-of-TidBITS Paul Kafasis relates what happened when he asked Siri who won Super Bowl XIII. Siri failed miserably, causing him to roll the dice again for every Super Bowl from 1 to 60 (even though 59 and 60 haven’t yet been played). The results are horrifyingly hilarious.
So, how did Siri do? With the absolute most charitable interpretation, Siri correctly provided the winner of just 20 of the 58 Super Bowls that have been played. That’s an absolutely abysmal 34% completion percentage. If Siri were a quarterback, it would be drummed out of the NFL.
At Daring Fireball, John Gruber provides further context and insights by comparing Siri with several other answer engines. In testing these platforms with a randomly chosen question—“Who won the 2004 North Dakota high school boys’ state basketball championship?”—ChatGPT and Kagi answered perfectly. DuckDuckGo received partial credit for providing relevant but incomplete information. (In my testing, Perplexity returned the correct answers but confused the girls’ results with the boys’.) Both Apple’s “new Siri” (enhanced by Apple Intelligence) and Google’s AI Overview failed, offering inconsistently wrong answers. “Old Siri” declined to answer the question, instead providing links like to a standard search engine, with the top link being a PDF containing the answer.
Nailing AI accuracy is trickier than it appears, and Apple clearly has much to learn. For now, restrict Siri usage to simple stuff and always double-check its answers.
