AI Models

Analyses were not run to provide direct comparisons between models and any figures here should be taken with an entire bucket of salt because these are not like-to-like. That said, the most general observation that search grounding significantly improved scores on these questions is likely supportable, as is the observations that even recent models struggled without search grounding.

Please note also there is a mismatch between the score (which was computed at analysis time by different models) and the scores on the individual report pages, which was recomputed using different criteria after the fact using a more consistent system of classification. In other words this is a mess that is good for raising questions but not as useful at answering them. Questions will be answered later by more focused runs.