Query: understanding hallucination rate on the chart

[d6y]

Many thanks for maintaining this leader board.

I'd like to check I've understood what the hallucination rate means on the chart.

E.g., for 1.3% hallucination rate, I think it means "summaries were evaluated by HHEM and only 1.3% of the summaries were predicted to have a hallucinations score (from HHEM) below 0.5".

Or is it perhaps: out of N summaries evaluated for a given model (Gemini 2 flash exp, for example), we found that 1.3% of the summaries actually contained hallucinations and the rest did not.

Help on understanding this would be much appreciated.