5 ms·
Yup. Their benchmark appears to consist of giving an agent XLSX files that use advanced Excel features, but no copy of Excel, and expecting the agents to interp
by ekidd 16d ago
Yup. Their benchmark appears to consist of giving an agent XLSX files that use advanced Excel features, but no copy of Excel, and expecting the agents to interpret the spreadsheet the same way Excel would. So the agents try LibreOffice instead, and if that doesn't work, they try to write Python scripts. But the behavior of the Python doesn't match the behavior of Excel.
To put it politely, this seems like a self-inflicted problem. If you really want your agents to interpret advanced Excel features exactly the way Excel would, have you considered maybe giving your agents Excel?