6 ms·

It always blows my mind that nobody at Google thought it would be a good idea to very carefully review the answer of the AI. In the second screenshot, the pro

It always blows my mind that nobody at Google thought it would be a good idea to very carefully review the answer of the AI. In the second screenshot, the prompt asks about CVE-2024-3400, and at first glance this appears ok. But in the affected systems section it states: > Also Hitachi Energy RTU500 firmware and Siemens Ruggedcom APE1808 firmware. I cannot find any reference that this Hitachi device is vulnerable to that CVE. Hitachi has a nice interface to list all vulnerabilities of their devices, this CVE is not part of it. In the Mitigation section any mention of Hitachi is also missing. Almost as if this device is not vulnerable. There is some more weirdness, like it doesn't mention the "portal" feature is also vulnerable.

ebursztein 1y ago
Thanks for looking in-depth in our post. The Hitachi RTU500 mention is not an hallucination, we did check for those. It is mentioned in the Mandiant threat intelligence data.
- infoSecer 1y ago
  Have you considered that Mandiant is wrong? I cannot find any evidence that it would be vulnerable. Hitachi doesn't even appear to be a technology partner of Palo Alto (https://technologypartners.paloaltonetworks.com/English/directory?q=Hitachi https://technologypartners.paloaltonetworks.com/English/dire...). As far as I can tell, the only connection between those is, that CISA released this alert which mentions multiple unrelated advisories in one post. Which happens to be the Siemens Palo Alto and another unrelated Hitachi advisory in RTU500: https://www.cisa.gov/news-events/alerts/2024/04/25/cisa-releases-eight-industrial-control-systems-advisories https://www.cisa.gov/news-events/alerts/2024/04/25/cisa-rele...
  - fc417fc802 1y ago
    Isn't the tool doing its job in that case? I wouldn't generally expect it to independently determine that an otherwise reliable source made a mistake. In fact I feel like that would be a really bad idea. Imagine if a relatively clueless intern left something out of a report because the textbook "seemed wrong".
    - infoSecer 1y ago
      I don't really know what its job is to be honest. Saying that the input data is wrong and the AI didn't hallucinate that data is also kind of a "trust me bro" statement. The Mandiant feed is not public, so I cannot check what was fed to it. I don't really care why its wrong. It is wrong. And using that as the example prompt in your announcement is an interesting choice.

ebursztein 1y ago

Thanks for looking in-depth in our post. The Hitachi RTU500 mention is not an hallucination, we did check for those. It is mentioned in the Mandiant threat intelligence data.

infoSecer 1y ago
Have you considered that Mandiant is wrong? I cannot find any evidence that it would be vulnerable. Hitachi doesn't even appear to be a technology partner of Palo Alto (https://technologypartners.paloaltonetworks.com/English/directory?q=Hitachi https://technologypartners.paloaltonetworks.com/English/dire...). As far as I can tell, the only connection between those is, that CISA released this alert which mentions multiple unrelated advisories in one post. Which happens to be the Siemens Palo Alto and another unrelated Hitachi advisory in RTU500: https://www.cisa.gov/news-events/alerts/2024/04/25/cisa-releases-eight-industrial-control-systems-advisories https://www.cisa.gov/news-events/alerts/2024/04/25/cisa-rele...
- fc417fc802 1y ago
  Isn't the tool doing its job in that case? I wouldn't generally expect it to independently determine that an otherwise reliable source made a mistake. In fact I feel like that would be a really bad idea. Imagine if a relatively clueless intern left something out of a report because the textbook "seemed wrong".
  - infoSecer 1y ago
    I don't really know what its job is to be honest. Saying that the input data is wrong and the AI didn't hallucinate that data is also kind of a "trust me bro" statement. The Mandiant feed is not public, so I cannot check what was fed to it. I don't really care why its wrong. It is wrong. And using that as the example prompt in your announcement is an interesting choice.