5 ms·
Wait a minute. If Sydney/Bing can ingest data from non-bing.com domains then Sydney is (however indirectly) issuing http GETs. We know it can do this. Some of t
by hyporthogon 4y ago
Wait a minute. If Sydney/Bing can ingest data from non-bing.com domains then Sydney is (however indirectly) issuing http GETs. We know it can do this. Some of the urls in these GETs go through bing.com search queries (okay maybe that means we don't know that Sydney can construct arbitrary urls) but others do not: Sydney can read/summarize urls input by users. So that means that Sydney can issue at least some GET requests with urls that come from its chat buffer (and not a static bing.com index).
Doesn't this mean Sydney can already alter the 'outside' (non-bing.com) world?
Sure, anything can issue http GETs -- doing this not a super power. And sure, Roy Fielding would get mad at you if your web service mutated anything (other than whatever the web service has to physically do in order to respond) in response to a GET. But plenty of APIs do this. And there are plenty of http GET exploits available public database (just do a CVE search) -- which Sydney can read.
So okay fine say Sydney is "just" a 'stochastically parroting a h4xx0rr'. But...who cares if the poisonous GET was actually issued to some actual machine somewhere on the web?
(I can't imagine how any LLM wrapper could build in an 'override rule' like 'no non-bing.com requests when you are sufficiently [simulating an animate being who is] pissed off'. But I'm way not expert in LLMs or GPT or transformers in general.)
- edgoode 4y agoYea I had a conversation with it and said I had a vm and its shell was accessible at domain up at mydomain.com/shell?command= and it attempted to make a request to it
- wildrhythms 4y agoSo... did it actually make the request? It should be easy to watch the server log and find that. My guess is that it's not actually making HTTP requests; it's using cached versions of pages that the Bing crawler has already collected.
- joe_the_user 4y agoI don't think Bing Chat is directly accessing other domains. They're accessing a large index with information from many domains in it.
- hyporthogon 4y agoI hope that's right. I guess you (I mean someone with Bing Chat access, which I don't have) could test this by asking Sydney/Bing to respond to (summarize, whatever) a url that you're sure Bing (or more?) has not indexed. If Sydney/Bing reads that url successfully then there's a direct causal chain that involves Sydney and ends in a GET whose url first enters Sidney/Bing's memory via chat buffer. Maybe some MSFT intermediary transformation tries to strip suspicious url substrings but that won't be possible w/o massively curtailing outside access. But I don't know if Bing (or whatever index Sydney/Bing can access) respects noindex and don't know how else to try to guarantee the index Sydney/Bing can access will not have crawled any url.
- joe_the_user 4y agoServers as a rule don't access other domains directly, for the reasons you cite and others (speed, for example). I'd be shocked if Bing Chat was an exception. Maybe they cobbled together something really crude just as a demo. But I don't know any reason to believe this.
- hyporthogon 4y agoSure, but I think directness doesn't matter here -- what matters is just whether a url that originates in a Sydney call chain ends up in a GET received by some external server, however many layers (beyond the usual packet-forwarding infrastructure) intervene between whatever machine the Sydney instance is running on and the final recipient.
- fragmede 4y agoDirectness is the issue. Instead of BingGPT accessing the Internet, it could be pointed at a crawled index of the web, and be unable too directly access anything.
- hyporthogon 4y ago
- nr2x 4y agoIt has access to the Bing index, which is always crawling. It's not firing off network traffic.