9 ms·
even with gpt 4, it hallucinates enough that it’s not reliable, forgetting to open/close brackets and quotes. This sounds like it’d be a big improvement.
by cwxm 3y ago
even with gpt 4, it hallucinates enough that it’s not reliable, forgetting to open/close brackets and quotes. This sounds like it’d be a big improvement.
- ztratar 3y agoNah, this was solved by most teams a while ago.
- bel423 3y agoI feel like I’m taking crazy pills with the amount of people saying this is game changing. Did they not even try asking gpt to format the output as json?
- worik 3y ago> I feel like I’m taking crazy pills....try asking gpt to format the output as json You are taking crazey pills. Stop gpt-? is unreliable! That is not a bug in it, it is the nature of the beast. It is not an expert at anything except natural language, and even then it is an idiot savant
- jonplackett 3y agoNot that it matters now but just doing something like this works 99% of the time or more with 4 and 90% with 3.5. It is VERY IMPORTANT that you respond in valid JSON ONLY. Nothing before or after. Make sure to escape all strings. Use this format: {“some_variable”: [describe the variable purpose]}
- SamPatt 3y ago99% of the time is still super frustrating when it fails, if you're using it in a consumer facing app. You have to clean up the output to avoid getting an error. If it goes from 99% to 100% JSON that is a big deal for me, much simpler.
- jonplackett 3y agoExcept it says in the small print to expect invalid JSON occasionally, so you have to write your error handling code either way
- davepeck 3y agoYup. Is there a good/forgiving "drunken JSON parser" library that people like to use? Feels like it would be a useful (and separable) piece?
- golol 3y agoHonestly, I suspect asking GPT-4 to fix your JSON (in a new chat) is a good drunken JSON parser. We are only scraping the surface of what's possible with LLMs. If Token generation was free and instant we could come up with a giant schema of interacting model calls that generates 10 suggestions, iterates over them, ranks them and picks the best one, as silly as it sounds.
- 8organicbits 3y agoWouldn't you use traditional software to validate the JSON, then ask chatgpt to try again if it wasn't right?
- girvo 3y agoIn my experience, telling it "no thats wrong, try again" just gets it to be wrong in a new different way, or restate the same wrong answer slightly differently. I've had to explicitly guide it to correct answers or formats at times.
- whateveracct 3y agoIt forgets commas too