7 ms·
For coding it is still 10x worse than gpt4. I asked it to write a simple database sync function and it gives me tons of pseudocode like `//sync object with best
by sebgr 3y ago
For coding it is still 10x worse than gpt4. I asked it to write a simple database sync function and it gives me tons of pseudocode like `//sync object with best practices`. When I ask it to give me real code it forgets tons of key aspects.
- diplodinkus 3y agoAgreed, but I do find gpt4 has been increasing the amount of pseudo code recently. I think they are a/b testing me. I find myself asking if how much energy it wasted giving me replies that I then have to tell it to fix.. Which is of course a silly thing to do, but maybe someone at oAI is listening?
- FrenchDevRemote 3y agoIf you mean through the user friendly chat GPT website, they're probably making it output as few tokens as possible to cut costs
- FrustratedMonky 3y agoThat can't be, because I can ask it a simple question that an answer is maybe 1 sentence, and it repeats the question then provides a whole novel. So ton of tokens.
- madeofpalk 3y agoGPT still writes like a highschooler trying to hit a high word count :(
- droopyEyelids 3y agoLike a content mill trying to keep you on the page for as long as possible! Which it was trained on.
- gtirloni 3y agoYou can ask it to be very concise. I added it to my custom instructions and it has helped a lot.
- gumballindie 3y agoWow, imagine paying so they can experiment on you and limit what you get. I so wish i found such … useful clients for my own projects.
- FrenchDevRemote 3y agoIt's not experimentation, it's probably one of the only things that allowed them to make gpt 3.5 turbo 10 TIMES cheaper than the previous model.
- inciampati 3y agoExcept: you can feed it an entire programming language manual, all the docs for all the modules you want to use, and _then_ it's stunningly good, whipping chatgpt4 that same 10x.
- davedx 3y agoI honestly don’t have time for that level of prompt engineering. So, chatGPT wins (for me)
- machiaweliczny 3y agoYeah but if their model would be accessible it would already have good vscode extension
- roflyear 3y agoRight "may as well do it myself" - I think this is the natural limit these things will reach. Just my opinion.
- p1esk 3y agoGpt4 has 128k context length now.
- whimsicalism 3y agogpt4 turbo
- cowthulhu 3y agoIf you need a lot of revisions/tweaks, the price could be pretty prohibitive.
- chubot 3y agoHow do you do this? Links / more info?
- FrustratedMonky 3y agoCan you just tell it to focus on a particular language and have it go find the manuals? If it is so easy to add manuals, maybe they should just make options to do that for you.
- wouldbecouldbe 3y agoYeah but to be honest been a pain last days to get gpt 4 to write full pieces of code for more the 10-15 lines. Have to re-ask many times and at some point it forgets my initial specifications.
- hfjjbf 3y ago[dead]
- s1gnp0st 3y agoEarlier in the year I had ChatGPT 4 write a large, complicated C program. It did so remarkably well, and most of the code worked without further tweaking. Today I have the same experience. The thing fills in placeholder comments to skip over more difficult regions of the code, and routinely forgets what we were doing. Aside all the recent OpenAI drama, I've been displeased as a paying customer that their products routinely make their debut at a much higher level of performance than when they've been in production for a while. One would expect the opposite unless they're doing a bad job planning capacity. I'm not diminishing the difficulty of what they're doing; nevertheless, from a product perspective this is being handled poorly.
- nomel 3y agoTry using the playground, with a more code specific system prompt, or even put key points/the whole thing into the system prompt. I see better performance, compared to the web.
- nmfisher 3y agoThis was one of the main reasons I cancelled my ChatGPT Pro subscription in favour of Claude…but unfortunately Claude is now doing the same thing too.
- UrineSqueegee 3y ago[dead]
- sp332 3y agoOpenAI just had to pause signups after demo day because of capacity issues. They also switched to making users pay in advance for usage instead of billing them after.
- swatcoder 3y agoBecause they're ultimately training data simulators and not actually brilliant aritifical programmers, we can expect Microsoft-affiliated models like ChatGPT4 and beyond to have much stronger value for coding because they have unmediated access to GitHub content. So it's most useful to look at other capabilities and opportunities when evaluating LLM's with a different heritage. Not to say we shouldn't evaluate this one for coding or report our evaluations, but we shouldn't be surprised that it's not leading the pack on that particular use case.
- whimsicalism 3y agoidk we're just "have more kids" simulators and we do pretty good at programming as a side-task
- swatcoder 3y agoSure, and those of us who have more robust preparation and expoure generally do a better job of it.
- preommr 3y agoSomeone doesn't get good at programming with low quality learning sources. Also, a poor comparison because models are not people - might as well complain about how NPCs in games behave because they fail at problems real people can solve.
- whimsicalism 3y agoWe are both substrate that has been aggressively optimized for a task with a lot of side benefits. "NPC"s are not optimized at all, they are coded using symbolic rules/deterministic behavior.
- YetAnotherNick 3y agoGithub full (public) scrape is available to anyone. GPT-4 was trained before Microsoft deal so I don't think it is because of Github access. And GPT-4 is significantly better in everything compared to second best model for that field, not just coding.
- jpeter 3y agoIncluding all of Github in your training dataset seems like a good idea
- vasili111 3y agoAm I only one that thinks that Claude 2 is not bad for programming questions? I do not think it is best one for programming questions but I do not think that it is bad too. I have received multiple times very good response from Claude 2 on Python and SQL.
- dinvlad 3y agoI find all of them, gpt4 or not, just suck, plain and simple. They are only good for only the most trivial stuff, but any time the complexity rises even a little bit they all start hallucinate wildly and it becomes very clear they're nothing more than just word salad generators.
- charlesischuck 3y agoI have built large scale distributed gpu (96gpus per job) dnn systems and worked on very advanced code bases. GPT4 massively sped up my ability to create this. It is a tool and it takes a lot of time to master it. Took me around 3-6 months of every day use to actually figure out how. You need to go back and try to learn it properly, it's easily 3-5x my work output.