8 ms·
I would pay hundreds of dollars per month for the combination of cursor and claude - I could not get my head around it when my beginner lever colleague said "I
by binocarlos 2y ago
I would pay hundreds of dollars per month for the combination of cursor and claude - I could not get my head around it when my beginner lever colleague said "I just coded this whole thing using cursor".
It was an entire web app, with search filters, tree based drag and drop GUIs, the backend api server, database migrations, auth and everything else.
Not once did he need to ask me a question. When I asked him "how long did this take" and expected him to say "a few weeks" (it would have taken me - a far more experienced engineer - 2 months minimum).
His answer was "a few days".
What I'm not saying is "AGI is close" but I've seen tangible evidence (only in the last 2 months), that my 20 year software engineering career is about to change and massively for the upside. Everyone is going to be so much more productive using these tools is how I see this.
- apwell23 2y agoSo what is his plan to fix all the bugs that claude hallucinated in the code ?
- JanSt 2y agoI'm confident you have not used Cursor Composer + Claude 3.5 Sonnet. I'd say the level of bugs is no higher than that of a typical engineer - maybe even lower.
- hobs 2y agoThere's no LLM for which that is true or we'd all be fired.
- dagw 2y agoIf all you bring to the table is the ability to reimplement simple web apps to spec, then sooner or later you probably will be fired.
- joshuacc 2y agoIn my experience it is true, but only for relatively small pieces of a system at the time. LLMs have to be orchestrated by a knowledgeable human operator to build a complete system any larger than a small library.
- ben_w 2y agoIn the long term, sure. Short term, when that happens, we're going to be on Wile E. Cyote physics and keep up until we look down and notice the absence of ground.
- threeseed 2y agoIt's only as good as its training data. Step outside of building basic web/CRUD apps and its accuracy drops off substantially. Also almost every library it uses is old and insecure.
- whatshisface 2y agoThat last point represents the biggest problem this technology will leave us with. Nobody's going to train LLMs on new libraries or frameworks when writing original code takes an order of magnitude longer than generating code for the 2023 stack.
- Workaccount2 2y agoWith LLM's like gemini, which have massive context windows, you can just drop the full documentation for anything in the context window. It dramatically improves output.
- SubiculumCode 2y agoI use phind which does searches to provide additional context
- mewpmewp2 2y agoYet most work seems to be CRUD related and most SaaS businesses starting up just really need those things mainly.
- apwell23 2y agoI am confident you didn't understand my comment. I didn't say anything about "level of bugs".
- dagw 2y agoClaude is actually surprisingly good at fixing bugs as well. Feed it a code snippet and either the error message or a brief description of the problem and it will in many cases generate new code that works.
- JanSt 2y agoYes, the value of a single engineer can easily double. Even a junior - and it's much easier for them to ask Claude for help than the senior engineer on the team (low barrier for unblock).
- threeseed 2y ago> 20 year software engineering career is about to change I have also been developing for 20+ years. And have heard the exact same thing about IDEs, Search Engines, Stack Overflow, Github etc. But in my experience at least how fast I code has never been the limiting factor in my project's success. So LLMs are nice and all but isn't going to change the industry all that much.
- pluc 2y agoThere will be a whole industry of people who fix what AI has created. I don't know if it will be faster to build the wrong thing and pay to have it fixed or to build the right thing from the get go, but after having seen some shit, like you, I have a little idea.
- dumbfounder 2y agoCorrection: a whole industry of AI that will fix what AI has created.
- Workaccount2 2y agoThat industry will only form if LLMs don't improve from here. But the evidence, both theoretical and empirical, is quite the opposite. In fact one of the core reasons transformers gained so much traction is because they scale so well. If nothing really changes in 3-5 years, then I'd call it a flop. But the writing is on the wall that "scale = smarts", and what we have today still looks like a foundational stage for LLM's.
- 2y ago
- orwin 2y agoI really believe that the front-end part can be mostly automated (the html/CSS at least), copilot is close imho (microsoft+github, I used both), but really they're useless to do anything else complex without making to much calls, proposing bad data structures, using bad /old code design.
- JanSt 2y agoCopilot is pretty bad compared to cursor with sonnet. I have used Copilot for quite a long time so I can tell.
- skydhash 2y agoThe frontend part was already automated. We called it Dreamweaver and RAD tools.
- epicureanideal 2y agoThank you, now I realize where I've had this feeling before! Working with AI-generated code to add new features feels like working with Dreamweaver-generated code, which was also unpleasant. It's not written the same way a human would write it, isn't written with ease of modification in mind, etc.
- aniviacat 2y agoCurrent LLMs fail if what you're coding is not the most common of tasks. And a simple web app is about as basic as it gets. I've tried using LLMs for some libraries I'm working on, and they failed miserably. Trying to make an LLM implement a trait with a generic type in Rust is a game of luck with very poor chances. I'm sure LLMs can massively speed up tasks like front-end JavaScript development, simple Python scripts, or writing SQL queries (which have been written a million times before). But for anything even mildly complex, LLMs are still not suited.
- ben_w 2y ago> Current LLMs fail if what you're coding is not the most common of tasks Succeeding on the most common tasks (which isn't exactly what you said) is identical to "they're useful".
- abm53 2y agoAnd I would go further… these “common tasks” cover 80% of the work in even the most demanding engineering or research positions.
- layer8 2y agoThat’s absolutely not my experience. I struggle to find tasks in my day to day work where LLMs are saving me time. One reason is that the systems and domains I work with are hardly represented at all on the internet.
- scruple 2y agoI have the same experience. I'm in gamesdev and we've been encouraged to test out LLM tooling. Most of us at/above the senior level report the same experience: it sucks, it doesn't grasp the broader context of the systems that these problems exist inside of, even when you prompt it as best as you can, and it makes a lot of wild assed, incorrect assumptions about what it doesn't know and which are often hard to detect. But it's also utterly failed to handle mundane tasks, like porting legacy code from one language and ecosystem to another, which is frankly surprising to me because I'd have assumed it would be perfectly suited for that task.
- StefanWestfal 2y agoI am curiouse, how complex was the app? I use cursor too and am very satisfied with it. It seem that is very good at code that must have been written so many times before (think react components, node.js REST api endpoints etc.) but it starts to fall of when moving into specific domains. And for me that is the best case scenario, it takes away the part we have to code / solve already solved problems again and again so we can focus more on the other parts of software engineering beyond writing code.
- Roark66 2y agoI can't understand how anyone can use these tools (copilot especially) to make entire projects from scratch and expand them later. They just lead you down the wrong path 90% of the time. Personally I much prefer Chatgpt. I give it specific small problems to resolve and some context. At most 100 lines of code. If it gets more the quality goes to shit. In fact copilot feels like chatgpt that was given too much context.
- sensanaty 2y agoI hear it all the time on HN that people are producing entire apps with LLMs, but I just don't believe it. All of my experiences with LLMs have been that for anything that isn't a braindead-simple for loop is just unworkable garbage that takes more effort to fix than if you just wrote it from scratch to begin with. And then you're immediately met with "You're using it wrong!", "You're using the wrong model!", "You're prompting it wrong!" and my favorite, "Well, it boosts my productivity a ton!". I sat down with the "AI Guru" as he calls himself at work to see how he works with it and... He doesn't. He'll ask it something, write an insanely comprehensive prompt, and it spits out... Generic trash that looks the same as the output I ask of it when I provide it 2 sentences total, and it doesn't even work properly. But he still stands by it, even though I'm actively watching him just dump everything he just wrote up for the AI and start implementing things himself. I don't know what to call this phenomenon, but it's shocking to me. Even something that should be in its wheelhouse like producing simple test cases, it often just isn't able to do it to a satisfactory level. I've tried every one of these shitty things available in the market because my employer pays for it (I would never in my life spend money on this crap), and it just never works. I feel like I'm going crazy reading all the hype, but I'm slowly starting to suspect that most of it is just covert shilling by vested persons.
- flir 2y agoJust for fun, give it a function you wrote, and ask it if it can make any improvements. I reckon I accept about a third of what it suggests.
- 2y ago
- skapadia 2y agoDid you take a look at the code generated? Was it well designed and amenable to extension / building on top of? I've been impressed with the ability to generate "throw away" code for testing out an idea or rapidly prototyping something.
- nativeit 2y agoConsidering the current state of the industry, and the prevailing corporate climate, are you sure your job is about to get easier, or are you about to experience cuts to both jobs and pay?
- cml123 2y agoyes, but does your colleague even fully understand what was generated? Does he have a good mental map of the organization of the project? I have a good mental map of the projects I work on because I wrote them myself. When new business problems emerge, I can picture how to solve them using the different components of those applications. If I hadn't actually written the application myself, that expertise would not exist. Your colleague may have a working application, but I seriously doubt he understands it in the way that is usually needed for maintaining it long term. I am not trying to be pessimistic, but I _really_ worry about these tools crippling an entire generation of programmers.
- alonsonic 2y agoAI assistants are also quite good at helping you create a high level map of a codebase. They are able to traverse the whole project structure and functionality and explain to you how things are organized and what responsibilities are. I just went back to an old project (didn't remember much about it) and used Cursor to make a small bug fix and it helped me get it done in no time. I used it to identify where the issue might be based on logs and then elaborate on potential causes before then suggesting a solution and implementing it. It's the ultimate pair programmer setup.
- insane_dreamer 2y ago> I just went back to an old project (didn't remember much about it) and used Cursor to make a small bug fix and it helped me get it done in no time. That sounds quite useful. Does Cursor feed your entire project code (traversing all folders and files) into the context?
- Anamon 2y agoDo you ever verify those explanations, though? Because I occasionally try having an LLM summarise an article or document I just read, and it's almost always wrong. I have my doubts that they would fare much better in "understanding" an entire codebase. My constant suspicion is that most results people are so impressed with were just never validated.
- 2y ago
- gtvwill 2y agoI've coded python scripts that let me take csv data from hornresp and convert it to 3d models I can import into sketchup. I did two coding units at uni, so whilst I can read it... I can't write it from scratch to save my life. I can debug and fix scripts gpt gives me. I did the hornresp script in about 40 mins. It would have taken me weeks to learn what it produced. I'm not a mathematician, hell i did general maths at school. Currently I've been talking through scripting a method to mix dsd audio files natively without converting to tradional pcm. I'm about to use gpt to craft these scripts. There is no way I could have done this myself without years of learning. Now all I have to do is wait half a day so I can use my free gpt o credits to code it for me (I'm broke af so can't afford subs). The productivity gains are insane. I'd pay for this in a heartbeat if I could afford it.
- insane_dreamer 2y agoThe problem is that it only works for basic stuff for which there is a lot of existing example code out there to work with. In niche situations it's not helpful at all in writing code that works (or even close). It is helpful as a quick lookup for docs for libs or functions you don't use much, or for gotchas that you might otherwise search StackOverflow for answers to. It's good for quick-and-dirty code that I need for one-off scripts, testing, and stuff like that which won't make it into production.
- SJC_Hacker 2y agoYeah AI can give you a good base if its something thats been done before (which admittedly, 99% of SE projects are), especially in the target language. Yeah, if you want tic-tac-toe or snake, you can simply ask ChatGPT and it will spit out something reasonable. But this is not much better than a search engine/framework to be honest. Asking it to be "creative" or to tweak existing code however ...
- rurp 2y agoFairly standard greenfield projects seem to be the absolute best scenario for an LLM. It is impressive, but that's not what most professional software development work is, in my experience. Even once I know what specifically to code I spend much more time ensuring that code will be consistent and maintainable with the rest of the project than with just getting it to work. So far I haven't found LLMs to be all that good at that sort of work.
- skywhopper 2y agoI hear these stories, and I have to wonder, how useful is the app really? Was it actually built to address a need or was it built to learn the coding tool? Is it secure, maintainable, accessible, deployable, and usable? Or is it just a tweaked demo? Plenty of demo apps have all those features, but would never serve as the basis for something real or meet actual customer needs.
- charlie0 2y agoSounds like CRUD boilerplate. Sure, it's great to have AI build this out and it saves a ton of time, but I've yet to see any examples (online or otherwise) or people building complex business rules and feature sets using AI. The sad part is beginners using the boilerplate code won't get any practice building apps and will completely fail at the complex parts of an app OR try to use AI to build it and it will be terrible code.