6 ms·

I did a similar thing but with backend-heavy code, and I agree with this assessment: > In particular, I asked ChatGPT to write a function by knowing precisely

by ivoras 2y ago

I did a similar thing but with backend-heavy code, and I agree with this assessment: > In particular, I asked ChatGPT to write a function by knowing precisely how I would have implemented it. This is crucial since without knowing the expected result and what every line does, I might end up with a wrong implementation. In my eyes, it makes the whole idea of AI coding moot. If I need to explain every step in detail - and it does not "understand" what it's doing; I can virtually the statistical trial-and-error behind its action - then what's the point? I might as well write it all myself and be a bit more sure the code ends up how I like it. link: https://www.linkedin.com/feed/update/urn:li:activity:7289241001691377664/ https://www.linkedin.com/feed/update/urn:li:activity:7289241...

doug_durham 2y ago
Because as op pointed out it's faster. It has ready access to the correct usage of different libraries.
- WD-42 2y ago
  So does the language features of basically any modern editor.
  - rfw300 2y ago
    Not really - there's a difference between having the docstring of a function available for you to read, and a model which has learned from thousands of examples how to use a particular API and integrate it into a larger set of instructions. The latter is vastly faster and takes much less human work than the former.
    - girvo 2y ago
      Except when it consistently gets said particular API wrong. I was using it to do basic graphql-yoga setup with R1 and then Claude Sonnet 3.5 and they both output incorrect usage, and got stuck in a loop trying to fix it. If it can’t do something that basic and that common using a language and toolset with that much training data, then I’m pessimistic personally. I’m yet to see Copilot be useful for any of my juniors when we pair, it gets in the way far more than it helps and it is ruining their deeper understanding, it seems. I’ll continue trying to use these tools, but I swear you’re overselling their abilities even still.
      - simonw 2y ago
        The way to fix that is to find an example of correct usage of that API and paste that example in at the start of the prompt. This technique can reliably make any good LLM fluent in an API that it's never seen in its training data.
        throwup238 2y ago
        At this point with Cursor you can have it index the online docs by giving it a base URL and have it automatically RAG the relevant content into the chat (using the @ symbol to reference the docs). Both Windsurf and Cursor also support reading from URLs (iirc Aider does too). I’ve had better luck with manually including the page but including the indexed docs is usually enough to fix API mistakes.
        WD-42 2y ago
        Begs the question again: if you need to go out of your way to find an example of correct usage of the api to paste into the prompt, why are you even bothering? I find copilot useful when I already know what I want and start typing it out, at a certain point the scope of the problem is narrowed sufficiently for the LLM to fill the rest in. Of course this is more in line of “glorified autocomplete” than “replacing junior devs” that a keep hearing claims of.
        simonw 2y ago
        "if you need to go out of your way to find an example of correct usage of the api to paste into the prompt, why are you even bothering?" Because it's faster. Here's an example: https://tools.simonwillison.net/ocr https://tools.simonwillison.net/ocr That's an entirely client-side web page you can use to open a PDF which then converts every page to an image (using PDF.js), then runs each image through the Tesseract.js OCR program and lets you copy out the resulting text. I built the first version of that in about 5 minutes while paying attention to a talk at a conference, by pasting in examples of PDF.js and Tesseract.js usage. Here's that transcript: https://gist.github.com/simonw/6a9f077bf8db616e44893a24ae1d36eb https://gist.github.com/simonw/6a9f077bf8db616e44893a24ae1d3... I wrote more about that process here, including the prompts I used: https://simonwillison.net/2024/Mar/30/ocr-pdfs-images/ https://simonwillison.net/2024/Mar/30/ocr-pdfs-images/ That's why I'm bothering: I can produce useful software in just a few minutes, while only paying partial attention to what the LLM is doing for me.
- sneedle 2y ago
  [dead]
simonw 2y ago
LLMs are way, way faster at typing code than I am. I often dictate to them exactly what I need and it saves me a bunch of time.
- bcrosby95 2y ago
  So you would say typing speed matters?
  - onemoresoop 2y ago
    Sometimes it does. If typing slows you down enough to take you out of the zone then it probably does.
    - rubslopes 2y ago
      and, for tasks that take the LLM a minute, you can grab some coffee while it works and come back just to review. It's a great feeling.
  - simonw 2y ago
    Yes. Here's a neat essay that helps capture why I believe that. https://www.scattered-thoughts.net/writing/speed-matters/ https://www.scattered-thoughts.net/writing/speed-matters/
guelo 2y ago
It’s a better, faster, personalized Stack Overflow. Just like SO you might be led down the wrong path by an answer, but if you’re a programmer and you say you don’t get value out of Stack Overflow I don’t believe you.
- deleted 2y ago
  [deleted]
alkonaut 2y ago
There are tons of use cases. E.g. if you know an algorithm (take any pseudocode description from a moderately complex algorithm on Wikipedia for example) and you know the programming language, you still may be looking at an hour or two of typing just to get that pseudocode down into code using your own language, variable names, libraries. But this is the kind of thing a LLM excels at. It gives you 200 lines of impl right away, you have a good understanding of both what it should look like and how it should work. Slow and error prone to type but, but quick and easy to verify once done, that's the key use case for me.

by ivoras 2y ago

doug_durham 2y ago

Because as op pointed out it's faster. It has ready access to the correct usage of different libraries.

WD-42 2y ago
So does the language features of basically any modern editor.
- rfw300 2y ago
  Not really - there's a difference between having the docstring of a function available for you to read, and a model which has learned from thousands of examples how to use a particular API and integrate it into a larger set of instructions. The latter is vastly faster and takes much less human work than the former.
  - girvo 2y ago
    Except when it consistently gets said particular API wrong. I was using it to do basic graphql-yoga setup with R1 and then Claude Sonnet 3.5 and they both output incorrect usage, and got stuck in a loop trying to fix it. If it can’t do something that basic and that common using a language and toolset with that much training data, then I’m pessimistic personally. I’m yet to see Copilot be useful for any of my juniors when we pair, it gets in the way far more than it helps and it is ruining their deeper understanding, it seems. I’ll continue trying to use these tools, but I swear you’re overselling their abilities even still.
    - simonw 2y ago
      The way to fix that is to find an example of correct usage of that API and paste that example in at the start of the prompt. This technique can reliably make any good LLM fluent in an API that it's never seen in its training data.
      - throwup238 2y ago
        At this point with Cursor you can have it index the online docs by giving it a base URL and have it automatically RAG the relevant content into the chat (using the @ symbol to reference the docs). Both Windsurf and Cursor also support reading from URLs (iirc Aider does too). I’ve had better luck with manually including the page but including the indexed docs is usually enough to fix API mistakes.
      - WD-42 2y ago
        Begs the question again: if you need to go out of your way to find an example of correct usage of the api to paste into the prompt, why are you even bothering? I find copilot useful when I already know what I want and start typing it out, at a certain point the scope of the problem is narrowed sufficiently for the LLM to fill the rest in. Of course this is more in line of “glorified autocomplete” than “replacing junior devs” that a keep hearing claims of.
        simonw 2y ago
        "if you need to go out of your way to find an example of correct usage of the api to paste into the prompt, why are you even bothering?" Because it's faster. Here's an example: https://tools.simonwillison.net/ocr https://tools.simonwillison.net/ocr That's an entirely client-side web page you can use to open a PDF which then converts every page to an image (using PDF.js), then runs each image through the Tesseract.js OCR program and lets you copy out the resulting text. I built the first version of that in about 5 minutes while paying attention to a talk at a conference, by pasting in examples of PDF.js and Tesseract.js usage. Here's that transcript: https://gist.github.com/simonw/6a9f077bf8db616e44893a24ae1d36eb https://gist.github.com/simonw/6a9f077bf8db616e44893a24ae1d3... I wrote more about that process here, including the prompts I used: https://simonwillison.net/2024/Mar/30/ocr-pdfs-images/ https://simonwillison.net/2024/Mar/30/ocr-pdfs-images/ That's why I'm bothering: I can produce useful software in just a few minutes, while only paying partial attention to what the LLM is doing for me.
sneedle 2y ago
[dead]

simonw 2y ago

LLMs are way, way faster at typing code than I am. I often dictate to them exactly what I need and it saves me a bunch of time.

bcrosby95 2y ago
So you would say typing speed matters?
- onemoresoop 2y ago
  Sometimes it does. If typing slows you down enough to take you out of the zone then it probably does.
  - rubslopes 2y ago
    and, for tasks that take the LLM a minute, you can grab some coffee while it works and come back just to review. It's a great feeling.
- simonw 2y ago
  Yes. Here's a neat essay that helps capture why I believe that. https://www.scattered-thoughts.net/writing/speed-matters/ https://www.scattered-thoughts.net/writing/speed-matters/

guelo 2y ago

It’s a better, faster, personalized Stack Overflow. Just like SO you might be led down the wrong path by an answer, but if you’re a programmer and you say you don’t get value out of Stack Overflow I don’t believe you.

deleted 2y ago
[deleted]

alkonaut 2y ago

There are tons of use cases. E.g. if you know an algorithm (take any pseudocode description from a moderately complex algorithm on Wikipedia for example) and you know the programming language, you still may be looking at an hour or two of typing just to get that pseudocode down into code using your own language, variable names, libraries. But this is the kind of thing a LLM excels at. It gives you 200 lines of impl right away, you have a good understanding of both what it should look like and how it should work. Slow and error prone to type but, but quick and easy to verify once done, that's the key use case for me.