7 ms·
Interesting. This seems like a weakness of natural language understanding. If you rephrase your prompt slightly it would get it right. Try: // return even nu
by amasad 3y ago
Interesting. This seems like a weakness of natural language understanding. If you rephrase your prompt slightly it would get it right. Try:
// return even numbers that are also more than 10
const arrayFilter = (array) =>
It would do the right thing. The fine-tuned version gets your prompt right so maybe it benefited from natural language data. Will look more into it.
- doodlesdev 3y agoThat's really interesting, indeed I can reproduce this by changing the comment. I also managed to get correct output for this sample by renaming the function.
- eevilspock 3y agoclearly your original comment was unfair.
- int_19h 3y agoIs it, though? The major selling point of coding LLMs is that you can use natural language to describe what you want. If minor changes to wording - the ones that would not make any difference with a human - can result in drastically worse results, that feels problematic for real-world scenarios.
- throwaway675309 3y agoThe criticism stands if you have to continue to rewrite your "prompt" until you can coax out the correct desired output.
- SCLeo 3y agoI agree. Maybe it interpreted it as return the numbers that are more than 10 in the given array of even numbers. For example, if the instruction says "return person objects that are at least 20 years old", it might be more reasonable to generate: array.filter(item => item.age >= 20) as oppose to array.filter(item => (item instanceof Person) && (item.age >= 20))