5 ms·
What's the chance that these limericks are now in the training set? As others mention, it'd be interesting to come up with a way to synthesize something suffici
by cararemixed 2y ago
What's the chance that these limericks are now in the training set? As others mention, it'd be interesting to come up with a way to synthesize something sufficiently interesting so it always evades training fit.
- sftombu 2y agoPrevious answer to this question: https://news.ycombinator.com/item?id=40361419 https://news.ycombinator.com/item?id=40361419
- deleted 2y ago[deleted]
- causal 2y agoYour test is a good one but the point still stands that a novel dataset is the next step to being sure.
- dontupvoteme 2y agoOne could also programmatically (e.g. with nltk or spacy, replace nouns, named entities, etc) modify the dataset, even up to the point that every test run is unique. You could also throw in vector similarity if you wanted to keep words as more synonyms or antonyms.