6 ms·
Pelicans and bikes can be in the training set without them training for this specific benchmark.
by eminence32 2mo ago
Pelicans and bikes can be in the training set without them training for this specific benchmark.
- j_maffe 2mo agoYes and that would improve its ability to draw SVGs of pelicans on bikes, no?
- asasidh 2mo agoand that is bad because ?
- program_whiz 2mo agothe nature of the test was to see if the models can effectively compose an image of a novel concept outside the training set. If they are trained on it, it ceases to be an interesting test to some extent.
- cyanydeez 2mo agoit's still interesting because there's no pelican-on-bike model, and if you're training a model well enough, then it should be obvious when a model has reached "AGI" or whatever.
- wasabi991011 2mo agoI would urge you to re-read the blog post you are commenting on. It pretty clearly explains how it is an interesting test independently of "see[ing] if the models can effectively compose an image of a novel concept outside the training set".
- barrenko 2mo agoWould it? Tongue in cheek.
- deleted 2mo ago[deleted]
- freedomben 2mo ago> Yes and that would improve its ability to draw SVGs of pelicans on bikes, no? I would think the opposite because unless people have been hand drawing these with high quality, the training would be on much crappier versions that old AIs have done.