Tokens are just another representation of the image, like 3 matrices or electrical impulses in our brain. They probably encode most of the relevant information,

Tokens are just another representation of the image, like 3 matrices or electrical impulses in our brain. They probably encode most of the relevant information, the question is whether the linear algebra at the core of a transformer is good enough to use this data as efficiently as we do. I don't think we can currently say if it is or isn't adequate.