I think it should be quite obvious that perceptrons are far from the smallest units that are capable of learning. They store many bytes of information, require

I think it should be quite obvious that perceptrons are far from the smallest units that are capable of learning. They store many bytes of information, require a non-local update process, need numeric (i.e. symbolic) inputs and involve relatively complex computations. You can go much simpler. For example: https://medium.com/@VictorBanev/the-simplest-learning-machine-pt-2-e735367f546 https://medium.com/@VictorBanev/the-simplest-learning-machin... This is a description of a 5-line algorithm that learns and stores approximate probability of an event using just 1 byte of persistent memory.