intelligence (part i)
I saw a tweet today that said “How much of intelligence is compression”.
I don't know what the original poster meant by it, but it really resonated with me. There are probably a lot of different ways of interpreting it. When I saw “compression”, I thought of data compression — “encoding information using fewer bits than the original representation.” I'm fascinated by data compression.
Here is how I would describe it in really simplistic terms (and probably the extent of my actionable knowledge on the subject):
Compression is about finding and exploiting redundancy. Put another way, I think we could say compression is about finding patterns and expressing those patterns in more efficient ways. An example text:
Twinkle, twinkle, little star, How I wonder what you are! Up above the world so high, Like a diamond in the sky.
When this blazing sun is gone, When he nothing shines upon, Then you show your little light, Twinkle, twinkle, through the night.
A simple compression algorithm might move through this text and create tokens for the letter combinations it finds. On the first line there's already a repeat — “winkle,” is there twice. The algorithm could mark the first “winkle,” as token 0, and then perhaps replace the second “winkle,” with a pointer to token 0. This process can continue throughout the text, tokenizing “When”, “ight”, “you”, etc. If pointing to an already seen token can be done with less space than the token itself, we've just compressed the text. (This is all hand waving, I know only a little about this but I love it so much).
So then how does this relate to intelligence. After reading the tweet I sat for a while and thought about how intelligence could fit this model. I think perhaps that learning something new is mostly connecting something already learned with a new context. I think that's why we are always making analogies to other things when talking about complex ideas. For instance, in object oriented programming, I see all the time that lessons and tutorials will use the relationships between physical entities (like cars, trucks, cats, and dogs) to teach patterns of relating different objects together.
I read once in an old spiritual text that when a person has learned a subject, they ought to be able to condense the subject down a level — that is, they ought to be able to simplify the idea. The text went on to say that the wisest person to ever live would learn the most complex and esoteric subjects and be able to explain them on a child's level because the person's understanding was so complete.
So I see that intelligence could be seen as tokenizing bits of information that flow into the mind, and then when new information enters the mind, to be able to detect those already-seen bits and link back to the tokens. Maybe an example is that a person could learn to ride a bicycle and then learn to ride a motorcycle. The person doesn't need to learn how to stay balanced on two-wheels when learning to ride a motorcycle. They don't need to think about it, they can refer back to the earlier learning and apply it in the new context of going a lot faster and not having to pedal.
Maybe intelligence can be seen as the structure that results from all this linking and condensing.