How Do HTMs Learn?

Evaluating HTMs, Part 3: How do HTMs Learn?

"Hierarchical Temporary Memory, Concepts, Theory, and Terminology " by Hawkins and George, Section 2, "How Do HTMs Discover and Infer Causes", gives an overview of the internal mechanisms of HTMs.

Specifically, it gives an a overview of how HTMs learn. This prompted me to think about the difference between learning and discovery. They could be the same thing, but learning for humans often implies a teacher presenting information to be learned to a student. Discovery implies coming upon something new (perhaps a relationship between already known objects) and realizing that it is new and should be remembered.
Each node in an HTM uses the same algorithm. The nodes are arranged hiearchically, with representations typically showing a row of nodes at the bottom that take input. Each row above the bottom has progressively fewer nodes. The top node is a single node. Its output is a vector that represents a cause, or object related to the data in a causal fashion. In fact each node does this, passing its output vector to the next higher row of nodes. So causes are built up hierarchically. All data and discoveries include a time element. In a visual field, for instance, the time element could be no change in a part of the field, or changing color with time, or following a spot of color from one part of the visual field to another over a course of time.

Get used to the technical use of the term "belief" if you want to follow discussions about HTMs. This term is used extensively in probabilistic reasoning theory. "A belief is an internal state of each node," but it does correspond to a probability that there is a causal relationship in the data. "I believe the lion must have escaped from the zoo," is a sentence that conveys to us that a person lives where lions do not live in the wild; it differs from "I know ..." because the speaker is admitting there are other possible causes. In a simple HTM, in a lower node, a belief might be something like "28% probability that this is a horizontal line, 16% that this is two animal eyes, etc." Again, beliefs are represented in software by vectors, but they are not generally identical to the output vectors of the nodes.

In training or learning, the HTM forms new beliefs at the bottom of the hierarchy of nodes first. More complex beliefs can only be created once lower level beliefs exist, but the entire process is flexible. If a lower level node alters its belief, it tends to effect higher level nodes. So learning is not just memorization.

So how does a node do all this? Nodes are given a set number of "quantization points." Here the authors are not very clear. The input pattern is assigned to one of the quantization points. And/or "the node decides how close (spacially) the current input is to each of the quantization points and assigns a probablity to each quantization point." How it decides is presumably an algorithm. With enough quantization points, each input data set could be matched exactly to a point. Would that set up cause the node to fail? To do as the authors say, the assumption is there are less quantization points than there are possible inputs.

Step two is "the node looks for common sequences of these quantization points" and "represents each sequence with a variable." So you have to ask, why assign probabilities, why not just assign closest fits? In any case the output variable represents a sequence of quantization points based on the sequence of input data.

Admitting that the authors are introducing the topic, and its vocabulary, still I would have liked more than two short paragraphs on the internal operations of HTM nodes.

Interestingly (and copying what is known about the cortex of mammal brains) information can move both up and down the hierarchy of nodes. As just described, data moving up the hierarchy is temporal variables. Data going down the hierarchy represents the "distribution over the quantization points." That would be a probability distribution.

What I suspect the authors mean is that there is a mechanism to alter the quantization points themselves. Points with long-term zero percent probabilities don't help resolve ambiguity. The set of point probabilities being sent down the hierarchy allows the (lower) node to "take a relatively stable pattern from its parent node(s)" and "turn it into a sequence of spatial patterns."

The claim is that over time "patterns move from the bottom of the hierarchy to the top." In effect rather than sending the raw data up the hierarchy, the nodes send "names" of data sequences up the hierarchy.

That would be pretty cool, and I'd like to know exactly how it happens, but this is, afterall, only an introduction.

Next: Why is a hierarchy important?

Gambling, Addiction and Prediction

Why do people love to gamble? True, there is a range of how much various people like to gamble. At one end there are people are (or pretend to be) bored by it (I exclude people who, for whatever reason, won’t even try games of chance or skill). At the other end are people who lose their savings, homes, and families, or even fingers and lives to legal and illegal gambling operations.

It may all have to do with the brain’s built-in prediction mechanisms.

According to Jeff Hawkins’s theory of the sequential and predictive basis of human intelligence, the primary intelligent function of the brain is to remember sequences of data and use that memory to predict future sequences of data. This is true both for sensory data that we think of as naturally sequential, like the notes in a music tune, and for sensory data that we normally think of as non-sequential, like the features in a human face. According to the theory we construct a model of a human face from sequences of visual input (supplemented by other data, like touching one’s own face). If we see an eye, we expect to see a sequence of other features: another eye, a nose, mouth, eyebrows, hair, chin, ears. We expect to hear a voice when a mouth moves, but not when an eye blinks.

An important element in building a model of the world is surprise. Using memory, the brain predicts what it expects to see, hear, and feel next. When something different occurs, the element of surprise turns on higher functions in the brain. We may look at something more closely. Perhaps it is a face, but it is a strange face. Is this a sign of danger? Friend or foe, we need to memorize the new face and whatever we associate with it (perhaps the face belongs to someone who is being introduced by a friend; we need to remember that association, or the physical place where we met the person).

There is a bit of a nervous rush when a prediction is made that turns out to be false. The mind gears up to determine why it made a false prediction, both in order to deal with the unexpected situation and to be able to predict it in the future. Probably this rush is partly neurochemical. We’ll call it a bit of adrenaline, although it is possibly based on another neurotransmitter or hormone. Adrenaline rush is the lay term I will use.

Games can teach, and they can while the time away. Even when they are not played for money or prizes, they can have an addictive quality. In almost every game an element of prediction is involved. The outcome provides relief if it was predicted, and a rush of neural activity if it was not predicted.

Generally, people like adrenaline rushes, especially in small doses. They like a thrill more than they like boredom. Games are played partly because they provide small, safe levels of thrill in situations that would otherwise being predictable and boring.

Gambling involves making a prediction. It involves two highs for the human brain. Most of us enjoy winning. Getting money from winning gives the same motivation as getting a snack, affection, or a pay check.

I believe that it is losing, however, that makes gambling addictive. When you lose in a random game of chance, your brain is fooled. It made a prediction. It was surprised by the outcome. Now it is designed to frantically memorize what happened and construct a way of making a correct prediction.

Gamblers often come to believe, after a string of losses, that they are about to make a correct prediction or a series of correct predictions. This is how your brain works: long experience has taught it that nature is a set of sequences, and past sequences can predict future sequences. The brain is fooled by the artificial construction of random events like dice throws, roulette wheel spins, and card deals. Worse, there is a sometimes-confusing element of predictability to some of this (two dice will roll a seven more often than a two; in poker three of a kind comes up more often than four of a kind) that tends to reinforce the brains belief that it can make sense out of randomness. The brain persists in the false belief that random events become predictable if only you keep placing bets long enough.

There is a rush when losing. People, on a chemical level, don’t feel pain when they lose. The chips or money they gamble is abstract, removed from the necessities or pleasures they could buy if had not lost. But the loss forces the higher intelligence centers of the brain to gear up. It is like taking a tiny bit of a stimulant. It is pleasurable.

This is often why otherwise highly intelligent individuals will lose vast sums of money in casinos. It is precisely like drug addiction. They may, at some conscious level, when away from a casino, coldly say that they understand they cannot beat a random game with odds set so that the house wins. But in the casino they cannot get out of the loop of feeding themselves tiny bits of adrenaline with each wager, win or lose.