Understanding the Bitworm NuPIC HTM Example Program , Part 1

Now for my least favorite part of intellectual projects, figuring out someone else's computer code.
When I installed the NuPIC package, a program called Bitworm was run to show that NuPIC installed correctly. Bitworm's main program, RunOnce.py is written in Python script and might be characterized as a simplest meaningul example program, which makes it considerably more complicated than your typical Hello World one liner.

The explanation of, and instructions for running and playing with Bitworm can be found in Getting Started With NuPIC (see pages 14-23). If you open RunOnce.py (mine conveniently opened in IDLE, "Python's Integrated Development Environment") there is a good outline of the process too.

The point is to test an HTM (Hierarchical Temporal Memory) with a simple data set. If you got here without knowing about HTMs, see www.numenta.com or my glosss starting with Evaluating HTMs, Part 1.

Bitworm, or RunOnce, starts by creating a minimal HTM. It does this by importing nodes and components using functions that are part of the NuPIC package. It also sets some parameters which have already been built elsewhere. Then the HTM is trained using another already-created data set of bitworms, which are essentially short binary strings easily visualized if 1's as interpreted as black and 0's as white (or whatever colors you like). Later I'll want to look inside the nodes, and at how nodes are interconnected, in order to understand why this works, but for now I'll keep to the top-level-view.

To test if the NuPIC HTM network learned to distinguish 2 types of bitworms, the training data set is again presented to see what outputs the HTM gives. This is also known as pattern recognition, but in temporal memory talk we prefer the term inference. The bitworms are examples of causes (objects in most other systems), and the HTM infers, from the data, which causes are being presented to it.

That seems like too easy of a trick, infering causes based on the training set, so RunOnce also sees how the trained network does trying to infer cuases from a somewhat different set of data.

As output RunOnce gives us the percentages of correct inferences for the training set and second data set, plus some information about the network itself.

Presuming that you are using Windows and downloaded and setup the NuPIC package (see prior blog entry), to run Bitworm with RunOnce.py, open a command prompt (press Start, in the search box type Command. This should show Command Prompt at the top of the program list. Click it once. Since you will need Command Prompt often, you might also return to Start, right-click on Command Prompt, and Pin to Start Menu. Then it is always in your Start Menu. Or create a shortcut).

Type:

cd %NTA%\share\projects\bitworm

and hit Enter. That will get you in the right directory.

Then run RunOnce by typing the following and hitting Enter:

python RunOnce.py

If you get errors, you need to run the Command Prompt as an Administrator. Close the window, then right click on Command Prompt and choose Run As Administrator. Click through security warnings.

The output says there were two sets off 420 data vectors written. Inference with the training set as input data was 100% accurate. Inference with the 2nd data set was 97.85...% accurate.

As it says, you can also open report.txt. Here's what mine says:

General network statistics:
Network has 5 nodes.
Node names are:
category
fileWriter
level1
sensor
topNode

Node Level1 has 40 coincidences and 7 groups.
Node Level2 has 8 coincidences.
------------------------------
Performance statistics:

Comparing: training_results.txt with training_categories.txt
Performance on training set: 100.00%, 420 correct out of 420 vectors
Comparing: test_results.txt with test_categories.txt
Performance on test set: 97.86%, 411 correct out of 420 vectors
------------------------------
Getting groups and coincidences from the node Level1 in network ' trained_bitworm.xml

====> Group = 0
1 0 1 0 1 0 1 0 1 0 1 0 0 0 0 0

0 1 0 1 0 1 0 1 0 1 0 1 0 0 0 0

0 0 1 0 1 0 1 0 1 0 1 0 1 0 0 0

0 0 0 1 0 1 0 1 0 1 0 1 0 1 0 0

0 0 0 0 1 0 1 0 1 0 1 0 1 0 1 0

0 0 0 0 0 1 0 1 0 1 0 1 0 1 0 1

====> Group = 1

0 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0

1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0

0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 0

0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0

0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0

0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0

0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1

====> Group = 2

0 1 0 1 0 1 0 1 0 1 0 0 0 0 0 0

0 0 1 0 1 0 1 0 1 0 1 0 0 0 0 0

0 0 0 1 0 1 0 1 0 1 0 1 0 0 0 0

0 0 0 0 1 0 1 0 1 0 1 0 1 0 0 0

0 0 0 0 0 1 0 1 0 1 0 1 0 1 0 0

0 0 0 0 0 0 1 0 1 0 1 0 1 0 1 0

1 0 1 0 1 0 1 0 1 0 0 0 0 0 0 0

====> Group = 3

0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0

1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0

0 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0

0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 0

0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1

====> Group = 4

0 0 0 0 0 1 1 1 1 1 1 1 1 1 0 0

0 0 0 0 1 1 1 1 1 1 1 1 1 0 0 0

0 0 0 1 1 1 1 1 1 1 1 1 0 0 0 0

0 0 1 1 1 1 1 1 1 1 1 0 0 0 0 0

0 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0

1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0

0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 0

0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1

====> Group = 5

0 0 0 1 1 1 1 1 1 1 1 1 1 1 0 0

0 0 1 1 1 1 1 1 1 1 1 1 1 0 0 0

0 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0

1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0

0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 0

0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1

====> Group = 6

0 0 0 0 0 0 0 1 0 1 0 1 0 1 0 1

Full set of Level 2 coincidences:

0 -> [ 0. 0. 1. 0. 0. 0. 0. 0.]

1 -> [ 1. 0. 0. 0. 0. 0. 0. 0.]

2 -> [ 0. 0. 0. 1. 0. 0. 0. 0.]

3 -> [ 0. 0. 0. 0. 0. 1. 0. 0.]

4 -> [ 0. 1. 0. 0. 0. 0. 0. 0.]

5 -> [ 0. 0. 0. 0. 0. 0. 1. 0.]

6 -> [ 0. 0. 0. 0. 0. 0. 0. 1.]

7 -> [ 0. 0. 0. 0. 1. 0. 0. 0.]

Why Time is Necessary for HTM's Learning

In "Hierarchical Temporary Memory, Concepts, Theory, and Terminology " by Hawkins and George, Section 5, Why is Time Necessary to Learn? clarifies the role of temporal sequences and temporal pattern points in both learning and recognition (inference) by HTMs.

The authors use a good example, a cut versus uncut watermelon, to distinguish between pattern matching algorithms, and how HTM's learn to recognize patterns that are created by objects (causes, in HTM vocabulary). Any real world animal, when viewed, presents an almost infinite number of different visual representations. If you use a type of animal, say horses instead of a particular horse, the data is even more divergent. Pattern matching does not work well. But allow an HTM to view an animal or set of animals over time, and it will build up the ability to recognize an animal from different viewpoints: front, back, profile, or against most sorts of backgrounds.To do that requires data presented over time. Data that is close sequentially should be similar but not identical. Early data might be of a horse, head on, far away, which gradually resolves to a horse viewed close up. So over time the HTM can capture the totality of the horse.

Combining recognition of causes with names given by an outside source is also considered. Thus no amount of viewing a horse will tell an HTMs that human's call the thing "horse." You can do "supervised learning" with an HTM, training it to associate a name with a cause by imposing states on the top level of the HTM hierarchy. But it should be a simple extension to have a vocabulary learning HTM and an object learning HTM in a hierarchy with a learn-to-name-the-object HTM on top.

Once an HTM has learned to recognize images (or other types of data) it can recognize static images (or data). The authors say "The Belief Propagation techniques of the hierarchy will try to resolve the ambiguity of which sequences are active." I am not clear on that. It seems to me that static temporal patterns happen often enough in the real world so that some temporal pattern points will represent static causes. If the horse stands still in the real world, it would generate such temporal patterns. As the data goes up the hierarchy it tends to filter out ambiguity and stabilize causes, so a leaping horse should still be the same as a frozen image of a leaping horse at some point high enough in the hierarchy.

Section 6 is a sort of frequently asked questions part of the paper. I'm not sure if I'll cover all the sections or in what order, and I do want to go back to section 3.3 on belief propagation before closing out this series.

Evaluating HTMs, Part 5: Internal Operations of Nodes

"Hierarchical Temporary Memory, Concepts, Theory, and Terminology " by Hawkins and George, Section 4, How does each node discover and infer causes?, covers questions about the internal operations of nodes that were raised or partly covered in earlier sections.

Basically, each node simulataneously does both learning and recognition of spatial and temporal patterns. The output is information about the patterns that can be sent up and down the hierarchy of nodes.

Spatial patterns do not necessarilly mean space as in space-time continuum, although the example used in the essay is of a two dimensional visual space. Space is used in the mathematical sense. The data can be anything quantifiable. The space could be any number of dimensions. For instance the maximum daily surface temperature of the earth would consitute a space within a certain range of degrees Centigrade, to whatever desired level of precision, on a spherical 2D grid representing the surface of the earth to any desired degree of precision. The time sequence in this example would be daily samples. In a digital audio example the time sequence intervals might be something on the order of .0001 seconds.

The node has a significant number of "quantization points" available to categorize the spatial data. Only the most common data patterns, up to the number of quantization points, will be learned. Anything that is not one of the learned patterns will be assigned a probability that it is one of the learned patterns plus some noise.

Having leaned the quantization points, the node can start looking for common temporal sequences of them. Again, a limited number of points or memory units are allocated for learned temporal patterns. I can't find where the authors give them a name, so at the risk of being corrected later, I'll call them temporal pattern points. Again, temporal pattern matches don't need to be exact; some noise is tolerated.

Once learning has taken place (learning can continue), the node can work to infer causes. The patterns held in the quantization points, as well as the patterns of these in time held in the temporal pattern point, can be causes (or call them objects, which is the more typical if less precise vocabulary.). As time passes the data changes and the causes output to the higher level node(s) of the hierarchy change.

Another very important idea to wrap your head around is that the node, and the HTM, need to deal with probabilities. You might get an exact match with 100% probability, you might even be able to design special situations where you don't need to deal with probability. But the whole point of HTMs (from an engineering standpoint) is to be able to deal with complex real world data, in a manner similar to the human brain. So whether dealing with matching a spatial pattern to the quantization points, or a temporal pattern to the temporal pattern points, you need to think in terms of probability. There is a 42% change that it matches point 7, an 18% change it matches point 42, and a 40% chance that it matches none of the quantization points. With the temporal pattern it gets more complicated, since you can't assume that just because a spatial pattern is most likely point 7, it is not in a temporal pattern that goes, say 49, 3, 7 instead of 49, 3, 42.

Hey, but that is what computers are handy for, figuring probabilities and keeping track of them.

The output to the higher-level nodes might be though of as: There is a 12% probability that we are seeing temporal pattern point 7, 35% it is point 16, 48% it is point 34, and 5% it is point 49. It seems to us to be messy, but it is exactly the sort of thing the next level up is looking for. We call it a vector output, but it also can be thought as a set of data pairing probabilities and points that are arbitrarilly assigned to spatial-temporal patterns.

Hey, I think I am beginning to understand this stuff.

Which brings me back to one of those basic science and philosophy questions that made me interested in machine understanding in the first place. If everything is abstract, how do we (the HTM or a living human brain) get the picture of the world that seems all so familiar to us? If all the world does it produce neuronal impulses in our bodies, what makes red different from blue, and the junk on my desk resolve easily into envelopes, pens, gadgets, fake wood patterns and a host of other things?

Next: Why is Time Necessary to Learn?

Evaluating HTMs, Part 2: What HTMs Do

For the introduction See Evaluating HTMs, Part 1

"Hierarchical Temporary Memory, Concepts, Theory, and Terminology " by Hawkins and George, Section 1, What Do HTMs Do?, asserts that HTMs can discover causal relationships in data presented to them. Once the causality is established, they can infer the cause of a new input. They can predict (with some accuracy) future data sets, and they can use the above three abilities to direct (choose) behavior.

It should be pointed out immediately that each of these 4 abilities have been demonstrated by other computational systems. Neural network software that could discover certain types of relationships from raw data was available at least as early as the 1980's. Many mathematical forms of analysis can find causal relationships within data, for instance regression analysis. In any system, once a relationship is established, recognizing it should present no great difficulty. Nor should making predictions, or using the system to control behavior.

What is interesting about the set of claims for HTMs is that they work together holistically (like the human brain) and should be stackable. That is, the HTM systems should be able to deal with external relationships of increasing complexity by stacking HTM subsystems into an appropriate system. In addition, the HTMs can find relationships in time (sequential or temporal relationships).

In fact, if we call some of the data "objects," for an HTM the objects "have a persistent structure; they exist over time." The authors call the objects "causes." In theory an HTM system could deal with multiple forms of data coming in directly from the world, but usually (for now) the HTM deals with a specific subset of data (much as when a human, say, concentrates on music, or on reading). The data could be a computer file, or a stream of data from input devices.

For the HTM to work the causes should be relatively stable, but should generate data that changes over time, as a horse moving across a visual field, or a conversation between two people. Causes are typically multiple.

The discovery of the causal relationships is a learning process. During learning the HTM builds representations of causes in the form of vectors. The relationship is expressed as a set of probabilities for causes; this set is called a "belief." The causes, relationships, and beliefs can be quite complex if the HTM is complex enough. In particular, hierarchies of causes and beliefs can be learned.

The authors say an HTM, once it has gone through learning, can "infer causes of a novel input." This means that if it is presented with new data, it will try to match the data up to one of the causes it knows about. This is basically pattern recognition, and there are other systems, including certain neural networks, that do this well in certain situations. A good point made by is that if a million pixel visual field (of a scene with motion) is used as the input, it would be rare that an exact pattern would be input twice. So inference, matching a set of data to the closest cause, is a necessity. In the older neural networks causes were typically static; adding a time dimension to the data usually makes it easier for an HTM to learn and infer. I should point out, however, that "infer causes of novel input," to me can mean something more than is claimed by an HTM. For humans, it can mean a deduction, or even a deep set of deductions, rather than just recognizing a pattern or its degree of ambiguity. Then again, perhaps a sufficiently complete HTM system could do even that.

The ability to predict is the third leg of what HTMs do. In other words, given a sequence already encountered, an HTM will predict that sequence is happening again. This sounds like not much, but it is an ability that is crucial to machine understanding. In particular the authors point to priming. Given the latest data, the HTM makes a prediction and notes differences between what is predicted and what happens. If data is ambiguous or noisy, the HTM may fill in with the predicted data. If a prediction is fed back into the HTM as data, this is akin to thinking or imagining. Thus the machine could plan for the future. The authors claim "HTMs can do this well." Imagine a sheep-herding dog application. The better it can predict the behavior of the sheep, the less energy it should need to herd them.

Finally, HTMs can direct behavior. Of course almost any device, even simple mechanical ones, can direct behavior. A mouse trap, given a certain type of input, will engage in a known set behavior. Still, mentioning this for HTMs is important because that is exactly what we would expect artificial intelligence or machine understanding to be used for: behavior. An important point is that "From the HTM's perspective, the [output] system it is connected to is just another object in the world." In other words, an HTM can learn about how its own outputs act is causes in the world.

If you want to get an idea of the potential power of HTMs, before wading through a lot of other materials, section 1.4 of the paper "Direct Behavior", is a great starting point.

I'm excited after reading this part.

Possible Acronym: LIPD (learn, infer, predict, direct)

Next: How do HTMs discover and infer causes?