Understanding the Bitworm NuPIC HTM Example Program, Part 2: Network Creation Overview

Now that Bitworm is running (See Bitworm Part 1), there are a variety of options. In the Getting Started document the next steps are funning Bitworm with "temporally incoherent data" and then with noisy data. We could go to the data generation functions and play with them, then see how Bitworm reacts. I am more interested in how the network is created, and how it functions internally. An overview of this is covered in "Creating the Untrained HTM Network File," (starting page 21 of Getting Started).

One thing I found helpful is looking at the set of programs in \Numenta\nupic-1.71\share\projects\bitworm\runtimeNetwork\. These include what appears to be an older version of RunOnce.py that uses CreateNetwork.py for network creation. In the "plain" version of RunOnce the network creation segment has just four lines of code:

bitNet = Network()
AddSensor(bitNet, featureVectorLength = inputSize)
AddZeta1Level(bitNet, numNodes = 1)
AddClassifierNode(bitNet, numCategories = 2)

AddSensor(), AddZeta1Level(), and AddClassifier() are imported functions from nupic.network.helpers. They don't seem to be used other than for Bitworm, so they are worth discussing only in the context of understanding the node structure of Bitworm. This network appears to have 4 nodes in the Getting Started (page 22) illustration, but in CreateNetwork.py we find five listed: the sensor node, the category sensor node, an unsupervised node, a supervised node, and an effector node. Getting Started calls 3 of the nodes the same, but instead of supervised and unsupervised, refers to bottom-level and top-level nodes.

Jumping ahead in Getting Started, we find that bitNet = Network() does indeed create an HTM instance that nodes can be added to and arranged in.

The runtime version replaces these with a single command (but a lot more parameters):

createNetwork(untrainedNetwork = untrainedNetwork,
inputSize = inputSize,
maxDistance = maxDistance,
topNeighbors = topNeighbors,
maxGroups = maxGroups)

CreateNetwork.py can also be found in the runtime directory. Open it and the first thing you see
CreateNetwork starts by importing nupic.network. So there is a set of one or more functions or classes we can use to get an overview; we'll look inside them later, if necessary. The following line of code gives us our function parameters, some of which are set specifically for Bitworm. So CreateNetwork.py is not a general-purpose HTM creation function.

def createNetwork(untrainedNetwork,
inputSize = 16,
maxDistance = 0.0,
topNeighbors = 3,
maxGroups = 8):

Next we have some agreement with the plain RunOnce.py:

net = Network()

Network() is an imported function that creates the overall data structure for the HTM.

Nodes are created with the CreateNode() function. The type of node - sensor, category sensor, unsupervised (Zeta1Nodes), supervised (Zeta1TopNodes), and effectors - is chosen with the first parameter of CreateNode(). Among the other parameters of CreateNode you can see spatialPoolerAlgorithm and temporalPoolerAlgorithm. I don't think I having used "pooling" yet. Remember I wrote about quantization points? [See How do HTMs Learn?] There are a number of available points both for spatial and temporal patterns in the unsupervised nodes. They need to be populated, and they may change during the learning phase. Pooling appears to be NuSpeak for this process; a pooler algorithm is the code that matches up incoming data to quantization points.

I did not get as far as I would have liked today, but I am beginning to see some structure, and dinner is calling. Instead of calling this entry HTM Creation Classes and Functions, I'll call it an Overview.