ApeWorm, an Initial Machine Ape Project

The core of the system we seek is quite complex. We would like to minimize the inputs and outputs in order to focus our first project on the core aping system. At the same time, we want a general solution that is scalable to projects with larger, more complex input/output and memory requirements.

We will call our first attempt ApeWorms. These creature constructs will exist on a 5 x 5 grid and will have, for bodies, four connected line segments, the ends of which line up on the grid. Any two adjacent line segments will be able to have three relative positions: left, straight, and right {L, S, R}. An ApeWorm cannot have two segments overlapping each other. Figure 1 shows a few possible ApeWorm configurations.


Figure 1


While ApeWorms are physically simple, their neural networks, or control software, can be as complex as we need. What should an ApeWorm know about itself, and what should it be allowed to know about an exemplar ApeWorm, if aping it is to be a non-trivial task requiring an extensible aping machine?

The ApeWorm will have one end designated the zero {0} end, with joints {1, 2, 3} and a {4} end. Each segment will be designated by its terminal joint, {1, 2, 3, 4}. Each joint will signal its relative position, going from the lower to the higher numbers, as L, S, or R (after training). The zero end and 1 joint will be able to sense their positions in the grid, again after training. For illustration purposes joints may be color designated: 1. Black 2. Green 3. Blue 4. Red. [Training adds a degree of complexity, which I suspect will be important as we move to systems more complex than ApeWorm]

How would the ApeWorm know about its exemplar? Note that it would be a trivial task, if exact information about the exemplar is known, to assign that information to the ApeWorm directly in software. But if we built two ApeWorms in the flesh, so to speak, we should have autonomous systems that require such information to cross space, so that information about position becomes encoded in some other form.

We will allow the ApeWorm to have two "eyes" on the Exemplar worm. The eyes (retinas) will be at the bottom and left sides of the grid. Each eye can detect segments that are broadside to it, and their orientation by row or column and by distance from the eye.

It is tempting to ignore the temporal aspect in the first build of ApeWorm. It seems difficult enough to construct a general aping ability when the minimum requirement is to detect the position of the Exemplar worm and then provide for the ApeWorm to conform itself to a copy of that. However, going forward we will want temporal input and temporal aping.

Consider that the aping instinct evolved in animals only after basic lower functions were in place. An animal has to sense the environment, coordinate itself, catch food and escape predators just to survive. All that puts a high premium on temporal capabilities. Aping sits atop all those capabilities, just as language and culture sit atop aping. We have freed our ApeWorms from the need to eat, but we run the danger of making our aping system hard to integrate into robots and more complex programs if it is unable to at least deal with time sequences.

Next: Constructing the ApeWorm World

Aping as the Basis of Intelligence

Logic was thought to be the key paradigm for human intelligence from the Aristotle to Wittgenstein, and then in the early days of Artificial Intelligence as implemented on computers.

Careful consideration shows that aping, the ability to copy an action of another individual, is more central to intelligence. It is as much a mammal trait as a human one. Formal exercises in logic are not needed for one animal to learn behavior from another. The complex neural circuitry created by evolutionary pressures to enhance aping potential eventually became the basis of human language and its formalized reasoning skills known as logic.

Aping consists of observing (usually visually) what some other individual is doing, and then copying that behavior. Humans do this so naturally that we seldom think about it consciously.  Unless trained to do so (by aping others), practicing logic is a much harder skill. Yet logic circuits are easy to implement with electronic parts. Aping another individual requires very complex systems within the brain.

This essay, and the accompanying computer programs and physical systems, is an attempt to create machines capable of aping, which then should be able to handle other complex behaviors usually associated with human intelligence, including language, mechanical skills, reasoning, and possibly consciousness.

The advantage of aping skills goes beyond helping to ensure the survival of an individual animal. With aping a behavior with survival benefits can become cultural, spanning an unlimited number of generations of animals. This without any genetic change specific to the behavior itself. A generalized pool of neurons can become capable of generating a wide and flexible variety of behaviors, giving a species a heavy advantage over animals that can only engage in behavior based on  pre-programmed neural systems.

Pre-language example. Consider a simple aping process and how it differs from paradigms like ordinary remembered (learned) behavior. An animal, a young wolf,  has always simply crossed a cold stream by simply plunging through it. One day it sees an older wolf stop at the stream and proceed along the bank to a point where there is a rock in the center of the stream. The older wolf jumps to the rock, then to the far bank, thereby avoiding getting wet while crossing. The young wolf then apes that behavior. Afterwards, when crossing the stream in that area (when not in hot pursuit of prey) the young wolf  diverts to the special crossing point.

Note how the aping behavior differs from what might be other reasons for using the rock to cross. The young wolf could learn of the rock crossing by exploration, then would remember the location of the crossing when appropriate. You could argue that following an older animal might be an instinct, so this is not a real example of aping, just a side effect of following.

Aping behavior might have originated in following behavior, and the basic ability to learn from experience, but as more complex situations are involved, it becomes clear that aping is a special case requiring special capabilities.

Supposing this simple example constitutes a form of aping, what can we say about how it is accomplished by the wolf brain? The wolf is capable of recognizing an externalized self. The example wolf, to the aping wolf, represents a possibility for itself. Metaphorically, the aping wolf can see itself crossing the stream using the rock when it sees the example wolf. So the wolf-brain is capable of contemplating (in a very simple, non-philosophic way) the future. It can choose to ape the example, or it can refuse to cross the stream, or it can ignore the example and get wet and cold crossing the stream.

Aping, in the infants of actual apes including humans, is probably automatic. Only at a later time will the young apes start choosing whether or not to ape particular behaviors. There is typically outside support for aping in human children. We reward them when we like their aping efforts, but may discourage them when we don't like their efforts (as when they engage in behaviors culturally reserved for adults).

Consider the "simple" modern act of making a piece of toast with a bread toasting appliance. A child is not likely to learn to do this by accidentally taking a slice of bread, sticking it into the appliance, and pushing down the lever. A child learns this by watching a example or exemplar. The child understands that doing what an exemplar does leads to the same result. The child sees that taking a piece of bread, placing it in a certain way in the appliance, and pressing the lever results in the making of toast. Usually, if sufficiently coordinated, a child can do this on the first try. Aping often results in some level of success in achieving a goal on the first try.

To ape an exemplar the aping child must already have similar levels of muscle control and sense of where its body is in relation to its surroundings. It must understand that aping the exemplar's movements (or other behaviors like vocalizing) achieves a desired result. If an aping attempt is made that does not achieve the desired results, a child may conclude that something is missing from the aping process, a subtlety or trick.
To ape, the brain must have a number of capabilities, which might be summed up as the ability to coordinate the body of the subject with the body of the exemplar.

In humans, when children are quick to ape adults, we call them intelligent. If they are slow to imitate adult behaviors, we call the slow or less intelligent.

Language learning example. Human babies start learning to imitate vocalizations soon after birth. Vocal fussing often results in rewards, like feeding, encouraging further vocalizations. The process may seem like a long one when mechanical devices have long existed that can record and play back sounds and language. There is a fundamental difference between the mechanical play back model and the aping vocalization model that is critical to the reconstruction of how humans are capable of intelligent behavior, including understanding how the world works. [I prefer Machine Understanding as a term for what we are working towards with computerized robots capable of aping, because Artificial Intelligence has largely ignored a set of crucial problems that logic circuits have difficulty handling].

The human brain does not appear to be a recording medium in the sense that a phonograph, magnetic tape machine, or silicon-memory based sound recording and playing device is. The human ability to immediately repeat a phrase of language or hum a melody does not arise from a like mechanism. Humans have fairly good memories, but a weak ability to memorize.

Consider the problem, for the human brain, of learning to repeat its first few spoken words, say "mama, papa, no, puppy, bye, milk." In the ear there are a number of sensory cells that respond to different frequencies of sound. They send nerve impulses towards the brain. At the other end we have the sound making machinery: lungs, vocal cords, and mouth. An infant screaming soon after birth indicates that machinery works, but is nothing like language. Months of hearing and experimental vocalizations follow. There may be feedback, positive or negative, by parents and other exemplars, as well as careful repetition of simple words in the hope this will induce aping by the child.

In the end, however, success is the ability of the child to say a word that sounds, to its own ear, like the kind of words spoken by the exemplars. It will not be identical, nor will it ever be identical, in its fine structure, in fact it will be identifiably different, an individual voice. Six words mastered, many more will follow, all aped after exemplars. In addition, the words will have meaning: they will coordinate with objects or actions in the external world (or internal world, as with "more" or "hurts").

The neural machine that coordinates heard words and the speaking of words with an "ego" must be very complex and very capable. Recreate that machine and we would have a model that, perhaps with some specialized variations, would be able to ape visual input to body positions, or anything else required of it.

Predictive Memory and Particle Filters

I keep thinking about the Particle Filters AI method, so might as well right some of it down. Note that in AI Particle Filters has a particular meaning, different from the physics for particles in a real-world simulation or in a video game. In AI filtering is a method for localization of an object, which could be a large physical object like a robot. It involves probabilistic methods (Monte Carlo techniques) for predicting where the object might be, combined with sensory feedback. Typically it requires a pre-existing map of the territory (which for a robot is real, but which could be highly abstract in other cases) and a method for matching the feedback to the map. It has been most notably used as an essential software component of the Google self-driving car. A somewhat more extensive overview can be found at Particle Filters.

The reason I keep thinking about it is because I started noticing how I consciously (and probably subconsciously) localize myself in the physical world. Most notably, when I am navigating in the dark in my house. I do this to avoid backtracking: turning on a light, going back to the prior switch & turning it off, etc. In any case I know my house well, but in just a few feet in the dark it is easy to get a bit off course and bump into furniture or a wall where I expected a doorway. The stairs can be tricky too.

So I walk a few feet in what I guess is the right direction based on experience. Then I reach out to touch what I expect is a wall or piece of furniture. Or I note a change from rug to bare floor, or spot an LED. Usually I get the feedback I expect, but sometimes I find I am off course and need to make a correction.

In Particle Filter Localization terms, I project a probabilistic range of particles representing where I, the human robot, might be next. Then I check. Often I know exactly where I am after a check, but sometimes all I know is that I am not near a wall or furniture piece. Then I make another mental projection, take a step or two depending on my expectations and level of caution, and try again to touch something. I go through similar mental gymnastics when driving a car or taking a daylight walk, but the feedback is visual rather than touch.

I went through a similar analysis process after reading Jeff Hawkins theory of predictive memory in On Intelligence. The theory is that the brain largely operates by making constant predictions, based on memory of historical patterns, and checking them against sensory input. When input varies from predictions, neurons send special signals about detecting novelty to higher brain centers that deal with novelty. For instance, as you read this your brain knows a number of words might be included in this zebra. See, you expected "sentence" or maybe "essay" or the like, you were not expecting "zebra", and so the normal flow of processing was interrupted.

I have not seen a neural network model for particle filters. It could be that the resemblance between how human brains seem to work is only superficially like particle filters. More generally we are talking about feedback systems, which can take a number of forms. Even very primitive animals that lack differentiated neurons have feedback systems, so there could be many types of neural feedback schemes operating in the human brain.

I often notice my dog doing something that makes me think that most mammals, including pre-sapiens homo species, can do that. Yet when these processes, say chasing a rabbit, are abstracted, they can become quite complex. I can imagine chasing a rabbit. I can plan to chase a rabbit into a trap. I can pursue something more abstract, say a solution to a construction problem or even a math problem in a way that resembles chasing a rabbit. Let's see, the answer to the solution of this differential equation has disappeared, it is not obvious. I'll beat the bushes, using three techniques known to help solve this type of equation, most-likely technique first. Hopefully one technique will put the problem in a form that I recognize from the standard map for solving such equations.

How would particle filters be implemented with neural networks? Let me know your thoughts.

Stanford AI Class wrap up

I managed to muddle my way through the free Internet version of the Stanford Introduction to Artificial Intelligence (AI) course. "Congratulations! You have successfully completed the Advanced Track of Introduction to Artificial Intelligence ..." says my Statement of Accomplishment.

Before putting on my analyst mask, I would like to thank Stanford University and particularly the instructors, Sebastian Thrun and Peter Norvig, for conducting the course. In particular, for making it free of charge. I am hoping they will leave the instruction videos up for a while, there are some I would like to go over again.

I got a good review of Bayes Rule, basic probability, and some simple machine learning algorithms. I had not worked on Planning algorithms before, so that was of some interest. Markov Models had always been a bit vague to me, so that section helped me nail down the idea. Games and game theory seem to have made little progress since the 1940's, but I guess they have to be covered, and I did get clear on how MinMax works. Computer vision seemed kind of weak, but then you can't assume students know basic optics and at least we learned how to recognize simple features. Robotics was a prior interest for me, and I did not know about Thrun's obvious favorite, Particle Filters, which are a useful paradigm for spatial positioning (aka localization).

The last two units were on Natural Language Processing, and that is a good place to start a critique (keeping in mind that all this material was introductory). Apparently you can do a lot of tricks processing language, both in the form of sounds/speech and written text, without the algorithms understanding anything. They showed a way to do pretty decent inter-ethnic language translations, but the usefulness depends on humans being able to understand at least one language.

Plenty of humans do plenty of things, including paid work, without understanding what they are doing. I suppose that could be called a form of artificial intelligence. Pay them and feed them and they'll keep up those activities. But when people do things without understanding (I am pretty sure some of my math teachers fell into that category), danger lurks.

The Google Car that drives itself around San Francisco (just like Science Fiction!) just demonstrates that driving a Porsche proves little about your intelligence capabilities. Robot auto-driving was a difficult problem for human engineers to solve. They were able to solve it because they understood a whole lotta stuff. Particle Filters, which involve probability techniques combined with sensory feedback to map and navigate an environment, are a cool part of the solution. If I say "I understand now: I have been walking through a structure, and to get to the kitchen I just turn left at the Picasso reproduction," I may be using the word understand in a way that compares well with what we call the AI capabilities of the Google Car. Still, I don't think the Car meets my criteria for machine understanding. The car might even translate from French to English for its human cargo, but I still classify it as dumb as a brick.

Hurray! Despite my advancing age, lack of a PhD., less than brilliant business model, and tendency to be interested in too many different things to be successful in this age of specialization, no one seems to have gotten to the essence of how humans understand, and are aware of, the world and themselves.

If the human brain, or its neural network subcomponents, did Particle Filters, how would that work? I know from much practice that bumping around in the dark can lead to orientation or disorientation, depending on circumstances. On the other hand the random micro-fine movements of the eye might be a physical way of generating randomness to test micro-hypotheses that we are not normally consciously aware of.

We sometimes say (hear Wittgenstein in my voice) that someone has a shallow understanding of something. "Smart enough to add columns of numbers, not smart enough to do accounting," or "Good at game basics, but unable to make strategic decisions." Let me put it another way: in some ways the course itself was an intelligence test. I imagine it would be very rough for anyone without a background in algebra and basic probability theory. The students in the class already knew a lot, and had to learn difficult things.

I want to know how our bodies, our brains, learn difficult things. The only way I will be able to be sure that I understand how that is done is if I can build a machine that can do the same thing.

Brain Modeling Computational Trajectory

SGI is a manufacturer of high-performance computers, what might be called small supercomputers. I was listening to their analyst day (I own stock in SGI) this morning and saw an interesting slide, which I reproduce here:


With regard to machine understanding, this is the direct assault method. At some point when the human brain is modeled in sufficient detail the construct should display human memory, intelligence, and even consciousness or self-awareness. It is conceivable that a detailed computer model might exhibit artificial intelligence or understanding but leave us still unable to comprehend the essence of what is happening. More likely we will be enlightened and therefore able to construct working machine entities that have true intelligence and understanding, but are not exactly modeled on the human nervous system.

There are different levels for modeling biological systems of neurons and brain matter. We might model on the atomic, molecular, sub-cellular, cellular, or neural-functional level. It is not clear what level of detail is assumed in the SGI projection. Best guess is that is has a model for individual neurons, but the complexity is added by adding additional neurons to the network. That would be the main difference between modeling a "cellular neocortical column," a mesocircuit, a rat brain, and a human brain.

SGI already makes computers for researchers in this field. Of course other vendors' computers can be used. The advantage SGI brings to the table today is the ability to build a large model in computer memory chips (the processors see a big, unified memory), as opposed to hard disks.

I would be very impressed if someone could correctly model a functioning rat brain by 2014. Keep in mind that just because the computer power is available to do it, does not mean that any given team's model is correct. I wonder what proof on concept would consist of? To test such a brain you would need a test environment. That might be a simulation, but it could also be robotic. Keep in mind that much of what a rat brain does relates not to what we think of as awareness of the environment, but to maintaining body functions.

Another issue is the initial state problem. Suppose that you "dissect" a rat so that you know the relative placement of every neuron in its central nervous system. Still, to make your model work, you would need a functional initial state for all the the cells. You need to know how synapses are weighted. Probably someone is working on the boot-up of mammal brains during embryonic development. Even just getting the genetically re-programmed neurotransmitter types for each synapse to each cell seems like a more difficult problem that making a generalized computer model based on a neural map plus a generalized neuron.

Apparently there is plenty of work on this project for everyone.

Naive Bayes Classifier in Python

I have been busy with many things: indexing a book on software management; trying to learn math things I would have learned when I was 18 or 19 if the Vietnam War had not made me decide to major in Political Science; advising investors about the value of Hansen Medical's robotics technology and Iteris's visual analysis software for cars and highway monitoring.

So even though I continue to think about MU, and to study, I have had nothing in particular worth reporting here. However, I came across a good introductory page on using Bayes probability with the Python programming language. If that is what you need, here it is:

Text Categorization and Classification in Python with Bayes' Theorem

And I would love to play with a Kinect, but where would the time come from?

Handling Complex Numbers

I have not been logging my MU activity very well. I am trying to develop a simple system to test my ideas. The first one bogged down and went nowhere. At the same time looking at a lot of possibly useful math, including Hilbert spaces.

Today was thinking about neural network handling of complex numbers. Decided to let others do my thinking for me and came upon this paper after a search. Reminds me that I think all science journals should be published on the net with no charge to read; that would accelerate the advances we all desire.

Enhanced Artificial Neural Networks Using Complex Numbers by Howard E. Michel and A. S. S. Awwal. I don't know if I can use their specific model, but it is good food for thought.

Wandering through the multi-dimensional abyss

I have not posted to this blog for a long time partly because I lost my focus on the Numenta program within the Machine Understanding arena, and partly because I have been distracted by the need to earn money, and even the political silly season. On a good note my friend Dan Hamburg got elected to be Mendocino County Supervisor, and the government of California can now approve budgets with a majority vote. Which does not mean they will start doing budgets on time, or balance them, or run the state well; but they could.

I found, slogging through Judea Pearl's Probabilistic Reasoning in Intelligent Systems
and Terrence Fine's Probability and Probabilistic Reasoning for Electrical Engineering
, that my mind kept fuzzing up. Maybe I am slowing down in my old age, but the real problem was that my last formal training in probability was when I was 19, and interpreting clinical trial p values is guestimate work. I had to regress to a simpler text than what I used in college, and I can recommend Finite Mathematics with Applications
by A. W. Goodman and J.S. Ratti for introductions to simple probability, conditional probability, Bayes' Theorem, and even Markov Chains that were simple enough for me to feel I really understood easy examples and the concepts themselves.

But my wanderings have been further afield than that. I continue to be fascinated with tensors, and got a lot out of Introduction to Tensor Calculus, Relativity and Cosmology
by D. F. Lawdon. Again, I never got to tensors in college (I ended up a Political Science major), and thinking I was brighter than I really was (brightness is mainly a function of prepartion, I now know), started off with mathematical treatments that were too abstract for me to do more than pretend to follow.

I have even got stuck on Maxwell's equations for electromagnetism. Now we all should admit that if we read broadly in math and science we don't take the time to really understand everything; we trust our fellows to have done their homework before a set of facts or an equation is presented in a paper or textbook. We may like to feel we agree with quantum physics, but who except for professional physicists have the time to really look at the data and the math in detail? I have always assumed that Maxwell's equations are correct, and that if I needed to I could look up the definitions of curl, etc., and do the math. But that is not the same thing as the deep understanding one gets from working in electromagnetics on a regular basis.

I have wandered farther afield than that, to Lie groups and Galois theory, which may have nothing to do with machine understanding. Nevertheless, I wander. And I keep coming back to what is known about the structure of the cortex, of the actual tangles of nerve cells themselves, and in particular to the way pyramidal cells span multiple layers of the cortex with their intricate axons and dendrites. How do you create a math that represents such a tangle? Skipping that, you can do funtional units as Numenta does, or you can try the AI tradition with its tradition trying to get the end results without understanding the details of how neurons actually get stuff done.

Right now I have little paid work going on, so I may be writing in the blog more often. If paid work becomes available, there will be more delays.

Seeing Predictive Memory Everywhere

Numenta cancelled its planned October class. I stopped working my way through their examples. I created the index for Microsoft Excel 2010 Inside Out. My stepson got married.

Yet all the while I've been watching how my mind works in light of the predictive memory theory. What good would memories be if they did not allow animals to make predictions that help with survival? I have watched my mind make mistakes, in reading for example: wait, that doesn't make sense, I read "farming" for framing. I watch my dog Hugo make decisions (mostly to not obey me). I watch other people make decisions.

I also continue to ponder how the system works. There are computer models like Numenta's, and biological models. When I study math part of me is assessing its utility for modeling machine understanding. Even reading the Excel book, which I really liked, got me to thinking about how advance Excel tools might be used to model neurons or probabilistic reasoning.

But I can't say I have any breakthroughs to report. I can't even say I am going to be writing this blog on a regular basis. There are fires that need to be put out, and fires that need to be lit.

I started on a simple demo program, I mean really simple demo program, just to get going on flowing data through nodes. I started it in Visual Basic, with the intent of also doing it in Python and at least one other language, maybe C++. I might try to restart the MU project their, or I might go back to working through the Numenta examples. No promises. But if I do manage to get anything done, I'll post it.

Understanding Probability and Probabilistic Reasoning

I think I mentioned earlier that I have had problems with the Numenta model as described in "Towards a Mathematical Theory of Cortical Micro-circuits" because of the use of probability-based mathematics. It seems to me that neurons are deterministic mechanisms. But I have noted in the past that I can be pretty dim-witted at times, and decided to study Numenta's HTM systems anyway. I reminded myself that quantum physics has two different formulations, one based on matrix algebra and the other on the Schrodinger equation. They both work, and some brilliant person showed that they are formally the same long, long ago. So when thinking about or solving problems you can use whichever is easiest or gives the best insights. The same way some physics and math problems are easier in polar coordinates than in rectangular coordinates.

Months ago I ground to a halt in my reading of Judea Pearl's Probabilistic Reasoning in Intelligent Systems, which provides much of the background to the Numenta discussion. Yesterday I decided to tackle it again and commenced reading at page 143. I noticed that some notation was ambiguous, which is typical of expert writers who assume their readers are right up with them. So I decided to go back and make sure that P(A,B) really does mean the probability that both A and B are true. I thought I'd make sure I understood the Bayes interpretation of probability as well.

I ended up reading starting at page 29, Chapter 2, Bayesian Inference, 2.1, Basic Concepts, 2.1.1 Probabilistic Formulation and Bayesian Inversion. Note that I took two semesters of logic and one semester of probability in college, and as part of my profession deal with biostatistics, the kind reported from clinical trials, on a regular basis. Note also that I have studied philosphic issues of quantum physics and even the math involved.

Yet when I read this simple introduction this time, the scales fell from my eyes, or from my cortical networks.

With probabilistic reasoning, it is fair to say that we are not talking about rolling dice (even though Pearl uses the familar probabilities of two-die rolls to illustrate some points).

We are talking about the math of pobability theory. For most practical purposes, that is the math of fractions. Third or fourth grade stuff. (I had a fifth grade teacher I hated, Mrs. Lopez, who was all about memorizing things. We memorized the decimal equivalents of about 50 common fractions. I knew I could always get the decimal equivalent by dividing, so considered this a stupid exercise.)

When thinking about human memory, you can safely substitute "percentage of like situations" for probability.

Updating the "percentage of like situations" based on experience makes sense. Since we can test for novel situations, like "both A and B" or "A and not C, given B", by multiplying, adding, or subtracting fractions, these updates may effect a chain of knowledge or deductions across the brain (or mind, if you prefer).

Calling all background information and assumptions a person has K (I don't know why K, maybe it stands for Knowledge), I quote Pearl page 30: "However, when the background information undergoes changes, we need to identify specifically the assumptions that account for our beliefs and articulate explicitly K or some of its elements."

Many Philosophers, notably Ludwig Wittgenstein, have shown how reasoning goes awry when we use one word to mean multiple things, or one thing that is vague or complex. We think we are being clear using logic symbols or math equations or tech speak. But when something is amiss, it may not be a problem with our reasoning. It may be that we need to update our background assumptions.

See also Bayes' theorem

Evaluating HTMs, Part 1

I am now going to evaluate the particular model of Hierarchical Temporal Memory (HTM) developed by Jeff Hawkins, Dileep George, and members of their team at Numenta, NuPIC. My key question will be: could HTMs serve as a basis for machine understanding (MU)? Since there are many subgoals on the way to true MU, I will be evaluating HTM capabilities on a number of issues that are usually within the realm of AI (artificial intelligence).

This will be a long project. The current plan is to read and critically summarize "Hierarchical Temporary Memory, Concepts, Theory, and Terminology " by Hawkins and George; read and comment on the "Getting Started with NuPIC Guide;" downloading and trying NuPIC, and then assessing what whether to continue on the project.

In preparation I have previously read On Intelligence by Jeff Hawkins twice; read and commented on Towards a Mathematical Theory of Cortical Micro-circuits by George and Hawkins [see early entries of this blog]; took a course in neural networks at SDSU long ago; and read From Neuron to Brain to get detailed information on biological neurons. I know how to write software and I am fairly good at math.

Still, my head hurts just thinking about it, but here we go (into "HTM Concepts, Theory, and Terminology'):

In the Introduction the authors remind us that the human mind/brain has capabilities that computers have so far been unable to duplicate. HTMs are a memory system that can learn to solve certain problems. They are organized as hierarchical systems of nodes. HTMs are currently implemented as software on traditional computer hardware. "The learning curve can be steep."

That is all ground I have already covered in this blog. My learning curve is probably going to be steeper than that of most students who would be interested in this topic, but hopefully watching me struggle will be helpful to at least a few people.

Vitamin D for Machine Understanding?

As usual, I have too many projects going at once, and this MU blog suffers. I have been slogging through Pearl's Probablistic Reasoning in Intelligent Systems; bookmark last left at the end of the section "2.1.4 Recursive Bayesian Updating." Also continuing Kuffler et al's From Neuron to Brain, where I just finished reading about GABA, a neurotransmitter for inhibitory synapses. I doubt the specific biochemistry of synapses will be much help in MU, but sometimes it gets me thinking, and sometimes I do some biotechnology analysis and investing, so the database accumulation can't hurt.

People are already applying some of Numenta's HTM technology. In case you missed it, Vitamin D, Inc. has a beta you can try, Vitamin D Video. It apparently can pick out objects from a webcam video stream. On the one hand that is very impressive, on the other hand it is a long, long way from machine understanding. Typical of my hands-off style, for years I contemplated how I could get my computer to watch my aquarium, and inentify and track the fish in it. I never came up with a good enough plan to try to implement.

The section in Kuffler I just started reading made me think about animal rights and machine understanding issues because one of the animal models for the neurotransmitter discussed, an LHRH analogue (or apparently now GnRH analogue), was based in part on studying frogs. I have tended to have the view that understanding and consciousness go together, and that only a few species of animals have much in the way of understanding. I was thinking about whether experimenting with a frog's nerves is a form of torture. In the past I would have said no, but I am not so certain. Frog neural systems are very, very complicated, if only a tiny fraction compared by size to human neural systems. Since I can't say with certainty how human understanding or consciousness work, I can't be certain if a frog understands anything, or if a lack of understanding implies lack of conciousness. Which may just put the animal rights debate on a different level: should animals have rights just because (if) they are conscious?

One great thing about working with computer models for "artificial intelligence" is that no one is being tortured, except maybe the computer programmers.

Review of Towards a Mathematical Theory of Cortical Micro-circuits

This is a report on my first reading of "Towards a Mathematical Theory of Cortical Micro-circuits" [Dileep George & Jeff Hawkins, PLoS Computational Biology, October 2009, Volume 5, Issue 10]. There were some bits of it that I had to treat as black boxes, but on the whole I found it comprehensible.

The core concern of the paper is to map a particular attempt to use computers to implement the Memory-Prediction framework of human intelligence to what is known about how nerve cells actually function in the cortex of the human brain. The specific machine methodology is called Hierarchical Temporal Memory (HTM). The article begins by explaining how HTM operates, then proceeds to map HTM processes to the cortex.

HTM structures are called nodes, but it should be noted that the nodes have extensive internal structure. They can have multiple inputs and outputs. They can have multiple "coincidence pattern matrices" and Markov chain processes. There are four major equations sets necessary for calibrating the nodes. I suggest you take a look at the HTM Technology Overview if you have not already. In particular, the authors assume the readers understand how Markov chains work, which is not obvious in this context.

It should be noted that HTM nodes are typically found in a hierarchy of nodes, and that information can move both up and down the hierarchy. The bottom of the hierarchy would typically correspond to sensory input, the top of the hierarchy to conceptual modes.

In fact, I found it easier to understand how the HTM worked internally when the authors started explaining the details in terms of neuron processes. The Markov chain likelihood circuit (Figure 4), in addition to mapping neurons to an aspect of an HTM node, makes it clearer that a Markov chain, in this context, is a set of information that has (or could have) predictive value. Markov chains are learned expectations about probabilities that events will occur in a time sequence.

This is a good example of a premise of the Memory-Prediction framework. The Markov chains are a sort of memory, yet they also are used to process information to make predictions. In computer science we tend to separate out memory from logical operations, but in the brain they are mostly inseparable.

As to the nervous system side of the mirror, I was not surprised to see a lot of the more complex work being performed by pyramidal cells. This neuron type appears to be quite differentiated from the more ordinary neurons that mainly relay information between muscles or sensory cells and the brain. Their very size and location in the cortex (particularly in layers 2, 3, and 5) should arose anyone's curiosity. Whether they really play the roles assigned to them here is not generally known, but hypotheses are made that should be testable.

The paper also covers the use of HTMs to recognize visual objects. Since much of the work done by Numenta and others using HTMs is in this field, I won't comment further here except to say that I found the work on the subjective contour effect particularly intriguing (example of subjective contours: Kanizsa triangle).

Merely categorizing visual objects is a more difficult problem that was originally thought by computer scientists working on Artificial Intelligence (AI). It is not learning about the successes with HTMs with pattern recognition that excites me. It is the way that it is done, and the promise that implies. I want to understand how the brain works in detail, but the ultimate goal is to understand intelligence, awareness, and understanding itself. The way HTMs process information, including the flowing of information both up and down the hierarchy, seems like a necessary condition for attacking these problems of higher intelligence. I don't think anyone thinks HTMs will prove to be entirely sufficient, but they seem like a good starting place for further physical and philosophical investigations.

On Bayesian Belief Propagation

Towards a Mathematical Understanding of Cortical Micro-circuits [Dileep George & Jeff Hawkins, PLoS Computational Biology, October 2009, Volume 5, Issue 10] assumes some knowledge of Bayesian Belief Propagation. I was hoping the article would explain this topic as it went along, but apparently, to AI specialists, this is a well-known area. I found the Wikipedia entry, belief propagation, jumps into mathematical formulism rather quickly. Some words of explanation without math symbols would probably help many people who are interested in machine understanding; that is my goal here. If some of you Bayesian belief propagation experts think I have it wrong, or want to add anything, feel free to add comments. I have ordered a copy of the classic work on the subject, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, and so hope to be more fluent in this area at a later date.

Bayesian Belief Propagation (BBP) was formulated by Judea Pearl, who later authored Probabilistic Reasoning in Intelligent Systems. It appears to be a system for determining unknown probabilities within certain types of complex systems. Let's start with a review of a very simple probability model.

Much of early probability work was formulated in terms of a pair of ordinary gambling dice, six-sided cubes with faces numbered one through six. Thrown many times, they make a nice distribution pattern of numbers two through twelve. It can be shown with a little math and logic that there are percentages that are expected for each of these numbers. If you make a large number of dice throws, say more than 100, and you get results that differ by very much from the expected distribution of results, you may be looking at one of two things. It is possible that you have a distribution skewed purely by chance. On the other hand, you might want to investigate whether your dice and throwing systems are physically constructed to be truly random in the results they give.

Now imagine a black box, say the size of a shoe box. If you shake the box and then stop shaking, it gives you a number. In fact, imagine a room with a variety of such boxes. You shake them, keeping tracks of the resulting numbers for each. If one box consistently gives numbers between 2 and 12 inclusive, and the frequencies for each number match the pattern that we believe two dice would give, then we can conclude that the internal mechanism of the box, whatever it is, has a probability distribution that is created by the same logic that creates the dice probability distribution. There might even be a tiny pair of nano-dice inside.

In computer programming we tend to think in purely causal mechanics. If something does not happen as expected, we have a bug in our program. But in the case of living groups of biological neurons we cannot (so far) look at the inner mechanics of each neuron while watching how the whole group operates. However, if we can control the inputs and read the outputs of a system, then, if we can do the math, we might figure out the probabilities of individual neurons firing given known inputs from the other neurons in the system.

Formally, BBP allows us to calculate the probability characteristics for nodes we can't observe directly, if we can observe enough of the rest of the system. A node does not need to be a neuron. In an AI system, a node is a processing unit with inputs, outputs, or both, that connect it to the system of nodes in question. In our case the nodes are part of a Bayesian network, also known as a belief network. Communication (or messaging) within the network is in terms of probabilities. (See also Bayesian probability)



Think of a network of three people, where each of the people represents a node. Let there be a piece of paper we will call the message (it is blank; the paper is the message, in this case). Let each person have a single die and the rule: if you have the message, throw the die. If it comes up a one, a two, or a three, pass the message to the left. Otherwise, pass it to the right. If we watch the game progress, the message moves around or back and forth in a random pattern. However, it is not a Bayesian network, because it is cyclical. Bayesian networks are not cyclic. Messages more only in one direction.

Now start a series of games in which the message is always handed to person 1 (node 1). Person 1 throws the die and passes the message to person 2 if the result is 3 or less, and to person 3 if the result is 3 or more. Person 2 also throws a die and simply holds the message if the result is 3 or less, but passes if to person 3 if the result is 3 or more. When person 3 gets a message the game is over.

It is a simple game and a tad boring, but if we focus on the propagation time, the number of die throws from the beginning of a game until the end of a game, we can see a set of variations. Over the long run they should form a probability distribution of the times.

Now change the rules slightly. Say person 2 forwards the message if the die comes out 2 or higher. The range of possible results does not change, but the probability distribution will.

Now suppose you know about this type of game, but you can't see the game because it is in a tent. You see a message handed in, and can time how long it is before the message is handed back out by person 3.

By keeping track of the times, you can get the probability distribution. That should tell you the exact rules of the game being played, including the probability that person 1 will give the message directly to person 3, and how long person 2 might keep the message before passing it.

In more complex networks, you need to pin down the probabilities at more of the nodes, if you want to be able to characterize the probability rules for an unknown node.

Which is what the BBP algorithm is supposed to do. It calculates the probabilities for an unobservable node based on the observed nodes.

Bottom Up Machine Understanding

The first paragraph of "Towards a Mathematical Theory of Cortical Micro-circuits" [Dileep George & Jeff Hawkins, PLoS Computational Biology, October 2009, Volume 5, Issue 10] states:

Understanding the computational and informtion processing roles of cortical circuitry is one of the oustanding problems in neuroscience. ... the data are not sufficient to derive a computational theory in a purely bottom-up fashion

My own cortex, probably not wanting to give itself a headache by proceeding too rapidly into what looks like a dense and difficult paper, immediately drifted off into thoughts on deriving a computational theory in a purely bottom-up fashion.

The closer we get to the physical bottom, the easier it seems to be to understand that the project might work. Suppose we could model an entire human brain at the molecular level. We imagine a scanner that can tell us, for a living person who we admit is intelligent and conscious, where each molecule is to a sufficient degree of exactitude. We also would have a computational system for the rules of molecular physics, and appropriate inputs and outputs.

Unless you believe that the human mind is not material (a dualist or idealist philosophic view), such a molecular-detail model should run like a human brain. At first it should think (and talk and act, to the extent the outputs allow) exactly like the person whose brain was scanned.

However, that does not mean scientists would understand how the brain works, or how a computational machine could exhibit understanding. Reproducing a phenomena and understanding a phenomena are not the same thing. The advantage of such a molecular computional brain model would be that we could run experiments on it in a way that could not be done on human beings or even on other mammals. We could start inputting and tracing data flows. We could interrupt and view the model in detail at any time. We could change parameters and isolate subsystems. Perhaps, further in the future, such a model could even be constructed without having to scan a human brain for the initial state.

At present, for a bottoms-up approach that might actually be workable in less than a decade, we would probably want to do a neuron-by-neuron model (probably with all the non-neural supporting cells in the brain as well). However, a lot of new issues arise even at this seemingly low level, even if we presume we have some way to scan into the model all of the often-complicated axon to dendrite paths and synapses. If learning is indeed based on synapse strength (Hebbian hypothesis), we would need both a good model for synapse dynamics and a detailed original state of synapses. This would require modeling the synapses themselves at the molecular level, or perhaps one-level up at the some molecular aggregate level. In effect it would not be possible to model an adult brain that has exhibited intelligence and understanding. We would need to start with a baby brain and hope that it would go through a pattern of neural information development similar to that of a human child.

A complete neural-level model would be much easier to test intelligence hypothesese on than a molecular-level model. It would not in itelf indicate that we understand how humans understand the world. By running the model in two parallel instances (with identical starting states), but with selected malfunctions, we could probably isolate most of the sub-systems required for intelligence. This should help us build a comprehensible picture of how intelligence operates once it is established and of how it can be constructed by neural circuits from whatever the raw ingredients of intelligence turn out to be.

Despite our lack of such complete bottom-up models, I don't think it is too early to try to reconstruct how the brain works, or how to make machines intelligent. The paper outlines the HTM approach to this subject. HTM was based on a great deal of prior work in neuroscience and in modeling neural aggregates with computers. Often in science success has come from the combination of bottom-up and top-down approaches. Biological species, and fossil species, were long catalogued and studied before Darwin's theory of evolution revealed the connections between species. Darwin did not invent the concept of evolution of species, or of inherited traits, which many scientists already believed were shown by the fossil record and living organisms. He added the concept of natural selection, and suddenly evolution made sense. The whole picture popped into view in a way that any intelligent human being could see.

Synaptic Transmission Puzzle

"Over the past twenty years, a wide variety of modes of synaptic transmission have been discovered, in addition to simple, chemically mediated increases in permeability leading to excitation and inhibiation. Such modes of transmission include electrical exictation and inhibition, combined chemical-electrical synapses, chemical synapted changes produced by reductions in memberane permeability, and prolonged synaptic potentials mediated by chemical reactions in the postsynaptic cell."
-- page 208, From Neuron to Brain, Second Edition, by Stephen W. Kuffler, John G. Nicholls, and A. Robert Martin; 1984, Sinauer Associates, Inc., Suderland, Massachusetts

In Jeff Hawkins' On Intelligence, he presents a predictive-memory model for human intelligence which he believes can serve as a basis for intelligent machines [which I call machine understanding to emphasize that the machines would not merely be mimicking surface features of human intelligence, as is the case with, for example, expert systems]. I agree with him that while neuroscience and computing science have made great progress in understanding many details of the human brain, and in writing software, we need to have an overview that allows us to make progress on the centrals issues that interest us. Thus, in Hawkins' model, he does not worry about the exact nature of neural synapses.

But at the very least, we should be aware of how complicated human synapses are. This should allow us a greater freedom of thought when modeling the mechanics of machine understanding than if we simply assumed synapses all work alike. I suspect the Hebbian learning model for neurons would benefit from considering that the real world may complex, and in that complexity there may be keys to progress that we leave out with overly simple models.

What struck me most about the above paragraph about synapses, however, is the role played by evolution. Charles Darwin wrote about the process of species creation, but we have long grown used to the idea that evolution takes place on a molecular level. Each nerve cell, presumably, contains the full set of genes of the organism, but many different types of synapses are manifested. There must be controlling, blueprint genes that tell the cells which typese of synapses they are to form as they develop. In turn, we can expect that many different blueprints have been tried over the last four million years or so. The most successful gave their human organisms survival advantages.

It would be interesting to know how much synaptic variation exists in the current human population. Is this variation responsible, or partly responsible, for variations in basic intelligence capabilities of human beings?

This brings us back to the hard-wiring versus plastic debate. Human beings are very adaptable, as is shown by their ability to learn any of thousands of human languages as a child. We tend to think that we are very plastic and programmable creatures. But nerve transmission speeds and synaptic types are hard wired, as is the basic design of the brain. One might say we have the hard-wired capability to be flexible.

And that is what we aim to build into the new machines: hard wiring (a stable system with a design we understand and can reproduce) that is capable of showing the flexibility required to exhibit intelligence and understanding.

The Amazing Genetics of Nerves

I'll start with a quote from Stephen W. Kuffler, John G. Nicholls and A. Robert Martin's From Neuron to Brain (second edition, page 177; 1984 Sinauer Associates Inc.):

"... conduction velocity plays a significant role in the scheme of organization of the nervous system. It varies by a factor of more than 100 in nerve fibers that transmit different information content. In general, nerves that conduct most rapidly (more than 100 m/sec) are involved in mediating rapid reflexes, such as those used for regulating posture. Slower conduction velocities are associated with less urgent tasks such as regulating the distribution of blood flow to various parts of the body, controlling the secrection of glands, or regulating the tone of visceral organs."

Presumably much of this fine-tuning was achieved in mammals long before humans evolved. It is a great example of what can be achieved with evolution through natural selection. Apparently there is a cost to speedy nerve connections. There must be genes that can be turned on to produce extra structural elements (proteins, fats, etc) that speed up transmission velocity. There must be controlling genes that turn the structural genes on and off during nerve development, as appropriate.

Given these possibilities, there must be an overall genetic blueprint for nerve transmission velocity types. This blueprint would have been fine-tuned over time through the survival of the fittest. Having high cost, fast transmission where needed, and low-cost slower transmission where that will do, would give a slight evolutionary advantage.

The issue of transmission speed would be relatively simple comparted to creating a genetic blueprint for the orders-of-magnitude more complex human brain. Yet the construction process would be similar. A variety of brain cell types were already developed by mammals and primates. Probably (please comment if you know!) more cell types, including synapse types, evolved (from the usual mutation + selection process) for the human brain. The overall structure of the brain could be one blueprint, but it is an extremely complicated one. We know it involves axons and dendrites often both connecting to neighboring neurons and running long distances to connect to neurons after bypassing thousands, millions, or billions of closer neighbors.

This could only happen through evolution. Again, we have Charles Darwin to thank for setting us on the right path to understanding both nature and ourselves. As to machine understanding, the human brain is our best blueprint.

How I Spent My Summer

Obviously I have not spent time on the blog lately. I had a busy time indexing a technology book, and when that was done was behind on many other things. I have also been doing a considerable amount of studying on issues of machine understanding, but delayed my project of going through On Intelligence by Jeff Hawkins and explicating it. I have read it twice now; I am not ready for a third read. I did look at the web sites associated with the Hawkin's model, and will report on them when I have a chance to study them in more detail.

I am reading, or wading through, Neuron to Brain by Stephen W. Kuffler et al. The chapter on the structure of the visual cortex was near the front, which made interesting compare and contrast reading with On Intelligence. But I did not make the effort to write up my thoughts, and that must now wait for a vacation from other duties.

I have become better at tensor analysis, which had the added benefit of making Einstein's "The Foundation of the General Theory of Relativity" readable. I don't think tensors are going to be good models for neural networks, but the concept of invariance is well-developed with tensors and it seems central to the topic of maching understanding. By thinking about the structure of neurons in the brain, and invariance, I hope to at least exercise my mind in the vicinity of the problem.

Today I am fascinated by the possible application of metrics, inner-product spaces, and invariant angles to the question of how human babies construct their mental world. I was struck by the important of coordinating tactile senses with vision and auditory sensation. The main apparatus for this is the human arm. Excluding the hand, the arm consists of two big angles, one at the shoulder, the other at the elbow. There are a lot of degrees of freedom therein. To grab objects requires a lot of coordination. The angles are going to vary with muscle tone, which is controlled by the motor areas of the brain. But the general space should be an inner-product space. Changes of coordinates, say if the coordinate system was directed from the eyes, would not change the angles, lengths, or positions of the hands.

It makes me think about how flexible the human mind can be, and then contrast with how rigid people's thoughts can be. Once we learn about Euclidean space we tend to think we are walking around in it, with the ground as a two-dimensional plane. The idea that a coordinate system could be anchored by an imaginary line perpendicular to a line connnected the eyes strikes us as peculiar. Yet we use that coordinate system constantly, perhaps more than any other.

Numenta

While I am getting up to speed (a slow process, given my other commitments), there is no reason for someone who is interested in this topic to zoom to the cutting edge. A good place for a jump start, after you have read On Intelligence, is the Numenta web site. You can also check out the OnIntelligence.org web site, which sponsors some forums on Hawkins's predictive memory model of intelligence and has some links to related materials. Today I'll just describe what is at the Numenta site.

If you have not read On Intelligence, in particular Chapter 6, "How the Cortex Works," you can still get a lot out of reading materials at the Numenta site. However, I would advise reading the book first, old fashioned as that may sound.

You might be ready to jump right into "Hierarchical Temporal Memory Concepts, Theory and Terminology" by Jeff Hawkins and Dileep George. You are going to want to read it at some point. It is focused on HTMs (Hierarchical Temporal Memory technology). This is an attempt to implement the model of the human brain cortex, but in data-processing rather than biological terms. It is also the main technology being developed at Numenta.

On the other hand you may want to look at the HTM Technology Overview first. It is short and sweet, and also has a link to Problems That Fit HTMs. Of course if the HTM actually is a good model for the human cortex, HTMs should do well at problems that people (and often other mammals) do well at, but that computers, even using AI (artificial intelligence) techniques are not good at.

If you are a hands-on person, you could skip the theory and start with one of the software development kits. One is Vision Software, the other is NuPIC which now includes Vision. However, the NuPIC web page has a link to technical resources that would be important if you get serious about working with HTMs.

I'll write more about Numenta, just as I'll be writing more commentary on machine understanding and intelligence in general. I will be approaching HTMs from a critical perspective: do they implement a predictive memory model; is this the best implementation; and is the model really central to biological intelligence and machine understanding.

On Passions and Machine Understanding

Today I started reading On Intelligence by Jeff Hawkins for the third time. The first time I read the book I was impressed. The second time I read the book I believe I understood the more difficult parts. This time I am reading the book with two agendas: commenting on it in-depth, and developing my own ideas, complete with a mathematical framework, for machine understanding.

On page 1 of the prologue Jeff writes:

But I have a second passion that predates my interest in computers, one I view as more important. I am crazy about brains. I want to understand how the brain works, not just from a philosophical perspective, not just in a general way, but in a detailed nuts and bolts engineering way.

I suffer from passion attention deficit disorder (PADD). I am of the same temporal generation as Jeff Hawkins, a child of the space, computer, and nuclear age. I cannot even remember when I first wondered how people can be intelligent and conscious. Such wonder was in the air of the science fictions novels I read as a child. But I have other passions as well. So I have engaged in a variety of pursuits over the decades, only once in a while returning to the contemplation of machine understanding and its related issues of biology, philosophy, and technology.

Hopefully my interest in literature and writing, history, gardening, politics, and art will bring something to this enterprise. If nothing else, I can chronicle the field's development. And I hope to range into any topic that could be helpful in the endeavor, not just neuroscience and computer technology.