stuck

Maybe we need the equivalent of a quantum hypothesis for the brain, in the sense that it may not be possible to construct a corrrect model based on classic principles, no matter how complex.

Stanford AI Class thoughts, and a brief tour of AI history

"people who face a difficult question often answer an easier one instead, without realizing it"
— Daniel Kahneman,

Despite time management issues (which will only get worse this week) I managed to struggle through the first two weeks, or four units, of the online Stanford introduction to artificial intelligence course.

In the past I had already tried to struggle through Judea Pearl's Probabilistic Reasoning in Intelligent Systems. That was published well over 20 years ago, and yet this course uses many of the same examples. The course is much more about working out actual examples; it is practical, not so theoretical. We've covered Bayes networks and conditional probability, both concepts I had already learned because Numenta was using them. Pearl's book contains a lot of material about wrong directions to take; the Stanford course seems to be focussed on what actually works, at least for Google.

My impression is still that the Stanford AI paradigm, while very practical, is not going to provide the core methods for truly intelligent machines, which I characterize as machine understanding. I think this largely because I am ancient and have watched quite a few AI paradigms come and go over the decades.

When AI got started, let's say in the 1950's, there was an obsession with logic as the highest form of human intelligence (often equated with reasoning). That computers operated with logic circuits seemed a natural match. Math guys who were good at logic tended to deride other human activities as less difficult and requiring less intelligence. Certain problems, including games with limited event spaces (like checkers), could be solved more rapidly by computers (once a human had written an appropriate program) than by humans. By the sixties, at latest by the seventies, computers running AI programs would be smarter than humans. In retrospect, this was idiotic, but the brightest minds of those times believed it.

One paradigm that showed some utility was expert systems. To create one of these, experts (typical example: a doctor making a diagnosis) were consulted to find out how they made decisions. Then a flow chart was created to allow a computer program to make a decision in a similar manner. As a result a computer might appear intelligent, especially if provided by the then more difficult trick of an audio imitation voice output, but today no one would call such a system intelligent. That is no more intelligent than the early punch card sorters that once served as input and output for computers that ran on vacuum tubes.

In the 1980's there was a big push in artificial neural networks. This actually was a major step towards machines being able to imitate human brain functions. It is not a defunct field. Some practical devices today work with technologies developed in that era. But scaling, the problems grew faster than the solutions. No one could build an artificial human brain out of the old artificial neural networks. We know that if we can exactly model a human brain, down to the molecular (maybe even atomic) level, we should get true artificial intelligence. But simplistic systems of neurons and synapses are not easy to assemble into a funcioning human brain analog.

The Stanford model for AI has been widely applied to real world problems, with considerable success. This probabilistic model allows it to deal with more complex data than the old logic and expert system paradigms ever could. Machine systems really can learn new things about their environment and store and organize that information in a way that allows for practical decision making. Clearly that is one thing human brains can do, and it is a lot more difficult than playing in a set-piece world like tic-tac-toe or even chess.

Sad as the state of human reasoning can be at times, and as slow as we are to learn new lessons, and as proud as we are of our least bouts of creativity, (and as much as we may occasionally ignore the rule against run-on sentences), I think the Stanford model is not, by itself, going to lead to machine understanding. The human brain has a lot of very interesting structures at the gross level and at the synaptic level. Neurologists have not yet deciphered them. Their initial "programming," or hard-wiring is purely the result of human evolution.

When is imitated intelligence real intelligent? When does a machine (or a human, for that matter) understand something, as opposed to merely changing internal memory to reflect the external reality?

Then again, maybe a Stanford model computer/program/input/output system would have done better at the Stanford AI course than I have. I certainly have not been getting all the quizzes and homework problems right on the first try. On the other hand, I think it will be some good long time before a machine can read, say, a book on neurology and carry on an extended intelligent conversation about it.

Brain Modeling Computational Trajectory

SGI is a manufacturer of high-performance computers, what might be called small supercomputers. I was listening to their analyst day (I own stock in SGI) this morning and saw an interesting slide, which I reproduce here:


With regard to machine understanding, this is the direct assault method. At some point when the human brain is modeled in sufficient detail the construct should display human memory, intelligence, and even consciousness or self-awareness. It is conceivable that a detailed computer model might exhibit artificial intelligence or understanding but leave us still unable to comprehend the essence of what is happening. More likely we will be enlightened and therefore able to construct working machine entities that have true intelligence and understanding, but are not exactly modeled on the human nervous system.

There are different levels for modeling biological systems of neurons and brain matter. We might model on the atomic, molecular, sub-cellular, cellular, or neural-functional level. It is not clear what level of detail is assumed in the SGI projection. Best guess is that is has a model for individual neurons, but the complexity is added by adding additional neurons to the network. That would be the main difference between modeling a "cellular neocortical column," a mesocircuit, a rat brain, and a human brain.

SGI already makes computers for researchers in this field. Of course other vendors' computers can be used. The advantage SGI brings to the table today is the ability to build a large model in computer memory chips (the processors see a big, unified memory), as opposed to hard disks.

I would be very impressed if someone could correctly model a functioning rat brain by 2014. Keep in mind that just because the computer power is available to do it, does not mean that any given team's model is correct. I wonder what proof on concept would consist of? To test such a brain you would need a test environment. That might be a simulation, but it could also be robotic. Keep in mind that much of what a rat brain does relates not to what we think of as awareness of the environment, but to maintaining body functions.

Another issue is the initial state problem. Suppose that you "dissect" a rat so that you know the relative placement of every neuron in its central nervous system. Still, to make your model work, you would need a functional initial state for all the the cells. You need to know how synapses are weighted. Probably someone is working on the boot-up of mammal brains during embryonic development. Even just getting the genetically re-programmed neurotransmitter types for each synapse to each cell seems like a more difficult problem that making a generalized computer model based on a neural map plus a generalized neuron.

Apparently there is plenty of work on this project for everyone.

Stanford AI course

I signed up for the free Stanford course, An Introduction to Artificial Intelligence to be taught by Sebastian Thrun and Peter Norvig this fall. I figure it can't hurt, and free is a good price.

Naive Bayes Classifier in Python

I have been busy with many things: indexing a book on software management; trying to learn math things I would have learned when I was 18 or 19 if the Vietnam War had not made me decide to major in Political Science; advising investors about the value of Hansen Medical's robotics technology and Iteris's visual analysis software for cars and highway monitoring.

So even though I continue to think about MU, and to study, I have had nothing in particular worth reporting here. However, I came across a good introductory page on using Bayes probability with the Python programming language. If that is what you need, here it is:

Text Categorization and Classification in Python with Bayes' Theorem

And I would love to play with a Kinect, but where would the time come from?

Handling Complex Numbers

I have not been logging my MU activity very well. I am trying to develop a simple system to test my ideas. The first one bogged down and went nowhere. At the same time looking at a lot of possibly useful math, including Hilbert spaces.

Today was thinking about neural network handling of complex numbers. Decided to let others do my thinking for me and came upon this paper after a search. Reminds me that I think all science journals should be published on the net with no charge to read; that would accelerate the advances we all desire.

Enhanced Artificial Neural Networks Using Complex Numbers by Howard E. Michel and A. S. S. Awwal. I don't know if I can use their specific model, but it is good food for thought.

gradient covariance

Thinking, thinking, thinking. Trying to work towards an example.

Spent more time than I meant making sure I understand how gradients act, their covariant nature under coordinate changes. Wrote out a "simple" example, just a 2D transformation of the gradient of a simple function. But it still took awhile. I made an arithmetic, an algebra, and a calculus mistake the first time through; am I ever rusty!

If you are looking for an example of covariance of a tensor or vector, try it:

gradient covariance

Now to make neuron-like structures do the math for me ...