Neocortex - The Neuronal Bestseller

posted at Sat 22nd January 2011


All mammals have it. For example humans and dolphins have it. Sharks don't have it. Birds and reptiles do not have it neither. Since its market entry 100-150 million years ago in human brain regions it had undergone the biggest expansion of all brain regions and now reaches about 80% of the total human brain mass.

We are talking about the Neocortex, also called Neopallium (“new mantel”) or Isocortex (“equal rind”). It is the outer layer of the cerebral cortex which consists out of the Neocortex and the Allocortex. The human Neocortex is a 2.5mm thin sheet of neural tissue with about 30 billion neurons.

Animals with a cortex (e.g. dogs and cats) play more when they are young than animals without it. Probably this can be interpretated as a learning phase of the cortex. Higher cognitive functions as visual object recognition, understanding language, or action planning seem to be performed by the Neocortex. For this scientists would like to understand how the Neocortex works.

One interesting measure for animals is the Neocortex ratio: the ratio of the volume of the Neocortex compared to the total brain volume size without the Neocortex. The Neocortex ratio for humans is about 4.1. Animals having a high Neocortex ratio seem to be able to maintain social relationships to a larger number of other individuals (see Dunbar's number).

Neocortex: From WetWare to HardWare

One approach to understand what the Neocortex does could be to understand what its basic building blocks compute. The Neocortex is not only structured into six layers. Vernon Benjamin Mountcastle discovered in the 1950s that it also has a columnar organization: there are micro- and macro- (also called hyper-) columns.

Micro columns consist out of approximately 100 neurons and have a diameter of about 40-50 µm (1 µm = 0.001 mm). Macro columns which are also called cortical modules are bundles of about ~70 micro columns and have a diameter of about 500 µm (0.5 mm). Neurons within a minicolumn seem to encode similar features, whereas a hypercolumn “denotes a unit containing a full set of values for any given set of receptive field parameters”.

There are at least two interesting efforts one should know that focus on the understanding of the Neocortical function and especially its cortical modules. The first one is the Blue Brain Project - a scientific 10 year project headed by Henry Markram. The second one is Numenta - a company founded by Jeff Hawkins and tries to apply the Memory-Prediction theory of the neorcortical function to realworld applications.

Endeavor 1: The Blue Brain Project

There are a lot of videos out there by the Blue Brain Project team members starting from a high-level view on the project up to a talk with many details.

For the interested reader …aeh… watcher I suggest the following videos in the presented order (top → down introduction):

Especially video 3 gives a good impression how detailed the Blue Brain Project team tries to model the Neocortex: in his talk Markram mentions that they differ between 50 morphological classes of neurons in each of the 6 layers (i.e. consider 300 different neuron types based on their morphology/structure). And as if this was not bad enough one can find different electrical bevahior subclasses in each of these 300 morphological classes since the neurons have typically about 20-40 different ion channel types out of a pool of 200 possible ion channel types.

What I was mainly interested is whether the team has already a hypothesis of what the Neocortex does: what does it compute and/or represent? But their website is rather empty concerning this question. Nevertheless, in video 3 Markram comes clean and at least mentions an interesting new idea which he calls a “copernican revolution from soma-centric to dendro-centric representations”:

Currently the action potential paradigm in neuroscience states:

  1. dendrites process information for somatic spike output and
  2. spatio-temporal patterns of spikes are the neuronal correlate of perceptions.

His new idea:

  1. perceptions before the spikes: the neuronal correlate of perceptions are already the activations at the dendrites
  2. spatio-temporal patterns of spikes are just there to maintain and animate perceptions
  3. the resting potential has something to do with the duration of perceptions
  4. the neocortical circuit provides the rules to build and animate dendritic objects, i.e. neuronal correlates of real world entities

On the one hand I think the idea is interesting since nobody has mentioned it before and it could be a good alternative theory of the neuronal correlate of perceptions. On the other hand it is a little bit philosophical since it reaches into the realm of the consciousness / awareness character of perceptions.

I see the Blue Brain Project at one end of a spectrum of different endeavors to understand the Neocortex: it tries to model it as exactly as possible.

At the other end, where we just focus on copying the idea of what it does I see the …

Endeavor 2: HTM theory

The Hierarchical Temporal Memory (HTM) theory of the Neocortex provides a closed theory of what the Neocortex could do.

It was first described by Jeff Hawkins in his book On Intelligence which I occasionally found in a bookstore many years ago when searching for a book to read in my summer holidays at the beach. I really enjoyed it!

Watch these videos, it provides you the newest ideas of the theory:

The theory was recently augmented and is now called HTM Cortical Learning Algorithms. It is described in a nutshell in Wikipedia's article about Hierarchical Temporal Memory (HTM) as well. The details can be read in a new document published by Numenta directly: HTM Cortical Learning Algorithms.

Here are the key ideas of HTM:

  • the Neocortex is a hierarchical sequence memory: it learns typical temporal sequences of inputs in an unsupervised manner and can later be used for re-recognizing these
  • the basic building blocks are so called HTM nodes which consist out of multiple cells. Regions are composed out of many nodes and are connected in a hierarchical manner
  • the nodes are clearly motivated by the columnar organization of the biological Neocortex
  • the cells in a node get their input from a small restricted input area. Thus the concept of receptive fields is adopted in the theory
  • the input is classified and represented by a sparse population code: given similar inputs, a similar sparse subset of nodes will become active
  • sparsity is implemented by lateral inhibition - another concept stolen from biology
  • sequence learning: a cell A that is currently active forms connections to nearby cells that were active before. If later these other cells are active, A switches to the so called predictive state and thereby it represents a prediction of what will happen next
  • the response of a region to an input includes nodes (columns) in both active and predictive states. The predictive states lead to greater temporal stability seen by the parent region
  • the output of a region is its node/cell activation state

Here is the summary from Numenta what happens in one time step:

In summary, when a new input arrives, it leads to a sparse set of active columns. One or more of the cells in each column become active, these in turn cause other cells to enter a predictive state through learned connections between cells in the region. The cells activated by connections within the region constitute a prediction of what is likely to happen next. When the next feed-forward input arrives, it selects another sparse set of active columns. If a newly active column is unexpected, meaning it was not predicted by any cells, it will activate all the cells in the columns. If a newly active column has one or more predicted cells, only those cells will become active. The output of a region is the activity of all cells in the region, including the cells active because of feed-forward input and the cells active in the predictive state.

Source: Numenta's document on HTM Cortical Learning Algorithms

Not enough? Continue here! - More material

public/neocortex.txt · Last modified: 2012/02/19 15:36 (external edit) · []
Recent changes RSS feed Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki