About Self-Organized Maps

Introduction

The Self-Organizing Map (SOM) is an unsupervised neural network learning method. The base model of it is developed by Teuvo Kohonen in 1982. It defines a mapping from high-dimensional data into a lower dimension grid so that the input data features and neighbor relations are preserved as well as possible. This new representation is often easier to handle. A common way is to map the data into two dimensions which is easy to visualize.

The Principle

There are various different SOM models. The base model of Self-Organizing Map defines a mapping from d-dimensional real valued input space with input vectors xi ∈ ℜd into a lower dimensional map grid. A parametric reference vector mj of the same dimension than the input data vectors is associated for each node nj on the map. An input vector is mapped into a map node to whose parameter vector it has the smallest distance. Usually the Euclidean distance is used, but also others are possible.

Learning Algorithm

At first the map reference vectors are initialized e.g. with random vectors in the input space. Then the map is trained with the input vectors. All input vectors are looped through several times. For each input vector xi the best matching node nc is found and the reference vector of that node and its neighbors' are adapted towards the input vector using formula 

    mj = mj + α h(c,j)(xi - mj) .
α is a learning factor, which usually decreases during training.h(c,j) is a neighborhood function, where c is the index of the winning node and j is iterated over the map grid. Typical neighborhoods are bubble neighborhood for which 
    h(c,j) = 1, when ||nc - nj|| ≤ σ ,
             0, otherwise
and gaussian neighborhood, for which h(c,j) is defined as
    h(c,j) = e-||nc - nj||2 / 2σ2.
σ is a function determining the neighborhood radius. Usually it decreases towards one during training. The distance between nodes nk is often calculated using Euclidean metric and it depends on the used topology. Most common topologies are rectangular and hexagonal shown in the Figure 1 below.

Figure 1. Rectangular and hexagonal 4x4 map grids. Node (1,1) is colored black and its neighbors inside radius 1 are colored grey.

Further Reading