Kohonen Self-Organizing Maps.*;
This example categorizes characters of different fonts into groups of the same letter (or general appearance).
In Code Examples 1 and 2, only simple vectors of a few elements were used. Example 3 uses much longer vectors, that we would interpret in two dimensions, though the code only sees as individual rows.
At first, I had some trouble trying to grasp the architecture of a net like this; on the surface one would think that a second dimension had been added to the arrays, complicating the entire algorithm. However, after more careful examination, there's still only the single dimension.
At first glance, one would expect input like this to entail two dimensions, vertical and horizontal ( 7 x 9 ):
But the truth is there's still only one dimension, and nothing has really changed from the previous examples (1 & 2) except the size of the input.
The example input from above is actually only one row of data (63 elements):
The only part of this example that is more complicated is the structure and updating of the weights. In last two code examples for self-organizing maps, the the weights were updated in a linear fashion. In other words, after computing input, the winning node was updated, as were its neighbors to left and right (visualizing the nodes grouped in one straight line). However, Example 3 entails updating the winning node, the neighbors to the left and right, and the neighbors above and below (visualizing the nodes grouped in two-dimensions). The 5 x 5 structure used in the code would look like this:
Let's think of an example "winner", node 11. Not only will its weights be updated, but also those of 10 and 12. What's more, the weights of the three nodes above 11, which are 5, 6, and 7, will be updated, as well as those of the three nodes below 11, which are 15, 16 and 17. This is what is referred to as a rectangular neighborhood.
After a preset number of iterations, the neighborhood (radius) is reduced to only the "winning" nodes.
Of course, we're not limited to only a rectangular neighborhood, a linear neighborhood is still a good approach. A diamond shaped neighborhood, or circular, or almost any configuration could be used. The end results will vary a little, and experimentation is the best way to see the differences.
Iterations: 101 Neighborhood radius reduced after 80 iterations. Clusters for training input: Vector (Pattern 0, A1) fits into category 20 Vector (Pattern 1, B1) fits into category 18 Vector (Pattern 2, C1) fits into category 21 Vector (Pattern 3, D1) fits into category 18 Vector (Pattern 4, E1) fits into category 18 Vector (Pattern 5, J1) fits into category 10 Vector (Pattern 6, K1) fits into category 18 Vector (Pattern 7, A2) fits into category 20 Vector (Pattern 8, B2) fits into category 8 Vector (Pattern 9, C2) fits into category 21 Vector (Pattern 10, D2) fits into category 11 Vector (Pattern 11, E2) fits into category 8 Vector (Pattern 12, J2) fits into category 10 Vector (Pattern 13, K2) fits into category 2 Vector (Pattern 14, A3) fits into category 20 Vector (Pattern 15, B3) fits into category 18 Vector (Pattern 16, C3) fits into category 21 Vector (Pattern 17, D3) fits into category 20 Vector (Pattern 18, E3) fits into category 18 Vector (Pattern 19, J3) fits into category 10 Vector (Pattern 20, K3) fits into category 18
It might take a little interpreting. A1, A2 and A3 are all A's. B1, B2 and B3 are all B's, and so on. Note the A's all got grouped into the same category. Two of the B's were grouped together. All the C's are together. Same with the J's
The D's didn't get grouped together.
The idea, of course, is that all letters should be neatly grouped. There are a number of factors that go into the successes and errors of the pattern recognition process. One is the resolution of the characters (i.e.: larger resolutions would yield better results because the patterns would be clearer; 70x90 vs. 7x9). Another factor is the random initial weight assignments. Also, alpha could be set differently.