WINNOW .*;

Example 2: Character Recognition

In this example, the Winnow algorithm is used to classify and recognize multiple variations of 16 different letters (A through P).  The idea is to recognize a letter despite which font is used.

Figure 1.  A Winnow network with multiple classes.  The code below implements 16 classes.

 

"Fonts" look like the illustration in Figure 1 below.  They're simple text files, 35 rows of 70 characters (either "1" or "0").

00000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000011111111000000000000000000000000000000
00000000000000000000000000000111111111100000000000000000000000000000
00000000000000000000000000001111111111110000000000000000000000000000
00000000000000000000000000011111111111111000000000000000000000000000
00000000000000000000000000111111101111111000000000000000000000000000
00000000000000000000000000111111100111111100000000000000000000000000
00000000000000000000000001111111000011111110000000000000000000000000
00000000000000000000000011111110000001111111000000000000000000000000
00000000000000000000000111111100000000111111100000000000000000000000
00000000000000000000001111111100000000111111110000000000000000000000
00000000000000000000001111111000000000011111110000000000000000000000
00000000000000000000011111110000000000001111111000000000000000000000
00000000000000000000111111100000000000000111111100000000000000000000
00000000000000000001111111000000000000000111111100000000000000000000
00000000000000000001111111000000000000000011111110000000000000000000
00000000000000000011111110000000000000000001111111000000000000000000
00000000000000000111111110000000000000000001111111100000000000000000
00000000000000001111111100000000000000000000111111110000000000000000
00000000000000011111111111111111111111111111111111111000000000000000
00000000000000111111111111111111111111111111111111111100000000000000
00000000000001111111111111111111111111111111111111111110000000000000
00000000000001111111100000000000000000000000000111111110000000000000
00000000000011111110000000000000000000000000000011111111000000000000
00000000000011111110000000000000000000000000000011111111000000000000
00000000000111111110000000000000000000000000000001111111100000000000
00000000001111111100000000000000000000000000000000111111110000000000
00000000011111111000000000000000000000000000000000011111111000000000
00000000111111110000000000000000000000000000000000001111111100000000
00000001111111110000000000000000000000000000000000001111111110000000
00000011111111100000000000000000000000000000000000000111111111000000
00000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000

 

Figure 1.  35 x 70 ASCII rendering of an "A" in the Arial font.  This is the content of "A-Arial.txt", one of the many files in the samples collection.

 

Figure 2.  The feature collecting window used in this algorithm resembles that used in the spam filtering examples.  It's moved from left to right, line by line.  However, this window encompasses multiple lines, rather than a single line.

 

Modifications can be made to:

  • FEATURE_SIZE - columns x rows of the features window.
  • THRESHOLD - thickness threshold for widening weight gaps.
  • PROMOTION_FACTOR - positive training.
  • DEMOTION_FACTOR - negative training.
  • TRAINING_CYCLES - number of times each sample is presented for training.

 

The code below inputs text files (containing "font" data similar to that in Figure 1) with which to build a feature database.

Click here to download the sample files.

Example Results

FEATURE_SIZE = 8, THRESHOLD = 0.05, PROMOTION_FACTOR = 1.23, DEMOTION_FACTOR = 0.83, TRAINING_CYCLES = 1, and all samples were presented in alphabetical order.

Each successive sampling of a letter should yield fewer errors (you can see this simply by "eye-balling" the data).

In the beginning, there's lots of errors, but toward the end there are far fewer.

Anywhere you see "Actual = ..., but Expected = ..." indicates an error during the training process.

The results are lengthy, so I'll only show those for letters "A" and "I":

processing: A-Arial.txt
No recognized features in A-Arial.txt

processing: A-ComicSans.txt
Actual = A, but Expected = B
Actual = A, but Expected = C
Actual = A, but Expected = D
Actual = A, but Expected = E
Actual = A, but Expected = F
Actual = A, but Expected = G
Actual = A, but Expected = H
Actual = A, but Expected = I
Actual = A, but Expected = J
Actual = A, but Expected = K
Actual = A, but Expected = L
Actual = A, but Expected = M
Actual = A, but Expected = N
Actual = A, but Expected = O
Actual = A, but Expected = P

processing: A-Corsiva.txt
Correctly recognized A

processing: A-Courier.txt
Correctly recognized A

processing: A-Digital.txt
Correctly recognized A

processing: A-Impact.txt
Correctly recognized A

processing: A-Tahoma.txt
Correctly recognized A

processing: A-Times.txt
Correctly recognized A

--- results abbreviated ---

processing: I-Arial.txt
Actual = I, but Expected = A
Actual = I, but Expected = B
Actual = I, but Expected = C
Actual = I, but Expected = D
Actual = I, but Expected = E
Actual = I, but Expected = F
Actual = I, but Expected = G
Actual = I, but Expected = H
Actual = I, but Expected = J
Actual = I, but Expected = K
Actual = I, but Expected = L
Actual = I, but Expected = M
Actual = I, but Expected = N
Actual = I, but Expected = O
Actual = I, but Expected = P

processing: I-ComicSans.txt
Actual = I, but Expected = B
Actual = I, but Expected = C
Actual = I, but Expected = D
Actual = I, but Expected = E
Actual = I, but Expected = F
Actual = I, but Expected = G
Actual = I, but Expected = H

processing: I-Corsiva.txt
Actual = I, but Expected = A
Actual = I, but Expected = B
Actual = I, but Expected = C
Actual = I, but Expected = D
Actual = I, but Expected = E
Actual = I, but Expected = F
Actual = I, but Expected = G
Actual = I, but Expected = H

processing: I-Courier.txt
Actual = I, but Expected = H

processing: I-Digital.txt
Correctly recognized I

processing: I-Impact.txt
Correctly recognized I

processing: I-Tahoma.txt
Actual = I, but Expected = E
Actual = I, but Expected = F
Actual = I, but Expected = G
Actual = I, but Expected = H

processing: I-Times.txt
Correctly recognized I

 

The algorithm starts off with the A's, so the first sample isn't even recognized.

The fonts for letter A were learned easily, but those for letter I really varied a lot.

 

Example Results 2

The code examples with extra error tracking functions can indicate just how well the training works. The results appear the same except for the list of numbers toward the end.

FEATURE_SIZE = 8, THRESHOLD = 0.05, PROMOTION_FACTOR = 1.23, DEMOTION_FACTOR = 0.83, TRAINING_CYCLES = 5.

This time, the samples were presented in random order.

Again, the results are lengthy so I've only shown the error statistics below:

Average Error frequency for every 16 samples...
At the beginning:
0.96875
0.96875
0.96875
0.9375
0.9375
0.9375
0.90625
0.90625
0.90625
0.90625
0.90625
0.90625
0.90625
0.90625
0.90625
0.90625
0.90625
0.90625
0.90625
0.9375
0.9375
0.9375
0.96875
0.96875
0.9375
0.9375
0.9375
0.8958333333333334
0.9270833333333334
0.9270833333333334
0.9270833333333334
0.9270833333333334
0.9270833333333334
0.8958333333333334
0.8958333333333333
0.8958333333333333
0.8645833333333333
0.8645833333333333
0.8645833333333334
0.8645833333333334
0.8958333333333334
0.8958333333333333
0.8958333333333333
0.9375
0.9375
0.9375
0.9375
0.9375
0.90625
0.9375
0.9375
0.90625
0.9375
0.9375
0.9375
0.9375
0.9375
0.9375
0.90625
0.90625
0.90625
0.90625
0.90625
0.90625
0.9375
0.9375
0.9375
0.96875
0.96875
0.96875
0.96875
0.96875
0.96875
0.96875
1.0
1.0
1.0
1.0
1.0
0.96875
0.9375
0.9375
0.9375
0.9375
0.9375
0.9375
0.9375
0.9375
0.9375
0.9375
0.9375
0.9375
0.9375
0.9375
0.90625
0.9375
0.96875
0.96875
0.9375
0.9375
0.9375
0.9375
0.9375
0.9375
0.9375
0.9375
0.9375
0.9375
0.890625
0.859375
0.859375
0.8072916666666666
0.7604166666666666
0.7604166666666666
0.7604166666666666
0.7135416666666666
0.6822916666666666
0.6822916666666667
0.6822916666666667
0.6822916666666667
0.6510416666666667
0.6510416666666666
0.6510416666666666
0.6197916666666667
0.6666666666666667
0.6666666666666666
0.6979166666666666
0.75
0.796875
0.796875
0.828125
0.8333333333333334
0.8645833333333334
0.8645833333333334
0.8333333333333334
0.8020833333333334
0.8333333333333334
0.8333333333333334
0.8333333333333333
0.8645833333333333
0.8229166666666666
0.8541666666666666
0.8541666666666666
0.8541666666666667
0.8072916666666667
0.7760416666666666
0.7760416666666666
0.7864583333333333
0.7552083333333333
0.7552083333333333
0.7447916666666666
0.7760416666666666
0.7760416666666667
0.7760416666666667
0.7447916666666666
0.7135416666666666
0.7552083333333333
0.7239583333333333
0.6739583333333332
0.6427083333333332
0.6375
0.66875
0.66875
0.653125
0.6427083333333333
0.6114583333333333
0.6531250000000001
0.653125
0.60625
0.60625
0.60625
0.5854166666666666
0.5854166666666666
0.5854166666666667
0.6354166666666667
0.6145833333333334
0.6666666666666666
0.625
0.5833333333333335
0.5746527777777779
0.5850694444444445
0.5694444444444444
0.5694444444444444
0.5277777777777778
0.5746527777777778
0.532986111111111
0.5106646825396824
0.5158730158730158
0.4575396825396825
0.44712301587301584
0.3935515873015873
0.4143849206349206
0.38313492063492066
0.39355158730158735
0.3831349206349207
0.39702380952380956
0.38660714285714287
0.3918154761904762
0.3362599206349206
0.3243551587301587
0.32435515873015874
0.36602182539682543
0.419593253968254
0.407874503968254
0.4193328373015873
0.42974950396825395
0.48332093253968256
0.47290426587301587
0.5041542658730158
0.48071676587301587
0.47655009920634916
0.4646453373015873
0.5063120039682539
0.4893849206349206
0.48938492063492056
0.4856646825396825
0.4544146825396825
0.39972718253968254
0.34764384920634916
0.37498759920634916
0.36504441738816734
0.3377006673881674
0.28214511183261176
0.2652180284992785
0.2152180284992785
0.21633409992784994
0.2163340999278499
0.2230305284992785
0.16324791980362635
0.16827024123219778
0.22382579678775333
0.2238257967877533
0.2082007967877533
After final sample.

 

If these are plotted on a chart, we'd see this:

 

The successive training cycles brought the error frequency down considerably.

 

public void footer() {
About | Contact | Privacy Policy | Terms of Service | Site Map
Copyright© 2009-2012 John McCullock. All Rights Reserved.
}