Backpropagation .*;

Example Code 1

based on article and code by Christopher M. Frenz

"Give Your .NET App Brains and Brawn with the Intelligence of Neural Networks"

MSDN Magazine, May 2005

http://msdn.microsoft.com/msdnmag/issues/05/05/NeuralNetworks/default.aspx

 

This example was presented by Frenz as a simple introduction to feed-forward nets with some backpropagation. The goal of this network is to predict a patients risk for heart disease:

"The premise is that you have been hired by a group of doctors who are trying to predict their patients' risk for developing heart disease. Over the years they have monitored changes in potential risk factors of past patients, such as blood pressure, weight, and so on, and recorded whether these patients developed heart disease. The neural network under development will be trained with this information so that the doctors can predict the heart disease risk for their patients and take appropriate preventative action."

-- Christopher M. Frenz

To make predictions for this problem, the net uses 3 input neurons, 3 hidden neurons, and 1 output neuron.

One of the interesting things to note in Frenz's design, is that the input is not binary. His design scales integral input values for use in the network.

The INPUTS array includes the values for "Change in Cholesterol", "Change in Weight", and "Family History". You can tinker with these values for experimentation.

 

Example Results 1

For input:

Change in Cholesterol = 5

Change in Weight = -7

Family History = 1

Results:

1000  0  Expected: -1.0  Actual: -0.975
1000  1  Expected: -1.0  Actual: -0.977
1000  2  Expected: 1.0  Actual: 0.969
1000  3  Expected: 1.0  Actual: 0.969
1000  4  Expected: -1.0  Actual: -0.983
1000  5  Expected: 1.0  Actual: 0.969
1000  6  Expected: -1.0  Actual: -0.974
1000  7  Expected: -1.0  Actual: -0.983
Training Error: 0.026

2000  0  Expected: -1.0  Actual: -0.982
2000  1  Expected: -1.0  Actual: -0.983
2000  2  Expected: 1.0  Actual: 0.978
2000  3  Expected: 1.0  Actual: 0.978
2000  4  Expected: -1.0  Actual: -0.988
2000  5  Expected: 1.0  Actual: 0.978
2000  6  Expected: -1.0  Actual: -0.983
2000  7  Expected: -1.0  Actual: -0.988
Training Error: 0.018

3000  0  Expected: -1.0  Actual: -0.985
3000  1  Expected: -1.0  Actual: -0.986
3000  2  Expected: 1.0  Actual: 0.982
3000  3  Expected: 1.0  Actual: 0.982
3000  4  Expected: -1.0  Actual: -0.990
3000  5  Expected: 1.0  Actual: 0.982
3000  6  Expected: -1.0  Actual: -0.987
3000  7  Expected: -1.0  Actual: -0.990
Training Error: 0.015

4000  0  Expected: -1.0  Actual: -0.987
4000  1  Expected: -1.0  Actual: -0.988
4000  2  Expected: 1.0  Actual: 0.985
4000  3  Expected: 1.0  Actual: 0.985
4000  4  Expected: -1.0  Actual: -0.991
4000  5  Expected: 1.0  Actual: 0.985
4000  6  Expected: -1.0  Actual: -0.989
4000  7  Expected: -1.0  Actual: -0.991
Training Error: 0.013

5000  0  Expected: -1.0  Actual: -0.989
5000  1  Expected: -1.0  Actual: -0.989
5000  2  Expected: 1.0  Actual: 0.986
5000  3  Expected: 1.0  Actual: 0.986
5000  4  Expected: -1.0  Actual: -0.992
5000  5  Expected: 1.0  Actual: 0.986
5000  6  Expected: -1.0  Actual: -0.990
5000  7  Expected: -1.0  Actual: -0.992
Training Error: 0.011

6000  0  Expected: -1.0  Actual: -0.989
6000  1  Expected: -1.0  Actual: -0.990
6000  2  Expected: 1.0  Actual: 0.988
6000  3  Expected: 1.0  Actual: 0.988
6000  4  Expected: -1.0  Actual: -0.993
6000  5  Expected: 1.0  Actual: 0.988
6000  6  Expected: -1.0  Actual: -0.991
6000  7  Expected: -1.0  Actual: -0.993
Training Error: 0.010

7000  0  Expected: -1.0  Actual: -0.990
7000  1  Expected: -1.0  Actual: -0.991
7000  2  Expected: 1.0  Actual: 0.988
7000  3  Expected: 1.0  Actual: 0.988
7000  4  Expected: -1.0  Actual: -0.993
7000  5  Expected: 1.0  Actual: 0.988
7000  6  Expected: -1.0  Actual: -0.991
7000  7  Expected: -1.0  Actual: -0.993
Training Error: 0.010

8000  0  Expected: -1.0  Actual: -0.991
8000  1  Expected: -1.0  Actual: -0.992
8000  2  Expected: 1.0  Actual: 0.989
8000  3  Expected: 1.0  Actual: 0.989
8000  4  Expected: -1.0  Actual: -0.994
8000  5  Expected: 1.0  Actual: 0.989
8000  6  Expected: -1.0  Actual: -0.992
8000  7  Expected: -1.0  Actual: -0.994
Training Error: 0.009

9000  0  Expected: -1.0  Actual: -0.991
9000  1  Expected: -1.0  Actual: -0.992
9000  2  Expected: 1.0  Actual: 0.990
9000  3  Expected: 1.0  Actual: 0.990
9000  4  Expected: -1.0  Actual: -0.994
9000  5  Expected: 1.0  Actual: 0.990
9000  6  Expected: -1.0  Actual: -0.993
9000  7  Expected: -1.0  Actual: -0.994
Training Error: 0.008

Training iterations needed: 6435

The patient is at an increased risk for disease

Validating network training...
Expected: -1.0 Actual: -0.989
Expected: 1.0 Actual: 0.990
Expected: 1.0 Actual: 0.987
3 out of 3 Match.  Validation Successful.

Example Results 2

For input:

Change in Cholesterol = -2

Change in Weight = 1

Family History = 1

Results:

1000  0  Expected: -1.0  Actual: -0.975
1000  1  Expected: -1.0  Actual: -0.976
1000  2  Expected: 1.0  Actual: 0.968
1000  3  Expected: 1.0  Actual: 0.968
1000  4  Expected: -1.0  Actual: -0.983
1000  5  Expected: 1.0  Actual: 0.968
1000  6  Expected: -1.0  Actual: -0.952
1000  7  Expected: -1.0  Actual: -0.983
Training Error: 0.030

2000  0  Expected: -1.0  Actual: -0.983
2000  1  Expected: -1.0  Actual: -0.984
2000  2  Expected: 1.0  Actual: 0.977
2000  3  Expected: 1.0  Actual: 0.978
2000  4  Expected: -1.0  Actual: -0.988
2000  5  Expected: 1.0  Actual: 0.978
2000  6  Expected: -1.0  Actual: -0.967
2000  7  Expected: -1.0  Actual: -0.988
Training Error: 0.021

3000  0  Expected: -1.0  Actual: -0.987
3000  1  Expected: -1.0  Actual: -0.987
3000  2  Expected: 1.0  Actual: 0.982
3000  3  Expected: 1.0  Actual: 0.982
3000  4  Expected: -1.0  Actual: -0.991
3000  5  Expected: 1.0  Actual: 0.982
3000  6  Expected: -1.0  Actual: -0.975
3000  7  Expected: -1.0  Actual: -0.991
Training Error: 0.016

4000  0  Expected: -1.0  Actual: -0.989
4000  1  Expected: -1.0  Actual: -0.989
4000  2  Expected: 1.0  Actual: 0.984
4000  3  Expected: 1.0  Actual: 0.984
4000  4  Expected: -1.0  Actual: -0.992
4000  5  Expected: 1.0  Actual: 0.984
4000  6  Expected: -1.0  Actual: -0.981
4000  7  Expected: -1.0  Actual: -0.992
Training Error: 0.014

5000  0  Expected: -1.0  Actual: -0.990
5000  1  Expected: -1.0  Actual: -0.990
5000  2  Expected: 1.0  Actual: 0.986
5000  3  Expected: 1.0  Actual: 0.986
5000  4  Expected: -1.0  Actual: -0.993
5000  5  Expected: 1.0  Actual: 0.986
5000  6  Expected: -1.0  Actual: -0.985
5000  7  Expected: -1.0  Actual: -0.993
Training Error: 0.012

6000  0  Expected: -1.0  Actual: -0.991
6000  1  Expected: -1.0  Actual: -0.991
6000  2  Expected: 1.0  Actual: 0.987
6000  3  Expected: 1.0  Actual: 0.987
6000  4  Expected: -1.0  Actual: -0.993
6000  5  Expected: 1.0  Actual: 0.987
6000  6  Expected: -1.0  Actual: -0.988
6000  7  Expected: -1.0  Actual: -0.993
Training Error: 0.011

7000  0  Expected: -1.0  Actual: -0.992
7000  1  Expected: -1.0  Actual: -0.992
7000  2  Expected: 1.0  Actual: 0.988
7000  3  Expected: 1.0  Actual: 0.988
7000  4  Expected: -1.0  Actual: -0.994
7000  5  Expected: 1.0  Actual: 0.988
7000  6  Expected: -1.0  Actual: -0.989
7000  7  Expected: -1.0  Actual: -0.994
Training Error: 0.010

8000  0  Expected: -1.0  Actual: -0.992
8000  1  Expected: -1.0  Actual: -0.992
8000  2  Expected: 1.0  Actual: 0.989
8000  3  Expected: 1.0  Actual: 0.989
8000  4  Expected: -1.0  Actual: -0.994
8000  5  Expected: 1.0  Actual: 0.989
8000  6  Expected: -1.0  Actual: -0.990
8000  7  Expected: -1.0  Actual: -0.994
Training Error: 0.009

9000  0  Expected: -1.0  Actual: -0.993
9000  1  Expected: -1.0  Actual: -0.993
9000  2  Expected: 1.0  Actual: 0.990
9000  3  Expected: 1.0  Actual: 0.990
9000  4  Expected: -1.0  Actual: -0.995
9000  5  Expected: 1.0  Actual: 0.990
9000  6  Expected: -1.0  Actual: -0.991
9000  7  Expected: -1.0  Actual: -0.995
Training Error: 0.008

Training iterations needed: 6586

The patient is at a reduced risk for disease

Validating network training...
Expected: -1.0 Actual: -0.990
Expected: 1.0 Actual: 0.990
Expected: 1.0 Actual: 0.987
3 out of 3 Match.  Validation Successful.
public void footer() {
About | Contact | Privacy Policy | Terms of Service | Site Map
Copyright© 2009-2012 John McCullock. All Rights Reserved.
}