Act 16: Neural Networks

In this activity, we also classify objects, except this time we make use of neural networks. A neural network is a computational model of how neurons in brains work. In comparison to linear discriminant analysis, there is no need to set rules to classify. Rather, neural networks learn the rules by examples which it then applies to the objects to be classified.

For this activity, we make use of the two classes in Activity 15. Cole's neural network code was used for this activity. After setting a seed so that each run will be consistent, we first make a training set:

x = [0.38 0.39;
0.35 0.38;
0.33 0.38;
0.31 0.36;
0.13 0.27;
0.12 0.27;
0.09 0.2;
0.09 0.2]

Note that the first column is the pixel area/1000 of the object and the second column is the red color channel value of the object. We set the values for classifying the objects as

[0 0 0 0 1 1 1 1]

This means that the first four items correspond to the class 0, which is the white squash seed while the other items correspond to the small round seed. We can now run the test set since the neural network has learned how to classify the objects. Our inputs for the test set are

x1 = [0.35 0.38;
0.31 0.36;
0.12 0.27;
0.11 0.25;
0.11 0.24;
0.1 0.23;
0.1 0.22;
0.09 0.22;
0.09 0.2;
0.09 0.2;
0.33 0.38;
0.1 0.24;
0.3 0.36;
0.28 0.35;
0.26 0.31;
0.24 0.3;
0.13 0.27;
0.38 0.39;
0.27 0.35;
0.27 0.34]

We set the learning rate to 1.0 and the training cycle to 1000. By manual classification, the output should be

[0 0 1 1 1 1 1 1 1 1 0 1 0 0 0 0 1 0 0 0]

The output of the neural network is:

[0.0081117 0.0258541 0.9832197 0.9897846 0.9904403 0.9936995 0.9940926 0.9957981 0.9962910 0.9962910 0.0134392 0.9932781 0.0346688 0.0675553 0.1581799
0.2973589 0.9757485 0.0039843 0.0923583 0.0977971 ]

Rounded off, this becomes:

[0 0 1 1 1 1 1 1 1 1 0 1 0 0 0 0 1 0 0 0]

We can see that the neural network has correctly classified the objects.

The value of the learning rate was then adjusted. I found out that decreasing the learning rate decreases the accuracy of the network. When set too low, the output values all become zero. Setting the training cycle too low will also have the same effect.

I will give myself a grade of 10 for this activity. Again, I would like to thank Earl for helping me collect the data in Activity 15, since it was used for this activity.