Functional Programming and Intelligent Algorithms

Tutorial 4.2: Linear and non-linear classifiers

Week 4: Testing Classifiers

2 Tutorial 4.2: Linear and non-linear classifiers

Reading: Stephen Marsland: Chapter 3.4-3.5 and the introduction for Chapter 4.

2.1 Problem 1: A Neural Network for the XOR problem

2.1.1 Step 1: A new module

1.
Review your solution to Problem 2 of Tutorial 3.2.
2.
Create a new module ANN, where you import your existing Perceptron module. import Perceptron
3.
Define a data type Network which consist of several layers. Add it to the ANN module.

2.1.2 Step 2: A network for XOR

pict (a) The problem

pict
(b) The solution


Figure 1: The XOR classification problem.

Recall the XOR classification problem in Figure 1a. We can use the solution (Figure 1b) to test our neural network and the recall function before we have to start thinking about training.

1.
Define a neural network xorNetwork with the weights given in Figure 1b. Use the Network type which we defined above.
2.
Show the network in GHCi, to check that you have not made a syntax error or similar. xorNetwork
3.
Now we need to implement a recall function. Add the type declaration to your ANN module. recallNetwork :: Network > [Double] > [Double]
4.
Implement recallNetwork, using recallLayer from Problem 2 of Tutorial 5.
5.
Discuss: Which is the output layer? Is it first or last element in the list which makes up the Network?
6.
Finally, test your definitions with the following evaluations: recallNetwork xorNetwork [0,0] 
recallNetwork xorNetwork [0,1] 
recallNetwork xorNetwork [1,0] 
recallNetwork xorNetwork [1,1]
7.
Discuss: Is the output as expected?
2.1.3 Step 3: Bug search

There are two common sources of errors in this network/implementation.

1.
What is the value of the threshold function at 0? I.e. does the neuron fire when the sum is exactly 0. In floating point problems, this hardly matters, but with the binary xor problem it does. If your test fails, try to change the threshold function.
2.
What is the sign of the quasi-input x0? Very often we use -1, but some authors use +1.


7th April 2017
Hans Georg Schaathun / hasc@ntnu.no