Overview

In this mini project, you will need to apply the feedforward neural network (FNN) on the MNIST dataset for hand written digit recognition.

Dataset

The MNIST database of handwritten digits has a training set of 60,000 digit images, and a test set of 10,000 images. It is a subset of a larger set available from NIST. The digits have been size-normalized and centered in a fixed-size image.

Method

A starter script “main_ha4.py” is provided for downloading MNIST dataset, splitting datasets, and feature normalization. You don’t need to changes these coding lines. You might need to install the following packages:

• tensorflow
• keras
• numpy
• scipy
• matplotlib
• sklearn

For every image, we will use its pixel-wise intensities as features.Each gray-scale image of 28 by 28 pixels will be converted into a vector of 784.In other words, each image is considered to have 784 features. This will define the first layer of the FNN. To define a FNN model, you will also need to specify the following hyper-parameters:

• number of hidden layers and number of neural units for each layer;
• learning rate
• activation function (sigm, tanh_opt) and
• output function (‘sigm’,’linear’, ‘softmax’);

There is no need for you to code the neural network algorithm from scratch. You can use either third-party libraries or the Tensorflow implementations provided along in the lecture:

https://colab.research.google.com/drive/1VvpND8QXd…

p=sharing

Question 1. Please further split the 60,000 training images (and labels) into two subsets: 50,000 images, and 10,000 images. Use these two subsets for training models and validation purposes.In particular, you will train your FNN model using the 50,000 images and labels, and apply the trained model over the rest 10,000 images for evaluation purposes.

Please specify at least three sets of hyper-parameters (see the above). For each set, call the third-party functions or tensorflow to train a FNN model on the training samples (50,000 images in this case), and apply the learned model over the validation set (10,000 images in this case). For each model and its results, please compute its confusion matrix, average accuracy, per-class Precision and Recall. Report the model that achieves the top accuracy.

A sample function for calculating confusion matrix is provided in ‘util.py’

Question 2. Apply the top ranked model over the testing samples (10,000 images). Call the above function to compute the confusion matrix, average accuracy, per-class precision/recall rate.In addition, select and visualize TEN testing images for which your mode made wrong predications. Try to analyze the reasons of these failure cases.

Submission instructions: what to hand in

• Prepare a Single PDF file to describe your decision process while completing the above questions. Include all figures, tables, intermediate results, or other results that might help understand your efforts. No need to include full paragraphs of your source codes in the report. Coding snippet is allowed.
• Submit your PDF file and source codes through Canvas.
• No HARD COPY IS REQURIED.