1. Introduction
2. History and Overview about Artificial Neural Network
3. Single neural network
7. References
2. History and Overview about Artificial Neural Network
3. Single neural network
- 3.1 Perceptron
- 3.1.1 The Unit Step function
- 3.1.2 The Perceptron rules
- 3.1.3 The bias term
- 3.1.4 Implement Perceptron in Python
- 3.2 Adaptive Linear Neurons
- 3.2.1 Gradient Descent rule (Delta rule)
- 3.2.2 Learning rate in Gradient Descent
- 3.2.3 Implement Adaline in Python to classify Iris data
- 3.2.4 Learning via types of Gradient Descent
- 3.3 Problems with Perceptron (AI Winter)
- 4.1 Overview about Multi-layer Neural Network
- 4.2 Forward Propagation
- 4.3 Cost function
- 4.4 Backpropagation
- 4.5 Implement simple Multi-layer Neural Network to solve the problem of Perceptron
- 4.6 Some optional techniques for Multi-layer Neural Network Optimization
- 4.7 Multi-layer Neural Network for binary/multi classification
- 5.1 Overview about MNIST data
- 5.2 Implement Multi-layer Neural Network
- 5.3 Debugging Neural Network with Gradient Descent Checking
- 5.3.1 Theory
- 5.3.2 Implement in Python
7. References
Implement Gradient Descent Checking
In this section, we'll extend another method of class MLPClassifier in the section 5.2.2 so-calledgradient_checking() and write some codes to compute our numerical gradient.
We've done about install gradient checking method, but keep in mind, gradient checking is computationally expensive, therefore, we just use a fewer of dataset and turn off regularization, adaptive learning, momentum term because all of them can cause the numerical gradient calculation is not exactly. You can see more on gradcheck tips page.### Training Multi-layer Neural Network with gradient descent checking
No comments :
Post a Comment
Leave a Comment...