What is a Neural Network?, Human Brain, Models of Neuron, NeuralNetworks viewed as directed graphs, Feedback, Network Architectures,Knowledge representation, Artificial Intelligence and Neural Networks.
Introduction, Error-correction learning, Memory-based learning, Hebbianlearning, Competitive learning,Boltzamann learning, Credit Assignmentproblem, Learning with a Teacher, Learning without a Teacher, Learningtasks, Memory, Adaptation.
Statistical nature of thelearning process, Statistical learning theory, Approximately correct model oflearning.Single Layer Perceptrons: Introduction, Adaptive filtering problem,Unconstrained optimization techniques, Linear least-squares filters, Least-mean square algorithm, Learning curves, Learning rate annealing techniques,Perceptron, Perceptron convergence theorem, Relation between thePerceptron and Bayes classifier for a Gaussian environment.
Introduction, Some preliminaries, Back-propagation Algorithm, Summary of back-propagation algorithm, XORproblem, Heuristics for making the back-propagation algorithm performbetter, Output representation and decision rule, Computer experiment, Featuredetection, Back-propagation and differentiation.
Hessian matrix, Generalization, approximationof functions, Cross validation, Network pruning techniques, virtues andlimitations of back- propagation learning, Accelerated convergence of backpropagation learning, Supervised learning viewed as an optimization problem,Convolution networks.
Introduction, Cover’s theorem on theseparability of patterns, Interpolation problem, Supervised learning as an ill-posed Hypersurface reconstruction problem, Regularization theory,Regularization networks, Generalized radial-basis function networks, XORproblem, Estimation of the regularization parameter.
Approximation properties of RBF networks, Comparison of RBF networks and multilayerPerceptrons, Kernel regression and it’s relation to RBF networks, Learningstrategies, Computer experiment.Optimization using Hopfield networks: Traveling salesperson problem,Solving simultaneous linear equations, Allocating documents tomultiprocessors.
Iterated gradient descent,Simulated Annealing, Random Search,Evolutionary computation- Evolutionary algorithms, Initialization,Termination criterion, Reproduction, Operators, Replacement, Schematheorem.