Neural Network Tutorial: Strength

Share this: Google+
< Previous | Next | Contents >

Why Use Neural Network?

Imagine you have a magic box. You can teach this magic box with anything. This magic box can learn and can be trained to do a specific task such as to recognize a pattern and do it very well. Once the magic box has been trained and is becoming smarter, it can help you to identify things, to predict things and find pattern that you did not even realize before. Do you like to have such magic box?

Having an artificial neural network is similar to have such magic box. In fact, in this tutorial you will learn how to build your own magic box and how to train that magic box for your own purpose. Exciting, isn’t it?

We often use neural network as a magic box. The explicit mathematical model is hidden under the Neural Network model. For example, if we use linear regression as explicit model, we know the model is linear and we can test statistically if the data fit into the linear model or not. If our data does not fit the model, we will find other model to fit the data. Using Neural Network we can get the same capability as linear or non-linear regression but having neural network as diagram, the actual the regression model is not shown explicitly. The statistical test (such as t-test, or F test) are not usually performed explicitly. In fact, we can use one generic neural network model with so many hidden layers to fit any data.

Three most important characteristics of neural networks are

Adaptability
Parallel processing
Pattern Recognition
Modularity
Generalization

To understand the important of Artificial Neural Network, we need to compare it with the conventional software.

In traditional software, programmers must write code of rules (which we call as algorithm) to instruct the computer what to do. An algorithm is derived from mathematical models. To employ algorithms, your problems must be well defined. Otherwise, the rules cannot be written. In contrast, Neural Network learns "by example". Instead of giving the computer rules and codes, you only feed a neural network with data.

Once a Neural Network program is build, no more programming is needed. When you have a new problem, such as new pattern, you only need to train again. Instead of doing programming, we train the neural network using data as our training examples.

Why we can use neural network simply by training it instead of programming it? That is because the Neural Network adapts & formulates its own "algorithm" that you even need to know what is going on inside the Neural Network (that is why I call it a magic box). A neural network is so flexible that it has ability to approximate any function. The universal approximation theorem [1] states that a feed-forward network with a single hidden layer containing a finite number of neurons such as a multilayer perceptron, can approximate any continuous functions, regardless the choice of the activation function, as long as it is arbitrary bounded and non-constant [2] . It means that even a simple neural network is powerful enough to represent many interesting functions.

Recall in high school mathematics that a function is mapping of inputs to an output. A typical function can take one or more independent variables (input variables) and only one dependent variable (output variable). In contrast, neural network model can take many input variables and produces many output variables. The input and output values of a neural network can either be discrete (to represent binary, nominal and ordinal scale) or continuous real values (to represent interval and ratio scale variable type) or mixture between discrete and continuous. This is clearly an advantage compared to many machine learning algorithms that only take certain type of variable and produce only certain type of variables.

Neural network is also called connectionist model. The basic unit of neural network is a neuron which simply combine the inputs from many other neurons and do simple activity to produce its output (such as squash the sum of the inputs). Several neurons can form a layer of neurons, or a sub-network to be combined with other layer or subnets. The capability to combine simple units into a larger unit and work together is clearly one of the strength of neural network modeling and design.

Second characteristic of neural network is parallel processing. Traditional software processes its input one at a time sequentially. In contrast, Neural Network does parallel processing of the whole input at once. It is true the training a neural network can take some time. However, once the learning is done, to use the Neural Network is very fast. Even if we use serial computer, there big potential to distribute the computation on each layer to many electronic chips and machines. This kind fast computation potential would make us see more of neural networks applications on chips sellable in the near future.

Thirdly, Neural Network also has capability to perform non-linear tasks to seek patterns in the data that no one knows are there. It can recognize pattern and predict the pattern classification based on the data alone without explicit rules from the programmer. This third characteristics make Neural Network as one of the most indispensable tools for pattern recognition, data mining, machine learning and artificial intelligence.

Fourth, one strength of neural network is its modularity. A neural network can be thought as a part (or called subnet) of larger neural network. A subnet can be combined with other subnets in a hierarchical manner to create a more useful and more functional neural network. Similarly, a neural network may be separated into smaller units that can be trained and tested in easier manner before we combine into larger unit. Plug and Play Sub Neural Network that will be explained later in this tutorial will show how it works with many examples.

Fifth, neural network has ability to learn and generalize what it learns. Through the examples of applications in this tutorial, you will see how neural network can learn from a few examples and generalize it to many unseen examples. This awesome strength of neural network to generalize happens only when we use minimal neurons and do not do over training (the term is overfitting). When we train the neural network too much, or use too much hidden neurons than necessary, neural network will do rote memorization of the training data and the ability to generalize to the new unseen data is degraded.

In summary, the strength of neural network is its characteristics. Neural network is one program that potentially can fit most problem without the need to program for each pattern. We only need to train the magic box to relearn new pattern. Fast processing due to parallel processing and one thing neural network do best (without programming it) is to recognize pattern.

< Previous | Next | Contents >

Read it off line on any device. Click here to purchase the complete E-book of this tutorial

This tutorial is copyrighted .

Preferable reference for this tutorial is

Teknomo, Kardi (2019). Neural NetworkTutorial. https:\\people.revoledu.com\kardi\tutorial\NeuralNetwork\