Share this:
Google+

< Previous | Next | Contents >

## Why Use Neural Network?

Imagine you have a magic box. You can teach this magic box with anything. This magic box can learn and can be trained to do a specific task such as to recognize a pattern and do it very well. Once the magic box has been trained and is becoming smarter, it can help you to identify things, to predict things and find pattern that you did not even realize before. Do you like to have such magic box?

Having an artificial neural network is similar to have such magic box. In fact, in this tutorial you will learn how to build your own magic box and how to train that magic box for your own purpose. Exciting, isn’t it?

We often use neural network as a
magic box. The **explicit** mathematical
model is **hidden** under the Neural
Network model. For example, if we use linear regression as explicit model, we
know the model is linear and we can test statistically if the data fit into the
linear model or not. If our data does not fit the model, we will find other model
to fit the data. Using Neural Network we can get the same capability as linear
or non-linear regression but having neural network as diagram, the actual the
regression model is not shown explicitly. The statistical test (such as t-test,
or F test) are not usually performed explicitly. In fact, we can use one
generic neural network model with so many hidden layers to fit any data.

Three most important characteristics of neural networks are

- Adaptability
- Parallel processing
- Pattern Recognition
- Modularity
- Generalization

To understand the important of Artificial Neural Network, we need to compare it with the conventional software.

In traditional software,
programmers must write code of rules (which we call as *algorithm*) to instruct the computer what to do. An algorithm is
derived from mathematical models. To employ algorithms, your problems must be
well defined. Otherwise, the rules cannot be written. In contrast, Neural Network
learns "by example". Instead of giving the computer rules and codes, you only feed
a neural network with data.

Once a Neural Network program is build, no more programming is needed. When you have a new problem, such as new pattern, you only need to train again. Instead of doing programming, we train the neural network using data as our training examples.

Why we can use neural network
simply by training it instead of programming it? That is because the Neural Network **adapts** & formulates its own "algorithm"
that you even need to know what is going on inside the Neural Network (that is
why I call it a magic box). A neural network is so flexible that it has ability
to approximate *any* function. The
universal approximation theorem
[1]
states that a feed-forward network with a single hidden layer containing a
finite number of neurons such as a multilayer perceptron, can approximate any
continuous functions, regardless the choice of the activation function, as long
as it is arbitrary bounded and non-constant
[2]
. It means that even
a simple neural network is powerful enough to represent many interesting
functions.

Recall in high school mathematics
that a function is mapping of inputs to an output. A typical function can take
one or more independent variables (input variables) and only *one* dependent variable (output
variable). In contrast, neural network model can take many input variables and
produces *many *output variables. The
input and output values of a neural network can either be discrete (to
represent binary, nominal and ordinal scale) or continuous real values (to
represent interval and ratio scale variable type) or mixture between discrete
and continuous. This is clearly an advantage compared to many machine learning
algorithms that only take certain type of variable and produce only certain
type of variables.

Neural network is also called *connectionist model*. The basic unit of
neural network is a neuron which simply combine the inputs from many other neurons
and do simple activity to produce its output (such as squash the sum of the
inputs). Several neurons can form a layer of neurons, or a sub-network to be
combined with other layer or subnets. The capability to combine simple units
into a larger unit and work together is clearly one of the strength of neural
network modeling and design.

Second characteristic of neural
network is parallel processing. Traditional software processes its input one at
a time sequentially. In contrast, Neural Network does **parallel processing** of the whole input at once. It is true the
training a neural network can take some time. However, once the learning is
done, to use the Neural Network is very fast. Even if we use serial computer,
there big potential to distribute the computation on each layer to many
electronic chips and machines. This kind
fast computation potential would make us see more of neural networks
applications on chips sellable in the near future.

Thirdly, Neural Network also has
capability to perform non-linear tasks to seek patterns in the data that no one
knows are there. It can **recognize
pattern** and predict the pattern classification based on the data alone
without explicit rules from the programmer. This third characteristics make
Neural Network as one of the most indispensable tools for pattern recognition,
data mining, machine learning and artificial intelligence.

Fourth, one strength of neural
network is its **modularity**. A neural
network can be thought as a part (or called *subnet*)
of larger neural network. A subnet can be combined with other subnets in a
hierarchical manner to create a more useful and more functional neural network.
Similarly, a neural network may be separated into smaller units that can be
trained and tested in easier manner before we combine into larger unit. Plug
and Play Sub Neural Network that will be explained later in this tutorial will
show how it works with many examples.

Fifth, neural network has ability
to learn and **generalize** what it
learns. Through the examples of applications in this tutorial, you will see how
neural network can learn from a few examples and generalize it to many unseen
examples. This awesome strength of neural network to generalize happens only when
we use minimal neurons and do not do over training (the term is *overfitting*). When we train the neural
network too much, or use too much hidden neurons than necessary, neural network
will do **rote memorization** of the
training data and the ability to generalize to the new unseen data is degraded.

In summary, the strength of neural network is its characteristics. Neural network is one program that potentially can fit most problem without the need to program for each pattern. We only need to train the magic box to relearn new pattern. Fast processing due to parallel processing and one thing neural network do best (without programming it) is to recognize pattern.

< Previous | Next | Contents >

Read it off line on any device. Click here to purchase the complete E-book of this tutorial

**
See Also
**
:

K means clustering
,
Similarity Measurement
,
Reinforcement Learning (Q-Learning)
,
Discriminant Analysis
,
Kernel Regression
,
Clustering
,
Decision Tree

This tutorial is copyrighted .

**
Preferable reference for this tutorial is
**

Teknomo, Kardi (2019). Neural NetworkTutorial. https:\\people.revoledu.com\kardi\tutorial\NeuralNetwork\