Neural Networks: Understanding Using Visual Basic
Warning: Hazardous Grammar ahead. This tutorial was written years ago when I was just beginning to learn good English writing.
Neural Networks as a subject was the most difficult one to learn when
I started taking interest in AI. Although, Internet was full of NN tutorials
but they all seemed cryptic and too much mathematical. Anything which
was available was hard to digest for a beginner in this field.
I swept through tons of code and tutorials just to understand what the hell, much hyped, NNs were. Then, one fine day, Eureka!
I finally understood them and their practical applications. I immediately decided to write code in Visual Basic(my fav lang). After an hour of writing code and few more for tweaking it, I finally produced something which I had only dreamt of. It was one of the the best moments of my life. Then I finally uploaded .........history is very boring. The real thing: There isn't a single NN tutorial available for Visual Basic Programmers. That is why I decided to write this tutorial.
Introduction to Neural Networks
Neural Network or more appropriately Artificial Neural Network is basically
a mathematical model of what goes in our mind(or brain). The brain of
all the advanced living creatures consists of neurons, a basic cell, which
when interconnected produces what we call Neural Network. The sole purpose
of a Neuron is to receive electrical signals, accumulate them and see
further if they are strong enough to pass forward.
So simple in its basic functionality but the interconnections of these produces beings(me, u and others) capable of writing about them. phew! The real thing lies not in neurons but the complex pattern in which they are interconnected. NNs are just like a game of chess, easy to learn but hard to master. As the moves of chess are simple, yet the succession of moves is what makes the game complex and fun to play. Imagine, a chess game in which you are allowed only one single move. Would that game be fun to play?
In the same way, a single neuron is useless. Well, practically useless. It is the complex connection between them and values attached with them(explained later) which makes brains capable of thinking and having a sense of consciousness(much debated).
As explained earlier, a neuron is basically a cell which accumulates electrical signals with different strengths. What it does more is that it compares the accumulated signal with one predefined value unique to every neuron. This value is called bias. Well, now I think is time to explain with an image. So here is it:
Image Source: http://www.interwet.psu.edu/f41.gif
The circles in the image represents neurons. This network or more appropriately
this network topology is called feed-forward multi layered neural network. It
is the most basic and most widely used network. I will be explaining this network
The network is called multi layered because it consists of more than two layers. The neurons are arranged in a number of layers, generally three. They are input, hidden/middle and output layers. The names signify the function of the layer.
This network is feed-forward, means the values are propagated in one direction only. There are many other topologies in which values can be looped or move in both forward and backward direction. But, this network allows the movement of values only from input layer to output layer. The functions of various layers are explained below:
- Input layer: As it says, this layer takes the inputs(the values you pass) and forwards it to hidden layer. You can just imagine input layer as a group of neurons whose sole task is to pass the numeric inputs to the next level. Well it depends on the implementation but, in my implementation the numeric input has to be in range 0 and 1 (both inclusive). The larger the number greater its strength. E.g. 0.51 is stronger than 0.39 but 0.93412 is stronger still. But, the interpretation of this strength depends upon the implementation and the problem NN is required to solve. E.g. for an OCR you connect every pixel with its respective input neuron and darker the pixel, higher the signal/input strength. In short shades of gray would correspond to input strength. But at last again, remember: Input layer never processes data, it just hands over it.
- Middle layer: This layer is the real thing behind the network. Without
this layer, network would not be capable of solving complex problems. There
can be any number or middle or hidden layers. But, for most of the tasks,
one is sufficient. The number of neurons in this layer is crucial. There is
no formula for calculating the number, just hit and trial works.
This layer takes the input from input layer, does some calculations and forwards to the next layer, in most cases it is the output layer.
- Output layer: This layer consists of neurons which output the result to you. This layer takes the value from the previous layer, does calculations and gives the final result. Basically, this layer is just like hidden layer but instead of passing values to the next layer, the values are treated as output.
- Dendrites: No! it is not some creature from X-Files, but a name given to straight lines joining two neurons of consecutive layers, which you can see in the image. They are just a passage(or method) through which values are passed from one layer to the next. There is a value attached with dendrite called weight. The weight associated with a dendrites basically determines the importance of incoming value. A weight with larger value determines that the value from that particular neuron is of higher significance. To achieve this what we do is multiply the incoming value with weight. So no matter how high the value is, if the weight is low the multiplication yields the final low value. I know it is hard to explain and understand, but take a look at code and you'll understand this concept more clearly.
So now lets get to the code and define the basic elements we'll need in NNs
The code above is pretty self explanatory. And for the things you haven't understood, don't worry that belongs to belong to the training section, which I would be explaining a little later on. The only thing that you need to understand now is bias. Bias is just another value or parameter associated with a neuron. In my and most of the implementations this value is added to the accumulated incoming value. You will better understand this when we see code for running the NN. For now see the code for creating the net:
Basically, what we are doing above is:
- Creating an array of Neurons and dendrites.
- Looping through all the neurons and assigning their respective baises a random value.
- Looping through all the dendrites and assigning their respective weights a random value.
What we have done above may be summed up as:
- Assigning all the input neurons the values provided in the 'ArrayOfInputs' array. For every other type of neuron, the value assigned is 0 because we have to calculate that value.
- Now for every hidden layer (also for output layer), the value is sum of values of previous layers multiplied with the weight of dendrites.
- To the resulting value, add up the bias value.
- Pass the resulting value through Activation function.
- Give back the values of output neurons as final results.
I will suppose you wouldn't like to get into the mathematical details(hurray!). In case you like, you can easily google it and find out.
Training is the most important part of a neural network and the one consisting
of the most mathematics. We'll be using Backpropagation method for training
the NN. I would be explaining only the basic idea how it is done and not all
the details. And you need not worry, just apply the formulae written in the
code and you can get the training up and ready. Always remember, it is not necessary
to know the details of any system in order to get something practical and working
out of it. The best example illustrating this principle is Charles Darwin(what?).
Yes, at the time when he wrote 'On the Origin of Species', DNA was not
known. So, he propounded the evolution without even knowing the method of how
it is done i.e. how traits are passed on from parents to offspring.
Back to NN training, first lets see the code:
So lets see what we have here in arguments, inputdata a sample input array and outputdata, its corresponding output which we require the NN to produce.
- Run the NN using inputdata as inputs. Now, we have all the values of individual neurons.
- Loop through last layer to first layer.
- Calculate the difference between output values desired and output values produced.
- Using that difference, adjust the values of bias and weights accordingly.
Yes, I know I haven't explained the training part in details(well practically zero). You see, I had written this code 1-2 years back and was not actively involved with NNs. During that period I have forgotten all the mathematics behind Backpropagation. But anyway, If you want to delve into thedetails, Google is to your rescue.
There are numerous applications of NNs limited only by your imagination. Innovation
is key to success, so dude use NNs to create something which will revolutionize
For the sake of writing :) a few applications are given below:
- Optical Character Recognization
- Stock market prediction
- Creating new art forms
- Modeling human behavior
- Loan risk analysis
- Classification of patterns, Image analysis, music generation, etc, etc.
There is more theory in this world about NNs than you can read in your lifetime.
There are Hopfield networks, Recurrent networks, NeuroGenetics, etc. They are
for people who really want to do serious research in this field. But in case,
you want to read further, you can. Internet is a great free information highway.
You can get anything you want on the internet if you know how to search effectively.
Even if you don't, train a NN to fetch NN tutorials for you :-)
Thanks for reading this basic tutorial, Hope you enjoyed the ride.
Click here to download the accompanying code.