# Neural Networks: Chapter 2 - Fundamentals

In our **previous article**, we saw how we can relate **Artificial Neural Networks **with the Nervous system of our body and how Human Intelligence can be mapped to the computation of signals. Today we will look into some basic concepts of Neural Networks.

## Single Layer Perceptron

The simplest neural network is known as **Perceptron**. It consists of 1 Input layer and 1 output layer. A simple instantiation of Neural Network which has a single-layer network with a set of inputs directly mapped to output by using a generalized variation of a linear function. Such a neural network is also referred to as the **Perceptron**.

The **architecture of the perceptron** is shown above, in which a single input layer transmits the features to the output node. The edges from the input to the output contain the **weights** w1 . . . w5 with which the features are multiplied and added at the output node. Subsequently, the **step** function is applied to convert the aggregated value into a class label. The sign function serves the role of an **activation function**.

To solve a more complex problem, the number of neurons needed for a single-layer perceptron increasesexponentially. To solve this we introducedepthin the network, whichincreases the number of hidden layersanddecreases computational loadand thedensity of neuronsper layer.

## Multi-Layer Perceptron

When the network has to solve a more complex equation, the single-layer perceptron does not hold well. As the complexity increase, we need to increase the number of layers in our neural network. This introduces depth in the network and proposed the term Deep Learning.

In multi-layer neural networks, the neurons are arranged in a layered fashion, in which the input and output layers are separated by a group of **hidden layers**. This layer-wise architecture of the neural network is also referred to as a **feed-forward network**.

The network is feed with the input and all the computations are done on subsequent layers known as **hidden layers**. These layers apply an **activation function** and the results are then aggregated at a final layer, also called the **output layer**.

## Training a Neural Network

Now that we have seen the basic architecture of a Neural Network, let's talk about how to train it. There are various frameworks available in the market like **Tensorflow**, **Pytorch**, etc. which makes it simple to define and train a neural network. But it is important to understand how the training process works and what is the mathematics behind it.

In ** Supervised learning** problems, we have the target variable with us, and all we need to do is update the weights and biases of the network in such a way that the input passed to the network gives us the desired output. In other words, we need to minimize the error between the expected output and the observed output. This difference can be mathematically depicted as a Loss Function.

The goal of the training process is to minimize theLoss function.

Two major parameters drive the training process, i.e. **weights and biases. **Once the network can understand the right combination of **weights and biases** for every layer, it is considered to be **trained**.

The choice ofActivation functions,Loss functions, andOptimizersneeds to be made from a list of options available. Theconcept doesn't work here. The choice of the parameters depends on the problem statement."One Size Fits All"

### Forward Pass

In this pass, the input for training is fed into the neural network. This results in a forward cascade of **computations across the layers**, using the current set of **weights**. The final predicted output can be compared to that of the training instance and the derivative of the **loss function** concerning the output is computed. The derivative of this loss now needs to be computed concerning the weights in all layers in the B**ackward pass.**

### Backward Pass

The main goal of the **Backward Pass** is to learn the **gradient of the loss function concerning the different weights **by using the chain rule of differential calculus. These gradients are used to **update the weights**. Since these gradients are learned in the backward direction, starting from the output node, this learning process is referred to as the **Backward Pass or ****Backpropagation****.**

## Conclusion

We have seen all the fundamental units required in building and training a **Neural Network.**

This was a high-level explanation of basic concepts. In future posts, we will deep dive into all the important units that are required in Neural Networks. Feel free to read about these concepts and **STAY TUNED** for articles covering **advanced concepts of Neural Networks.**