# First Step Towards Deep Learning: Neural Networks - By Hrithik Sharma

Updated: Apr 25, 2020

Hello there, I am back with another article as you know I am learning python for the development and deployment of Artificial Intelligence and Machine Learning. Till now I completed the module of python programming language, essential mathematics for Machine Learning & Machine Learning too. The next module of Deep Learning is started in which the study of Neurons, The study of layers of Neural Network, Weights, Activation Functions, and Backpropagation is carried out.

What is a neural network ? The idea behind the neural networks is to mimic or create the artificial neurons by which the machines will think like humans do. The human brain is considered the best tool for learning anything in this world. So, how the human brain adapts things and learn about things, these all tasks are done due to the presence of huge numbers of neurons in the human brain. Similarly, the same architecture is applied to the machines solving the interesting real-world problems based on binary class classification, multi-class classification or regression.

Take a look at the image below:

In the above image, there is Layer 1 with 3 neurons those are the carriers for the three inputs which are classified as Input 1, Input 2 & Input 3, the Layer 2 with two neurons called the Hidden layer, a Layer 3 with a single neuron which produces the output called Output layer and the lines which connect the inputs to neurons carry the weight which is a random number. The inputs are taken from the dataset which is used to train the neural network each input represents each independent variable of the dataset and some processing is happening in the neurons of the hidden layer and output layer. Now the question is what is happening in the name of processing in neurons of Hidden Layer and Output Layer.

Let's see what happens in each neuron. Take a look at image below:

Let's take a look at the single neuron of the hidden layer where {x1, x2, x3} are the inputs of the single row of the dataset and {w1, w2, w3} are the random numbers known as the weights. Firstly each weight is multiplied by each input then the sum is taken then on that sum activation is applied in the form of activation functions such as Threshold function, Sigmoid function, Rectified Linear Unit, Hyper tan function and SoftMax function which is depicted clearly in the image below:

This is the same neuron from the hidden layer which is shown in the above image. In this above neural network, the same processing is happening on the neurons of the output layer too but the activation functions may vary with respect to the problem. For example in the case of binary class classification, the Sigmoid activation is suitable, in multi-class classification the SoftMax activation is suitable and in case of regression, the Rectified Linear Unit is considered suitable. This all process is known as forward propagation.

## Let's Discuss Some Activation Functions

```
>>> import numpy as np
>>> import matplotlib.pyplot as plt
>>> %matplotlib inline
```

### 1. Sigmoid Function

```
>>> x = np.linspace(-10, 10, 100)
>>> y = 1/(1 + np.exp(-x))
```

```
>>> plt.figure(figsize = (8,4))
>>> plt.xlabel('Numbers')
>>> plt.ylabel('Probability')
>>> plt.title('Sigmoid Function')
```

```
>>> plt.plot(x,y, linewidth = 5, color = 'blue')
```

Sigmoid function is used in Logistic regression. Sigmoid function can minimize any number as a output in between range of 0 and 1. The curve of Sigmoid function is shown above.

### 2. Rectified Linear Unit

```
>>> def relu(x):
return max(0.0, x)
>>> X = [x for x in range(-10, 10)]
>>> Y = [relu(x) for x in X]
>>> plt.figure(figsize = (8,4))
>>> plt.xlabel('Numbers')
>>> plt.ylabel('Max')
>>> plt.title('RELU')
>>> plt.plot(X,Y, color = 'red', linewidth = 5)
```

Rectified Linear Unit returns the 0 if ant negative input is passed and maximum if the input is greater then zero.

### 3. Hyper Tan Function

```
>>> X = np.linspace(-10,10,100)
>>> Y = np.tanh(X)
>>> plt.figure(figsize = (8,4))
>>> plt.title('Hyper Tan Function')
>>> plt.xlabel('X')
>>> plt.ylabel('Y')
>>> plt.plot(X, Y, color = 'green', linewidth = 5)
```

Hyper tan function minimizes any number as an output in between range of -1 and 1.

### 4. SoftMax Function

```
>>> arr = arr = np.array([[0.1,0.35,0.25,0.3],[0.2,0.3,0.1,0.4],[0.1,0.3,0.2,0.4], [0.1,0.1,0.2,0.6]])
```

```
>>> print('The array at first index: ',arr[0])
>>> print('The array at second index: ',arr[1])
>>> print('The array at third index: ',arr[2])
```

```
>>> print('The array at fourth index: ',arr[3])
```

```
The array at first index: [0.1, 0.35, 0.25, 0.3]
The array at second index: [0.2, 0.3, 0.1, 0.4]
The array at third index: [0.1, 0.3, 0.2, 0.4]
```

```
The array at fourth index: [0.1, 0, 1, 0.2, 0.6]
>>> arr = [np.argmax(i) for i in arr]
>>> arr
[1, 3, 3, 3]
```

The softmax function is an activation function that turns numbers into probabilities which sum to one as depicted in NumPy array named as 'arr'. The softmax function produces an output as an array that represents the probability distributions of a list of outcomes.

Let's discuss some about Backpropagation after the predictions are given by the neural network then those predictions are compared with the original answers then error is calculated then to minimize the error then neural network is propagated backwards in a similar way by the involvement of the activation functions the forward propagation is done this process is repeated until we get the minimized or zero error rate.

In this article I discussed some concepts of the Neural Networks and some important activation functions used in deep learning.

This is all about my this article of learning python for Data Science as soon I learn different concepts I'll keep posting. So, there are more to go as I am learning.