# Neural Networks

### From Ufldl

Line 59: | Line 59: | ||

In this figure, we have used circles to also denote the inputs to the network. The circles | In this figure, we have used circles to also denote the inputs to the network. The circles | ||

- | labeled ``+1'' are called | + | labeled ``+1'' are called '''bias units''', and correspond to the intercept term. |

- | The leftmost layer of the network is called the | + | The leftmost layer of the network is called the '''input layer''', and the |

- | rightmost layer the | + | rightmost layer the '''output layer''' (which, in this example, has only one |

- | node). The middle layer of nodes is called the | + | node). The middle layer of nodes is called the '''hidden layer''', because its |

values are not observed in the training set. We also say that our example | values are not observed in the training set. We also say that our example | ||

- | neural network has 3 | + | neural network has 3 '''input units''' (not counting the bias unit), 3 |

- | hidden units | + | '''hidden units''', and 1 '''output unit'''. |

We will let <math>n_l</math> | We will let <math>n_l</math> | ||

Line 79: | Line 79: | ||

the value +1. We also let <math>s_l</math> denote the number of nodes in layer <math>l</math> (not counting the bias unit). | the value +1. We also let <math>s_l</math> denote the number of nodes in layer <math>l</math> (not counting the bias unit). | ||

- | We will write <math>a^{(l)}_i</math> to denote the | + | We will write <math>a^{(l)}_i</math> to denote the '''activation''' (meaning output value) of |

unit <math>i</math> in layer <math>l</math>. For <math>l=1</math>, we also use <math>a^{(1)}_i = x_i</math> to denote the <math>i</math>-th input. | unit <math>i</math> in layer <math>l</math>. For <math>l=1</math>, we also use <math>a^{(1)}_i = x_i</math> to denote the <math>i</math>-th input. | ||

Given a fixed setting of | Given a fixed setting of |