Neural Networks
From Ufldl
Line 59: | Line 59: | ||
In this figure, we have used circles to also denote the inputs to the network. The circles | In this figure, we have used circles to also denote the inputs to the network. The circles | ||
- | labeled ``+1'' are called | + | labeled ``+1'' are called '''bias units''', and correspond to the intercept term. |
- | The leftmost layer of the network is called the | + | The leftmost layer of the network is called the '''input layer''', and the |
- | rightmost layer the | + | rightmost layer the '''output layer''' (which, in this example, has only one |
- | node). The middle layer of nodes is called the | + | node). The middle layer of nodes is called the '''hidden layer''', because its |
values are not observed in the training set. We also say that our example | values are not observed in the training set. We also say that our example | ||
- | neural network has 3 | + | neural network has 3 '''input units''' (not counting the bias unit), 3 |
- | hidden units | + | '''hidden units''', and 1 '''output unit'''. |
We will let <math>n_l</math> | We will let <math>n_l</math> | ||
Line 79: | Line 79: | ||
the value +1. We also let <math>s_l</math> denote the number of nodes in layer <math>l</math> (not counting the bias unit). | the value +1. We also let <math>s_l</math> denote the number of nodes in layer <math>l</math> (not counting the bias unit). | ||
- | We will write <math>a^{(l)}_i</math> to denote the | + | We will write <math>a^{(l)}_i</math> to denote the '''activation''' (meaning output value) of |
unit <math>i</math> in layer <math>l</math>. For <math>l=1</math>, we also use <math>a^{(1)}_i = x_i</math> to denote the <math>i</math>-th input. | unit <math>i</math> in layer <math>l</math>. For <math>l=1</math>, we also use <math>a^{(1)}_i = x_i</math> to denote the <math>i</math>-th input. | ||
Given a fixed setting of | Given a fixed setting of |