Neural Networks
From Ufldl
Line 117: | Line 117: | ||
By organizing our parameters in matrices and using matrix-vector operations, we can take | By organizing our parameters in matrices and using matrix-vector operations, we can take | ||
advantage of fast linear algebra routines to quickly perform calculations in our network. | advantage of fast linear algebra routines to quickly perform calculations in our network. | ||
+ | |||
We have so far focused on one example neural network, but one can also build neural | We have so far focused on one example neural network, but one can also build neural | ||
- | networks with other | + | networks with other {\bf |
- | The most common choice is a <math>n_l</math>-layered network | + | architectures} (meaning patterns of connectivity between neurons), including ones with multiple hidden layers. |
- | where layer <math>1</math> is the input layer, layer <math>n_l</math> is the output layer, and each | + | The most common choice is a <math>\textstyle n_l</math>-layered network |
- | layer <math>l</math> is densely connected to layer <math>l+1</math>. In this setting, to compute the | + | where layer <math>\textstyle 1</math> is the input layer, layer <math>\textstyle n_l</math> is the output layer, and each |
+ | layer <math>\textstyle l</math> is densely connected to layer <math>\textstyle l+1</math>. In this setting, to compute the | ||
output of the network, we can successively compute all the activations in layer | output of the network, we can successively compute all the activations in layer | ||
- | <math>L_2</math>, then layer <math>L_3</math>, and so on, up to layer <math>L_{n_l}</math>, using Equations~(\ref{eqn-forwardprop1}-\ref{eqn-forwardprop2}). This is one | + | <math>\textstyle L_2</math>, then layer <math>\textstyle L_3</math>, and so on, up to layer <math>\textstyle L_{n_l}</math>, using Equations~(\ref{eqn-forwardprop1}-\ref{eqn-forwardprop2}). This is one |
- | example of a | + | example of a '''feedforward''' neural network, since the connectivity graph |
does not have any directed loops or cycles. | does not have any directed loops or cycles. | ||
+ | %We will write <math>\textstyle s_l</math> to denote the | ||
+ | %number of units in layer <math>\textstyle l</math> of the network (not counting the bias unit). | ||
+ | |||
Neural networks can also have multiple output units. For example, here is a network | Neural networks can also have multiple output units. For example, here is a network |