# Neural Networks

### From Ufldl

(Created page with "Consider a supervised learning problem where we have access to labeled training examples <math>(x^{(i)}, y^{(i)})</math>. Neural networks give a way of defining a complex, non-l...") |
|||

Line 8: | Line 8: | ||

diagram to denote a single neuron: | diagram to denote a single neuron: | ||

- | + | [[Image:SingleNeuron.png|400px|center]] | |

- | This | + | This "neuron" is a computational unit that takes as input <math>x_1, x_2, x_3</math> (and a +1 intercept term), and |

outputs <math>h_{W,b}(x) = f(W^Tx) = f(\sum_{i=1}^3 W_{i}x_i +b)</math>, where <math>f : \Re \mapsto \Re</math> is | outputs <math>h_{W,b}(x) = f(W^Tx) = f(\sum_{i=1}^3 W_{i}x_i +b)</math>, where <math>f : \Re \mapsto \Re</math> is | ||

called the '''activation function'''. In these notes, we will choose | called the '''activation function'''. In these notes, we will choose |

## Revision as of 05:36, 26 February 2011

Consider a supervised learning problem where we have access to labeled training
examples (*x*^{(i)},*y*^{(i)}). Neural networks give a way of defining a complex,
non-linear form of hypotheses *h*_{W,b}(*x*), with parameters *W*,*b* that we can
fit to our data.

To describe neural networks, we will begin by describing the simplest possible neural network, one which comprises a single "neuron." We will use the following diagram to denote a single neuron:

This "neuron" is a computational unit that takes as input *x*_{1},*x*_{2},*x*_{3} (and a +1 intercept term), and
outputs , where is
called the **activation function**. In these notes, we will choose
to be the sigmoid function:

Thus, our single neuron corresponds exactly to the input-output mapping defined by logistic regression.

Although these notes will use the sigmoid function, it is worth noting that
another common choice for *f* is the hyperbolic tangent, or tanh, function:

Here are plots of the sigmoid and tanh functions: