Softmax Regression
From Ufldl
(→Introduction) |
(→Cost Function) |
||
| Line 133: | Line 133: | ||
We now describe the cost function that we'll use for softmax regression. In the equation below, <math>1\{\cdot\}</math> is | We now describe the cost function that we'll use for softmax regression. In the equation below, <math>1\{\cdot\}</math> is | ||
| - | the '''indicator function,''' so that <math>1\{\hbox{ | + | the '''indicator function,''' so that <math>1\{\hbox{a true statement}\}=1</math>, and <math>1\{\hbox{a false statement}\}=0</math>. |
For example, <math>1\{2+2=4\}</math> evaluates to 1; whereas <math>1\{1+1=5\}</math> evaluates to 0. Our cost function will be: | For example, <math>1\{2+2=4\}</math> evaluates to 1; whereas <math>1\{1+1=5\}</math> evaluates to 0. Our cost function will be: | ||
<math> | <math> | ||
\begin{align} | \begin{align} | ||
| - | J(\theta) = - \frac{1}{m} \left[ \sum_{i=1}^{m} \sum_{j=1}^{k} 1\left\{y^{(i)} = j\right\} \log \frac{\theta_j^T x^{(i)}}{\sum_{l=1}^k e^{ \theta_l^T x^{(i)} }}\right] | + | J(\theta) = - \frac{1}{m} \left[ \sum_{i=1}^{m} \sum_{j=1}^{k} 1\left\{y^{(i)} = j\right\} \log \frac{e^{\theta_j^T x^{(i)}}}{\sum_{l=1}^k e^{ \theta_l^T x^{(i)} }}\right] |
\end{align} | \end{align} | ||
</math> | </math> | ||
