Softmax Regression
From Ufldl
m (Added missing brackets to log-likelihood.) |
m (Swapped partial and sum in gradient computation.) |
||
Line 64: | Line 64: | ||
<math> | <math> | ||
\begin{align} | \begin{align} | ||
- | \frac{\partial \ell(\theta)}{\partial \theta_k} &= \sum_{i=1}^{m}{\left[ | + | \frac{\partial \ell(\theta)}{\partial \theta_k} &= \frac{\partial}{\partial \theta_k} \sum_{i=1}^{m}{\left[ \theta^T_{y^{(i)}} x^{(i)} - \ln \sum_{j=1}^{n}{e^{ \theta_j^T x^{(i)} }}\right]} \\ |
&= \sum_{i=1}^{m}{ \left[ I_{ \{ y^{(i)} = k\} } x^{(i)} - \frac{1}{ \sum_{j=1}^{n}{e^{ \theta_j^T x^{(i)} }} } | &= \sum_{i=1}^{m}{ \left[ I_{ \{ y^{(i)} = k\} } x^{(i)} - \frac{1}{ \sum_{j=1}^{n}{e^{ \theta_j^T x^{(i)} }} } |