Softmax Regression

From Ufldl

Jump to: navigation, search
m (Added missing brackets to log-likelihood.)
m (Swapped partial and sum in gradient computation.)
Line 64: Line 64:
<math>
<math>
\begin{align}
\begin{align}
-
\frac{\partial \ell(\theta)}{\partial \theta_k} &= \sum_{i=1}^{m}{\left[\frac{\partial}{\partial \theta_k} \theta^T_{y^{(i)}} x^{(i)} - \ln \sum_{j=1}^{n}{e^{ \theta_j^T x^{(i)} }}\right]} \\
+
\frac{\partial \ell(\theta)}{\partial \theta_k} &= \frac{\partial}{\partial \theta_k} \sum_{i=1}^{m}{\left[ \theta^T_{y^{(i)}} x^{(i)} - \ln \sum_{j=1}^{n}{e^{ \theta_j^T x^{(i)} }}\right]} \\
&= \sum_{i=1}^{m}{ \left[ I_{ \{ y^{(i)} = k\} } x^{(i)} - \frac{1}{ \sum_{j=1}^{n}{e^{ \theta_j^T x^{(i)} }} }
&= \sum_{i=1}^{m}{ \left[ I_{ \{ y^{(i)} = k\} } x^{(i)} - \frac{1}{ \sum_{j=1}^{n}{e^{ \theta_j^T x^{(i)} }} }

Revision as of 04:07, 9 May 2011

Personal tools