Neural Network Vectorization
From Ufldl
Line 130: | Line 130: | ||
% Sparsity Penalty Delta | % Sparsity Penalty Delta | ||
sparsity_delta = - rho / rho_hat + (1 - rho) / rho_hat; | sparsity_delta = - rho / rho_hat + (1 - rho) / rho_hat; | ||
- | delta2 = (W2'*delta3(:,i) + beta*sparsity_delta).* fprime(z2(:,i)); | + | for i=1:m, |
+ | ... | ||
+ | delta2 = (W2'*delta3(:,i) + beta*sparsity_delta).* fprime(z2(:,i)); | ||
+ | ... | ||
+ | end; | ||
</syntaxhighlight> | </syntaxhighlight> | ||
- | + | Notice that the sparsity_delta is the same regardless of the example we are processing. | |
+ | |||
+ | Recall that when we vectorizing the gradient computations, <tt>delta2</tt> is replaced with matrices. Thus, to vectorize the <tt>sparsity_delta</tt>, we can simply add it using <tt>repmat</tt> while computing the <tt>delta2</tt> matrices. |