Linear Decoders
From Ufldl
| Line 1: | Line 1: | ||
== Sparse Autoencoder Recap == | == Sparse Autoencoder Recap == | ||
| - | In the sparse autoencoder implementation, we had 3 layers of neurons: | + | In the sparse autoencoder implementation, we had 3 layers of neurons: an input layer, a hidden layer and an output layer. In our previous description |
| - | In these | + | of autoencoders (and of neural networks), every neuron used the same activation function. |
| + | In these notes, we describe a modified version of the autoencoder in which some of the neurons use a different activation function. | ||
| + | This will result in a model that is sometimes simpler to apply. | ||
Recall that each neuron (in the output layer) computes the following: | Recall that each neuron (in the output layer) computes the following: | ||
