Linear Decoders
From Ufldl
Line 1: | Line 1: | ||
== Sparse Autoencoder Recap == | == Sparse Autoencoder Recap == | ||
- | In the sparse autoencoder implementation, we had 3 layers of neurons: | + | In the sparse autoencoder implementation, we had 3 layers of neurons: an input layer, a hidden layer and an output layer. In our previous description |
- | In these | + | of autoencoders (and of neural networks), every neuron used the same activation function. |
+ | In these notes, we describe a modified version of the autoencoder in which some of the neurons use a different activation function. | ||
+ | This will result in a model that is sometimes simpler to apply. | ||
Recall that each neuron (in the output layer) computes the following: | Recall that each neuron (in the output layer) computes the following: |