# 稀疏编码

(Difference between revisions)
 Revision as of 04:30, 8 April 2013 (view source)Kandeng (Talk | contribs)← Older edit Revision as of 08:16, 8 April 2013 (view source)Kandeng (Talk | contribs) (→中英文对照)Newer edit → Line 143: Line 143: ==中英文对照== ==中英文对照== - + :稀疏编码 Sparse Coding + :无监督学习 unsupervised method + :超完备基 over-complete bases + :主成分分析 PCA + :稀疏性 sparsity + :退化 degeneracy + :代价函数 cost function + :重构项 reconstruction term + :稀疏惩罚项 sparsity penalty + :范式 norm + :生成模型 generative model + :线性叠加 linear superposition + :加性噪声 additive noise + :特征基向量 basis feature vectors + :经验分布函数 the empirical distribution + :KL 散度 KL divergence + :对数似然函数 the log-likelihood + :高斯白噪音 Gaussian white noise + :先验分布 the prior distribution + :先验概率 prior probability + :源特征 source features + :能量函数 the energy function + :正则化 regularized + :最小二乘法 least squares + :凸优化软件convex optimization software + :共轭梯度法 conjugate gradient methods + :二次约束 quadratic constraints + :拉格朗日对偶函数 the Lagrange dual + :前馈结构算法 feedforward architectures ==中文译者== ==中文译者==

## 稀疏编码

\begin{align} \mathbf{x} = \sum_{i=1}^k a_i \mathbf{\phi}_{i} \end{align}

\begin{align} \text{minimize}_{a^{(j)}_i,\mathbf{\phi}_{i}} \sum_{j=1}^{m} \left|\left| \mathbf{x}^{(j)} - \sum_{i=1}^k a^{(j)}_i \mathbf{\phi}_{i}\right|\right|^{2} + \lambda \sum_{i=1}^{k}S(a^{(j)}_i) \end{align}

$\begin{array}{rc} \text{minimize}_{a^{(j)}_i,\mathbf{\phi}_{i}} & \sum_{j=1}^{m} \left|\left| \mathbf{x}^{(j)} - \sum_{i=1}^k a^{(j)}_i \mathbf{\phi}_{i}\right|\right|^{2} + \lambda \sum_{i=1}^{k}S(a^{(j)}_i) \\ \text{subject to} & \left|\left|\mathbf{\phi}_i\right|\right|^2 \leq C, \forall i = 1,...,k \\ \end{array}$

## 概率解释 [基于1996年Olshausen与Field的理论]

\begin{align} \mathbf{x} = \sum_{i=1}^k a_i \mathbf{\phi}_{i} + \nu(\mathbf{x}) \end{align}

\begin{align} D(P^*(\mathbf{x})||P(\mathbf{x}\mid\mathbf{\phi})) = \int P^*(\mathbf{x}) \log \left(\frac{P^*(\mathbf{x})}{P(\mathbf{x}\mid\mathbf{\phi})}\right)d\mathbf{x} \end{align}

\begin{align} P(\mathbf{x} \mid \mathbf{a}, \mathbf{\phi}) = \frac{1}{Z} \exp\left(- \frac{(\mathbf{x}-\sum^{k}_{i=1} a_i \mathbf{\phi}_{i})^2}{2\sigma^2}\right) \end{align}

\begin{align} P(\mathbf{a}) = \prod_{i=1}^{k} P(a_i) \end{align}

\begin{align} P(a_i) = \frac{1}{Z}\exp(-\beta S(a_i)) \end{align}

\begin{align} P(\mathbf{x} \mid \mathbf{\phi}) = \int P(\mathbf{x} \mid \mathbf{a}, \mathbf{\phi}) P(\mathbf{a}) d\mathbf{a} \end{align}

\begin{align} \mathbf{\phi}^*=\text{argmax}_{\mathbf{\phi}} < \log(P(\mathbf{x} \mid \mathbf{\phi})) > \end{align}

\begin{align} \mathbf{\phi}^{*'}=\text{argmax}_{\mathbf{\phi}} < \max_{\mathbf{a}} \log(P(\mathbf{x} \mid \mathbf{\phi})) > \end{align}

$\begin{array}{rl} E\left( \mathbf{x} , \mathbf{a} \mid \mathbf{\phi} \right) & := -\log \left( P(\mathbf{x}\mid \mathbf{\phi},\mathbf{a}\right)P(\mathbf{a})) \\ &= \sum_{j=1}^{m} \left|\left| \mathbf{x}^{(j)} - \sum_{i=1}^k a^{(j)}_i \mathbf{\phi}_{i}\right|\right|^{2} + \lambda \sum_{i=1}^{k}S(a^{(j)}_i) \end{array}$

\begin{align} \mathbf{\phi}^{*},\mathbf{a}^{*}=\text{argmin}_{\mathbf{\phi},\mathbf{a}} \sum_{j=1}^{m} \left|\left| \mathbf{x}^{(j)} - \sum_{i=1}^k a^{(j)}_i \mathbf{\phi}_{i}\right|\right|^{2} + \lambda \sum_{i=1}^{k}S(a^{(j)}_i) \end{align}

## 学习算法

L2 范式约束来学习基向量，同样可以简化为一个带有二次约束的最小二乘问题，其问题函数在域 $\mathbf{\phi}$ 内也为凸。标准的凸优化软件（如CVX）或其它迭代方法就可以用来求解 $\mathbf{\phi}$，虽然已经有了更有效的方法，比如求解拉格朗日对偶函数（Lagrange dual）。

## 中英文对照

KL 散度 KL divergence

## 中文译者

Language : English