深度学习 Bottleneck layer / Bottleneck feature

Posted 2021-01-05 z1141000271

最近在学习deeplearning的时候接触到了bottle-neck layer，好奇它的作用于是便扒了一些论文（论文链接放在文末吧），系统的了解一下bottle-neck feature究竟有什么用。

论文[1]中对bottle-neck feature的介绍：

对应的图示如下：

 

直观的理解是这玩意儿应该是用来降维用的，没错，那为什么用它比较好呢，另一篇论文[2]给了解释：

If we do not want to use the dimensionality reduction techniques, and want to obtain the features suitable for the classification as outcome of neural net training process, a bottle-neck has to be created in the neural net structure. The neural net has the ability of nonlinear compression of the input features and of classification of such compressed features. If the trained neural net with bottle-neck has a good classification accuracy, we know that the bottle-neck outputs represents the underlying speech well.（感兴趣的可以看看论文的背景，这样比较好理解） 

个人绝大非线性的压缩能力以及在网络中的可学习性是这个idea突出的地方（感觉过几个月回头看会觉得这个观点很好笑哈哈 姑且先写在这里吧）

 

 

reference:

[1] Efficient Processing of Deep Neural Networks: A Tutorial and Survey Vivienne Sze, Senior Member, IEEE, Yu-Hsin Chen, Student Member, IEEE, Tien-Ju Yang, Student Member, IEEE, Joel Emer, Fellow, IEEE

[2] PROBABILISTIC AND BOTTLE-NECK FEATURES FOR LVCSR OF MEETINGS Frantisek ˇ Grezl, ′ Martin Karafiat, ′ Stanislav Kontar′ and Jan Cernoc ˇ ky′ [email protected] group, Brno University of Technology, Czech Republic

 

链接：http://www.fit.vutbr.cz/research/groups/speech/publi/2007/grezl_BN_fea_icassp_2007.pdf

　　　https://arxiv.org/pdf/1703.09039.pdf   （要梯子）