Skip to Main content Skip to Navigation
Directions of work or proceedings

Special Issue "Information Transfer in Multilayer/Deep Architectures"

Abstract : The renewal of research interest in machine learning came with the emergence of the concept of big data during the late 2000s. Schematically, families of deep learning networks (DLN) emerged with industrial ambitions, taking advantage of the development of graphics cards (GPUs) to construct prediction models from massive amounts of collected and stored data and substantial means of calculation. It is illusory to want to learn a deep network involving millions of parameters without very large databases. We tend to think that more data lead to more information. In addition, the core of learning is all but a problem of data representation, not in the ‘data compression’ sense. For instance, in DLN, one representation (input layer) is replaced by a cascade of many representations (hidden layers), which means an increase of information (entropy). However, some questions remain: How does information spread in these inflationary networks? Is information transform conservative through the DLN? Can information theory quantify the learning capacity of these networks? How do generative models convert information from the observed space to the hidden space? Foreseen contributions include the following: - high-dimension feature selection and pattern correlations - information entropy in large data representation - information gain in decision trees - between layer dependencies - auto-encoding - network capacity and information loss - etc. This Special Issue has the ambition to collect responses to these questions from the theorical and applicative points of view. Prof. Vincent Vigneron Prof. Hichem Maaref Guest Editors
Document type :
Directions of work or proceedings
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-02398644
Contributor : Frédéric Davesne Connect in order to contact the contributor
Submitted on : Saturday, December 7, 2019 - 7:00:02 PM
Last modification on : Monday, December 13, 2021 - 9:17:14 AM

Identifiers

  • HAL Id : hal-02398644, version 1

Citation

Vincent Vigneron, Hichem Maaref. Special Issue "Information Transfer in Multilayer/Deep Architectures". Entropy, 2021. ⟨hal-02398644⟩

Share

Metrics

Record views

129