Understanding encoder–decoder structures in machine learning using information measures

We present a theory of representation learning to model and understand the role of encoder–decoder design in machine learning (ML) from an information-theoretic angle. We use two main information concepts, information sufficiency (IS) and mutual information loss to represent predictive structures in...

Full description

Saved in:

Bibliographic Details
Published in:	Signal processing Vol. 234; p. 109983
Main Authors:	Silva, Jorge F., Faraggi, Victor, Ramirez, Camilo, Egaña, Alvaro, Pavez, Eduardo
Format:	Journal Article
Language:	English
Published:	Elsevier B.V 01.09.2025
Subjects:	Cross-entropy loss Digital models Encoder expressiveness Encoder–decoder design Explainability Information bottleneck Information sufficiency Invariant models Representation learning Sparse models Representation learning Information bottleneck Invariant models Encoder expressiveness Digital models Encoder–decoder design Cross-entropy loss Information sufficiency Sparse models Explainability
ISSN:	0165-1684
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Be the first to leave a comment!