Understanding encoder–decoder structures in machine learning using information measures

We present a theory of representation learning to model and understand the role of encoder–decoder design in machine learning (ML) from an information-theoretic angle. We use two main information concepts, information sufficiency (IS) and mutual information loss to represent predictive structures in...

Ausführliche Beschreibung