Parameterization of the Stochastic Model for Evaluating Variable Small Data in the Shannon Entropy Basis

The article analytically summarizes the idea of applying Shannon’s principle of entropy maximization to sets that represent the results of observations of the “input” and “output” entities of the stochastic model for evaluating variable small data. To formalize this idea, a sequential transition fro...

Full description

Saved in:
Bibliographic Details
Published in:Entropy (Basel, Switzerland) Vol. 25; no. 2; p. 184
Main Authors: Bisikalo, Oleh, Kharchenko, Vyacheslav, Kovtun, Viacheslav, Krak, Iurii, Pavlov, Sergii
Format: Journal Article
Language:English
Published: Switzerland MDPI AG 17.01.2023
MDPI
Subjects:
Algorithms
Big Data
Control theory
Data analysis
Entropy
Entropy (Information theory)
Estimates
Evaluation
evaluation of small data
Hypotheses
Machine learning
Mathematical programming
Maximization
Maximum likelihood method
Mean square errors
measurement errors
Methods
Optimization
Parameter estimation
Parameter uncertainty
Parameterization
parametric optimization
Principles
Probability
Probability density functions
Probability distribution
Probability theory
Regression analysis
Shannon entropy
Statistical analysis
stochastic model
Stochastic models
Stochastic processes
Uncertainty (Information theory)
Variability
Ukraine
evaluation of small data
measurement errors
Shannon entropy
interval probabilities
normalized probabilities
machine learning
stochastic model
parametric optimization
ISSN:1099-4300, 1099-4300
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The article analytically summarizes the idea of applying Shannon’s principle of entropy maximization to sets that represent the results of observations of the “input” and “output” entities of the stochastic model for evaluating variable small data. To formalize this idea, a sequential transition from the likelihood function to the likelihood functional and the Shannon entropy functional is analytically described. Shannon’s entropy characterizes the uncertainty caused not only by the probabilistic nature of the parameters of the stochastic data evaluation model but also by interferences that distort the results of the measurements of the values of these parameters. Accordingly, based on the Shannon entropy, it is possible to determine the best estimates of the values of these parameters for maximally uncertain (per entropy unit) distortions that cause measurement variability. This postulate is organically transferred to the statement that the estimates of the density of the probability distribution of the parameters of the stochastic model of small data obtained as a result of Shannon entropy maximization will also take into account the fact of the variability of the process of their measurements. In the article, this principle is developed into the information technology of the parametric and non-parametric evaluation on the basis of Shannon entropy of small data measured under the influence of interferences. The article analytically formalizes three key elements: -instances of the class of parameterized stochastic models for evaluating variable small data; -methods of estimating the probability density function of their parameters, represented by normalized or interval probabilities; -approaches to generating an ensemble of random vectors of initial parameters.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:1099-4300
1099-4300
DOI:10.3390/e25020184