Application of improved distributed naive Bayesian algorithms in text classification

The naive Bayes classifier is a widely used text classification method that applies statistical theory to text classification. Due to the particularity of the text, related feature items may generate new semantic information, which may be lost when the traditional vector space model represents text....

Full description

Saved in:
Bibliographic Details
Published in:The Journal of supercomputing Vol. 75; no. 9; pp. 5831 - 5847
Main Authors: Gao, Hongyi, Zeng, Xi, Yao, Chunhua
Format: Journal Article
Language:English
Published: New York Springer US 01.09.2019
Springer Nature B.V
Subjects:
Algorithms
Bayesian analysis
Classification
Cloud computing
Comparative analysis
Compilers
Computer Science
Interpreters
Processor Architectures
Programming Languages
Text editing
Websites
Feature selection
Naive Bayesian algorithm
Distributed
Text classification
ISSN:0920-8542, 1573-0484
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The naive Bayes classifier is a widely used text classification method that applies statistical theory to text classification. Due to the particularity of the text, related feature items may generate new semantic information, which may be lost when the traditional vector space model represents text. This paper mainly studies the construction and improvement of distributed naive Bayes automatic classification system. The application of Hadoop cloud computing in web page classification is one of the focuses of this article. Firstly, the text classification system and Bayesian classification model are analyzed and discussed, including the representation and extraction of text information, text classification methods and Bayesian text classification methods. Then, in view of the shortcomings of the above-mentioned naive Bayesian text classification method, when training text, we use the mutual information method to check the correlation between the feature sets generated after feature selection, and then combine the features with higher correlation degree appropriately. Through a series of tests, the experimental data show that the improved text classification system can achieve better classification results.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0920-8542
1573-0484
DOI:10.1007/s11227-019-02862-1