Direct Parallel Perceptrons (DPPs): Fast Analytical Calculation of the Parallel Perceptrons Weights With Margin Control for Classification Tasks

Parallel perceptrons (PPs) are very simple and efficient committee machines (a single layer of perceptrons with threshold activation functions and binary outputs, and a majority voting decision scheme), which nevertheless behave as universal approximators. The parallel delta (P-Delta) rule is an eff...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on neural networks Vol. 22; no. 11; pp. 1837 - 1848
Main Authors: Fernandez-Delgado, M., Ribeiro, J., Cernadas, E., Ameneiro, S. B.
Format: Journal Article
Language:English
Published: New York, NY IEEE 01.11.2011
Institute of Electrical and Electronics Engineers
Subjects:
Accuracy
Activation
Algorithmics. Computability. Computer arithmetics
Algorithms
Analytical closed-form weight calculation
Applied sciences
Artificial Intelligence
Classification
Classification - methods
Closed-form solutions
Computer science; control theory; systems
Connectionism. Neural networks
Data processing. List processing. Character string processing
Databases, Factual
Exact sciences and technology
Kernel
Linear approximation
linear computational complexity
Linear Models
margin maximization
Mathematical analysis
Mathematical models
Memory and file management (including protection and security)
Memory organisation. Data processing
Neural Networks (Computer)
online learning
parallel delta rule
parallel perceptrons
pattern classification
Reproducibility of Results
Software
Statistical learning
Support vector machines
Tasks
Theoretical computing
Thresholds
Training
parallel perceptrons
Competitiveness
parallel delta rule
Modeling
Optimization
Direct method
Multidimensional analysis
Vector support machine
Linear complexity
Analytical closed-form weight calculation
Probability learning
Electronic vote
Threshold function
Single machine
Dimensionality
Statistical analysis
margin maximization
Error function
linear computational complexity
Neural network
Computational complexity
Hyperplane
Exact solution
Perceptron
Activation function
Storage management
Pattern classification
Time complexity
online learning
ISSN:1045-9227, 1941-0093, 1941-0093
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Parallel perceptrons (PPs) are very simple and efficient committee machines (a single layer of perceptrons with threshold activation functions and binary outputs, and a majority voting decision scheme), which nevertheless behave as universal approximators. The parallel delta (P-Delta) rule is an effective training algorithm, which, following the ideas of statistical learning theory used by the support vector machine (SVM), raises its generalization ability by maximizing the difference between the perceptron activations for the training patterns and the activation threshold (which corresponds to the separating hyperplane). In this paper, we propose an analytical closed-form expression to calculate the PPs' weights for classification tasks. Our method, called Direct Parallel Perceptrons (DPPs), directly calculates (without iterations) the weights using the training patterns and their desired outputs, without any search or numeric function optimization. The calculated weights globally minimize an error function which simultaneously takes into account the training error and the classification margin. Given its analytical and noniterative nature, DPPs are computationally much more efficient than other related approaches (P-Delta and SVM), and its computational complexity is linear in the input dimensionality. Therefore, DPPs are very appealing, in terms of time complexity and memory consumption, and are very easy to use for high-dimensional classification tasks. On real benchmark datasets with two and multiple classes, DPPs are competitive with SVM and other approaches but they also allow online learning and, as opposed to most of them, have no tunable parameters.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ObjectType-Article-2
ObjectType-Feature-1
ISSN:1045-9227
1941-0093
1941-0093
DOI:10.1109/TNN.2011.2169086