A Mixed-Integer linear programming based training and feature selection method for artificial neural networks using piece-wise linear approximations

•A piece-wise linear ANN training is proposed.•Non-convex layers in ANNs are approximated using piece-wise linear segments.•The method detects the optimum features to be selected for ANNs.•Layer and objective function convexities bring about unique solutions.•Convexity of the method benefits from co...

Full description

Saved in:
Bibliographic Details
Published in:Chemical engineering science Vol. 249; p. 117273
Main Authors: Sildir, Hasan, Aydin, Erdal
Format: Journal Article
Language:English
Published: Elsevier Ltd 15.02.2022
Subjects:
Artificial neural networks
Feature selection
Machine learning
Mixed-integer programming
Piece-wise linear artificial neural networks
Feature selection
Piece-wise linear artificial neural networks
Mixed-integer programming
Machine learning
Artificial neural networks
ISSN:0009-2509
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:•A piece-wise linear ANN training is proposed.•Non-convex layers in ANNs are approximated using piece-wise linear segments.•The method detects the optimum features to be selected for ANNs.•Layer and objective function convexities bring about unique solutions.•Convexity of the method benefits from computational advantages. Artificial Neural Networks (ANNs) may suffer from suboptimal training and test performance related issues not only because of the presence of high number of features with low statistical contributions but also due to their non-convex nature. This study develops piecewise-linear formulations for the efficient approximation of the non-convex activation and objective functions in artificial neural networks for optimal, global and simultaneous training and feature selection in regression problems. Such formulations include binary variables to account for the existence of the features and piecewise-linear approximations, which in turn, after one exact linearization step, calls for solving a mixed-integer linear programming problem with a global optimum guarantee because of convexity. Suggested formulation is implemented on two industrial case studies. Results show that efficient approximations are obtained through the usage of the method with only a few number of breakpoints. Significant feature space reduction is observed bringing about notable improvement in test accuracy.
ISSN:0009-2509
DOI:10.1016/j.ces.2021.117273