A Provably Accurate Randomized Sampling Algorithm for Logistic Regression

In statistics and machine learning, logistic regression is a widely-used supervised learning technique primarily employed for binary classification tasks. When the number of observations greatly exceeds the number of predictor variables, we present a simple, randomized sampling-based algorithm for l...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org
Main Authors: Chowdhury, Agniva, Ramuhalli, Pradeep
Format: Paper
Language:English
Published: Ithaca Cornell University Library, arXiv.org 31.03.2024
Subjects:
Algorithms
Approximation
Empirical analysis
Linear algebra
Machine learning
Multiplication
Regression
Regression analysis
Sampling
Supervised learning
ISSN:2331-8422
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract In statistics and machine learning, logistic regression is a widely-used supervised learning technique primarily employed for binary classification tasks. When the number of observations greatly exceeds the number of predictor variables, we present a simple, randomized sampling-based algorithm for logistic regression problem that guarantees high-quality approximations to both the estimated probabilities and the overall discrepancy of the model. Our analysis builds upon two simple structural conditions that boil down to randomized matrix multiplication, a fundamental and well-understood primitive of randomized numerical linear algebra. We analyze the properties of estimated probabilities of logistic regression when leverage scores are used to sample observations, and prove that accurate approximations can be achieved with a sample whose size is much smaller than the total number of observations. To further validate our theoretical findings, we conduct comprehensive empirical evaluations. Overall, our work sheds light on the potential of using randomized sampling approaches to efficiently approximate the estimated probabilities in logistic regression, offering a practical and computationally efficient solution for large-scale datasets.
AbstractList In statistics and machine learning, logistic regression is a widely-used supervised learning technique primarily employed for binary classification tasks. When the number of observations greatly exceeds the number of predictor variables, we present a simple, randomized sampling-based algorithm for logistic regression problem that guarantees high-quality approximations to both the estimated probabilities and the overall discrepancy of the model. Our analysis builds upon two simple structural conditions that boil down to randomized matrix multiplication, a fundamental and well-understood primitive of randomized numerical linear algebra. We analyze the properties of estimated probabilities of logistic regression when leverage scores are used to sample observations, and prove that accurate approximations can be achieved with a sample whose size is much smaller than the total number of observations. To further validate our theoretical findings, we conduct comprehensive empirical evaluations. Overall, our work sheds light on the potential of using randomized sampling approaches to efficiently approximate the estimated probabilities in logistic regression, offering a practical and computationally efficient solution for large-scale datasets.
Author Ramuhalli, Pradeep
Chowdhury, Agniva
Author_xml – sequence: 1
  givenname: Agniva
  surname: Chowdhury
  fullname: Chowdhury, Agniva
– sequence: 2
  givenname: Pradeep
  surname: Ramuhalli
  fullname: Ramuhalli, Pradeep
BookMark eNotzU9LwzAYgPEgCs65D-At4Ln1zZt_7bEMdYOCMncfaZrWjK6ZSTfUT6-gp-f2e27I5RhGR8gdg1wUUsKDiZ_-nKMAzJniqC7IDDlnWSEQr8kipT0AoNIoJZ-RdUVfYzibZviilbWnaCZHN2Zsw8F_u5a-mcNx8GNPq6EP0U_vB9qFSOvQ-zR5Szeujy4lH8ZbctWZIbnFf-dk-_S4Xa6y-uV5vazqzEgsM1QFaxy3qFuLCo0AcACi1Fq30pSdYrKAxhYNQ2etttJIhV1nBWiBbdPxObn_Y48xfJxcmnb7cIrj73GHJUfOFMiS_wA-u0-d
ContentType Paper
Copyright 2024. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml – notice: 2024. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID 8FE
8FG
ABJCF
ABUWG
AFKRA
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
HCIFZ
L6V
M7S
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
DOI 10.48550/arxiv.2402.16326
DatabaseName ProQuest SciTech Collection
ProQuest Technology Collection
Materials Science & Engineering Collection
ProQuest Central (Alumni)
ProQuest Central UK/Ireland
ProQuest Central Essentials - QC
ProQuest Central
Technology Collection
ProQuest One
ProQuest Central
SciTech Premium Collection
ProQuest Engineering Collection
Engineering Database
ProQuest Central Premium
ProQuest One Academic (New)
Publicly Available Content Database
ProQuest One Academic Middle East (New)
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Applied & Life Sciences
ProQuest One Academic (retired)
ProQuest One Academic UKI Edition
ProQuest Central China
Engineering Collection
DatabaseTitle Publicly Available Content Database
Engineering Database
Technology Collection
ProQuest One Academic Middle East (New)
ProQuest Central Essentials
ProQuest One Academic Eastern Edition
ProQuest Central (Alumni Edition)
SciTech Premium Collection
ProQuest One Community College
ProQuest Technology Collection
ProQuest SciTech Collection
ProQuest Central China
ProQuest Central
ProQuest One Applied & Life Sciences
ProQuest Engineering Collection
ProQuest One Academic UKI Edition
ProQuest Central Korea
Materials Science & Engineering Collection
ProQuest Central (New)
ProQuest One Academic
ProQuest One Academic (New)
Engineering Collection
DatabaseTitleList Publicly Available Content Database
Database_xml – sequence: 1
  dbid: PIMPY
  name: Publicly Available Content Database
  url: http://search.proquest.com/publiccontent
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Physics
EISSN 2331-8422
Genre Working Paper/Pre-Print
GroupedDBID 8FE
8FG
ABJCF
ABUWG
AFKRA
ALMA_UNASSIGNED_HOLDINGS
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
FRJ
HCIFZ
L6V
M7S
M~E
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
ID FETCH-LOGICAL-a529-2681be3c27dc262a400e0049777d5a9f61580bc8b12ecc7c5a562ffc40742dbf3
IEDL.DBID BENPR
IngestDate Mon Jun 30 09:24:32 EDT 2025
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a529-2681be3c27dc262a400e0049777d5a9f61580bc8b12ecc7c5a562ffc40742dbf3
Notes SourceType-Working Papers-1
ObjectType-Working Paper/Pre-Print-1
content type line 50
OpenAccessLink https://www.proquest.com/docview/2932316059?pq-origsite=%requestingapplication%
PQID 2932316059
PQPubID 2050157
ParticipantIDs proquest_journals_2932316059
PublicationCentury 2000
PublicationDate 20240331
PublicationDateYYYYMMDD 2024-03-31
PublicationDate_xml – month: 03
  year: 2024
  text: 20240331
  day: 31
PublicationDecade 2020
PublicationPlace Ithaca
PublicationPlace_xml – name: Ithaca
PublicationTitle arXiv.org
PublicationYear 2024
Publisher Cornell University Library, arXiv.org
Publisher_xml – name: Cornell University Library, arXiv.org
SSID ssj0002672553
Score 1.8651023
SecondaryResourceType preprint
Snippet In statistics and machine learning, logistic regression is a widely-used supervised learning technique primarily employed for binary classification tasks. When...
SourceID proquest
SourceType Aggregation Database
SubjectTerms Algorithms
Approximation
Empirical analysis
Linear algebra
Machine learning
Multiplication
Regression
Regression analysis
Sampling
Supervised learning
Title A Provably Accurate Randomized Sampling Algorithm for Logistic Regression
URI https://www.proquest.com/docview/2932316059
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpZ1LT8JAEMc3Cpp48h0fSPbgtTy2pds9GTQQTZA0wAFPZLs7xSbQYgtE_fTOlqIHEy8emyZNs9vO_HbmPzOE3CoOYLuA5CYCYTlSCktoAMsN0XlI5A0QeaFwj_f73ngs_CLglhWyyq1NzA21TpSJkdfRLSGKIHyLu8WbZaZGmexqMUJjl5RNpzKnRMr3nb4_-I6yMJcjM9ubdGbevKsu0_doXTNJhRqyCHN_GeHcs3QP__tOR6TsywWkx2QH4hOynys6VXZKntrUT5O1DGYftK3UyrSEoAMZ62QefYKmQ2m05PGUtmdTfObydU6RX2kvrwiKFB3AdCORjc_IqNsZPTxaxdwES7aYsJiLKAq2Ylwr5jKJfymYgwDnXLekCJFhvEagvKDJcP-4aklkoDBUjjkm6yC0z0kpTmK4INR0nwu41iGEzJHNhgBpu0oFjvLy-oVLUtkuzKT49rPJz6pc_X37mhwwRIRNhV-FlJbpCm7InlovoyytFltZNWrMIV75T8_-yxebcqzR
linkProvider ProQuest
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMw1Z1LTwIxEIAnPqMn3_FtD3pc0O6ypQdjiEogIiHIgRvptrO4iSy4ID7-k__RaRE9mHjz4HmTzXZnOvO18wI41gLRD5HITUbSC5SSnjSIXhiT81DEGyhdoXBN1OvFdls2ZuB9Wgtj0yqnNtEZatPX9o48T26JUITgW14MHj07NcpGV6cjNCZqcYOvz3RkG55Xr0i-J5yXr1uXFe9zqoCnClx6PCRQQ19zYTQPuSIdRovJQghTUDImD188jXQxOuO0OqELigghjnVgD5Emin167SzMB9b4u0zBu68rHR4KAnR_Ejt1ncLyKntJxjkbwcgR-PDwh8V3bqy88s9-wCrMN9QAszWYwXQdFl22qh5uQLXEGll_rKKHV1bS-sm2u2BNlZp-L3lDw-6UzZNPu6z00KUljO57jNic1Vy1U6JZE7uT9N90E1p_8fFbMJf2U9wGZjvrRcKYGGMeqLNTicoPtY4CXXS1GTuwP5VD53NfDzvfQtj9_fERLFVat7VOrVq_2YNlTig0qWTch7lR9oQHsKDHo2SYHTodYtD5Y5F9AHpGBAM
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+Provably+Accurate+Randomized+Sampling+Algorithm+for+Logistic+Regression&rft.jtitle=arXiv.org&rft.au=Chowdhury%2C+Agniva&rft.au=Ramuhalli%2C+Pradeep&rft.date=2024-03-31&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422&rft_id=info:doi/10.48550%2Farxiv.2402.16326