A soft computing approach for benign and malicious web robot detection
•We propose a method called SMART (Soft computing for MAlicious RoboT detection).•The method detects benign and malicious robots, and human visitors to a web server.•SMART selects its features on a particular web server by fuzzy rough set theory.•A graph-based clustering algorithm classifies session...
Uloženo v:
| Vydáno v: | Expert systems with applications Ročník 87; s. 129 - 140 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
New York
Elsevier Ltd
30.11.2017
Elsevier BV |
| Témata: | |
| ISSN: | 0957-4174, 1873-6793 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | •We propose a method called SMART (Soft computing for MAlicious RoboT detection).•The method detects benign and malicious robots, and human visitors to a web server.•SMART selects its features on a particular web server by fuzzy rough set theory.•A graph-based clustering algorithm classifies sessions into the three agent types.•Analyses on web logs suggest state-of-the-art results to detect both robot types.
The accurate detection of web robot sessions from a web server log is essential to take accurate traffic-level measurements and to protect the performance and privacy of information on a Web server. Moreover, the irrecoverable risks of visits from malicious robots that intentionally try to evade web server intrusion detection systems, covering-up their visits with fabricated fields in their http request packets, cannot be ignored. To separate both types of robots from humans in practice, analysts turn to heuristic methods or state-of-the-art soft computing approaches that have only been tuned to the specification of a kind of web server. Noting that the landscape of web robot agents is ever changing, and that behavioral patterns and characteristics vary across different web servers, both options are lacking. To overcome this challenge, this paper presents SMART, a soft computing system that simultaneously detects benign and malicious types of robot agents from web server logs and can automatically adapt to the session characteristics of a web server. The results of experiments over some access log file servers, each servicing different domains of the web, demonstrate outperformance of the proposed method on state-of-the-art ones for benign and malicious robot detection. |
|---|---|
| AbstractList | The accurate detection of web robot sessions from a web server log is essential to take accurate traffic-level measurements and to protect the performance and privacy of information on a Web server. Moreover, the irrecoverable risks of visits from malicious robots that intentionally try to evade web server intrusion detection systems, covering-up their visits with fabricated fields in their http request packets, cannot be ignored. To separate both types of robots from humans in practice, analysts turn to heuristic methods or state-of-the-art soft computing approaches that have only been tuned to the specification of a kind of web server. Noting that the landscape of web robot agents is ever changing, and that behavioral patterns and characteristics vary across different web servers, both options are lacking. To overcome this challenge, this paper presents SMART, a soft computing system that simultaneously detects benign and malicious types of robot agents from web server logs and can automatically adapt to the session characteristics of a web server. The results of experiments over some access log file servers, each servicing different domains of the web, demonstrate outperformance of the proposed method on state-of-the-art ones for benign and malicious robot detection. •We propose a method called SMART (Soft computing for MAlicious RoboT detection).•The method detects benign and malicious robots, and human visitors to a web server.•SMART selects its features on a particular web server by fuzzy rough set theory.•A graph-based clustering algorithm classifies sessions into the three agent types.•Analyses on web logs suggest state-of-the-art results to detect both robot types. The accurate detection of web robot sessions from a web server log is essential to take accurate traffic-level measurements and to protect the performance and privacy of information on a Web server. Moreover, the irrecoverable risks of visits from malicious robots that intentionally try to evade web server intrusion detection systems, covering-up their visits with fabricated fields in their http request packets, cannot be ignored. To separate both types of robots from humans in practice, analysts turn to heuristic methods or state-of-the-art soft computing approaches that have only been tuned to the specification of a kind of web server. Noting that the landscape of web robot agents is ever changing, and that behavioral patterns and characteristics vary across different web servers, both options are lacking. To overcome this challenge, this paper presents SMART, a soft computing system that simultaneously detects benign and malicious types of robot agents from web server logs and can automatically adapt to the session characteristics of a web server. The results of experiments over some access log file servers, each servicing different domains of the web, demonstrate outperformance of the proposed method on state-of-the-art ones for benign and malicious robot detection. |
| Author | Zabihimayvan, Mahdieh Rude, H. Nathan Sadeghi, Reza Doran, Derek |
| Author_xml | – sequence: 1 givenname: Mahdieh surname: Zabihimayvan fullname: Zabihimayvan, Mahdieh email: zabihimayvan.2@wright.edu, mahdieh.zabihi@gmail.com – sequence: 2 givenname: Reza surname: Sadeghi fullname: Sadeghi, Reza email: sadeghi.2@wright.edu – sequence: 3 givenname: H. Nathan surname: Rude fullname: Rude, H. Nathan email: howard.rude@wright.edu – sequence: 4 givenname: Derek surname: Doran fullname: Doran, Derek email: derek.doran@wright.edu |
| BookMark | eNp9kD1PwzAQhi1UJErhDzBZYk7wRxonEktVUUCqxAKz5TiX4qi1g-1Q8e9xVCaGTrfc8957zzWaWWcBoTtKckpo-dDnEI4qZ4SKnJQ5IcUFmtNK8KwUNZ-hOamXIiuoKK7QdQg9SYuEiDnarHBwXcTaHYYxGrvDahi8U_oTd87jBqzZWaxsiw9qb7RxY8BHaLB3jYu4hQg6Gmdv0GWn9gFu_-YCfWye3tcv2fbt-XW92maasypmomVd3bBUmXS84m3V6SVUqqigbrgSohaUNEBbwhhXdUlKoWmnaKN1A6WoKF-g-1Nu6vg1Qoiyd6O36aSkdcFLxnnKXaDqtKW9C8FDJ7WJauoZvTJ7SYmcrMleTtbkZE2SUiZrCWX_0MGbg_I_56HHEwTp9W8DXgZtwGpojU9-ZOvMOfwXeD-IFA |
| CitedBy_id | crossref_primary_10_3390_electronics14101945 crossref_primary_10_1016_j_smhl_2018_07_001 crossref_primary_10_1016_j_knosys_2020_105875 crossref_primary_10_1016_j_jnca_2020_102577 crossref_primary_10_1016_j_knosys_2021_107074 crossref_primary_10_1007_s10489_020_01754_9 crossref_primary_10_1007_s13042_023_01905_7 crossref_primary_10_1016_j_jnca_2024_103975 crossref_primary_10_1007_s42979_024_03099_3 crossref_primary_10_1016_j_ijmedinf_2019_103976 crossref_primary_10_1088_1742_6596_1679_3_032043 crossref_primary_10_1016_j_knosys_2019_105331 crossref_primary_10_1109_TFUZZ_2019_2895572 crossref_primary_10_1145_3447815 crossref_primary_10_1007_s11042_022_14258_0 crossref_primary_10_1162_neco_a_01391 crossref_primary_10_1007_s00146_022_01398_y |
| Cites_doi | 10.1016/j.comcom.2005.01.003 10.1016/j.neucom.2010.02.023 10.1016/j.eswa.2015.02.017 10.1016/j.eswa.2012.01.210 10.1007/s10618-010-0180-z 10.1080/03081079008935107 10.1016/j.comnet.2010.05.010 10.1016/j.asoc.2012.08.028 10.1016/S0165-0114(01)00032-X 10.1016/j.fss.2014.04.029 10.1016/j.comnet.2008.09.021 10.4236/jdaip.2015.31001 10.1093/gbe/evv097 10.1007/BF01001956 10.1016/j.cose.2009.05.004 10.1080/03081087.2015.1083529 10.1023/A:1013228602957 |
| ContentType | Journal Article |
| Copyright | 2017 Elsevier Ltd Copyright Elsevier BV Nov 30, 2017 |
| Copyright_xml | – notice: 2017 Elsevier Ltd – notice: Copyright Elsevier BV Nov 30, 2017 |
| DBID | AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D |
| DOI | 10.1016/j.eswa.2017.06.004 |
| DatabaseName | CrossRef Computer and Information Systems Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Computer and Information Systems Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Advanced Technologies Database with Aerospace ProQuest Computer Science Collection Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Computer and Information Systems Abstracts |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISSN | 1873-6793 |
| EndPage | 140 |
| ExternalDocumentID | 10_1016_j_eswa_2017_06_004 S0957417417304116 |
| GroupedDBID | --K --M .DC .~1 0R~ 13V 1B1 1RT 1~. 1~5 4.4 457 4G. 5GY 5VS 7-5 71M 8P~ 9JN 9JO AAAKF AABNK AACTN AAEDT AAEDW AAIAV AAIKJ AAKOC AALRI AAOAW AAQFI AARIN AAXUO AAYFN ABBOA ABFNM ABMAC ABMVD ABUCO ABYKQ ACDAQ ACGFS ACHRH ACNTT ACRLP ACZNC ADBBV ADEZE ADTZH AEBSH AECPX AEKER AENEX AFKWA AFTJW AGHFR AGJBL AGUBO AGUMN AGYEJ AHHHB AHJVU AHZHX AIALX AIEXJ AIKHN AITUG AJBFU AJOXV ALEQD ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD APLSM AXJTR BJAXD BKOJK BLXMC BNSAS CS3 DU5 EBS EFJIC EFLBG EJD EO8 EO9 EP2 EP3 F5P FDB FIRID FNPLU FYGXN G-Q GBLVA GBOLZ HAMUX IHE J1W JJJVA KOM LG9 LY1 LY7 M41 MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. PQQKQ Q38 RIG ROL RPZ SDF SDG SDP SDS SES SPC SPCBC SSB SSD SSL SST SSV SSZ T5K TN5 ~G- 29G 9DU AAAKG AAQXK AATTM AAXKI AAYWO AAYXX ABJNI ABKBG ABUFD ABWVN ABXDB ACLOT ACNNM ACRPL ACVFH ADCNI ADJOM ADMUD ADNMO AEIPS AEUPX AFJKZ AFPUW AGQPQ AIGII AIIUN AKBMS AKRWK AKYEP ANKPU APXCP ASPBG AVWKF AZFZN CITATION EFKBS FEDTE FGOYB G-2 HLZ HVGLF HZ~ R2- SBC SET SEW WUQ XPP ZMT ~HD 7SC 8FD AGCQF JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c328t-7d2f9b21010f383d8fc5e8a48e9b3a779710be1d0223a96067c1fa1bccbe67813 |
| ISICitedReferencesCount | 25 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000407183900011&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0957-4174 |
| IngestDate | Sun Sep 07 03:34:18 EDT 2025 Sat Nov 29 07:06:24 EST 2025 Tue Nov 18 22:39:57 EST 2025 Fri Feb 23 02:29:07 EST 2024 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Web Robot Detection Web crawler Fuzzy Rough Set Theory Malicious web agents Markov clustering algorithm |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c328t-7d2f9b21010f383d8fc5e8a48e9b3a779710be1d0223a96067c1fa1bccbe67813 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| PQID | 1943623338 |
| PQPubID | 2045477 |
| PageCount | 12 |
| ParticipantIDs | proquest_journals_1943623338 crossref_citationtrail_10_1016_j_eswa_2017_06_004 crossref_primary_10_1016_j_eswa_2017_06_004 elsevier_sciencedirect_doi_10_1016_j_eswa_2017_06_004 |
| PublicationCentury | 2000 |
| PublicationDate | 2017-11-30 |
| PublicationDateYYYYMMDD | 2017-11-30 |
| PublicationDate_xml | – month: 11 year: 2017 text: 2017-11-30 day: 30 |
| PublicationDecade | 2010 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | Expert systems with applications |
| PublicationYear | 2017 |
| Publisher | Elsevier Ltd Elsevier BV |
| Publisher_xml | – name: Elsevier Ltd – name: Elsevier BV |
| References | Jain, Bhandare (bib0016) 2011; 2 Radzikowska, Kerre (bib0025) 2002; 126 Article Baz. (Nov. 2013). Ruan, Fuhry, Parthasarathy (bib0026) 2013 Doran, Morillo, Gokhale (bib0009) 2013 Parthasarathy, Ruan, Satuluri (bib0022) 2011 Zabihi, Jahan, Hamidzadeh (bib0040) 2014 Heinz, Selkrig, Belousoff, Lithgow (bib0013) 2015; 7 Suchacka, Sobkow (bib0034) 2015 Dutta, Meilicke, Stuckenschmidt (bib0011) 2015 Gržinić, Mršić, Šaban (bib0012) 2015 Bomhardt, Gaul, Schmidt-Thieme (bib0005) 2005 Doran, Gokhale (bib0008) 2011; 22 Kwon, Oh, Kim, Lee, Kim, Cha (bib0018) 2012; Vol. 4 Stevanovic, Vlajic, An (bib0033) 2013; 13 Satuluri, Parthasarathy, Ucar (bib0029) 2010 Zabihi, Jahan, Hamidzadeh (bib0041) 2014; 6 Qian, Wang, Cheng, Liang, Dang (bib0024) 2015; 258 Doran, Gokhale (bib0007) 2009 Amigó, Gonzalo, Verdejo (bib0002) 2013 Lourenço, Belo (bib0021) 2006; Vol. 263 Lee, Cha, Lee, Lee (bib0019) 2009; 28 Stassopoulou, Dikaiakos (bib0031) 2009; 53 Imam Reza International University. (Aug. 2015). Sisodia, Verma, Vyas (bib0030) 2015; 3 Sun, Councill, Giles (bib0035) 2010; Vol. 1 Sadeghi, Hamidzadeh (bib0028) 2016 Liao, Xu (bib0020) 2015; 42 Rude, Doran (bib0027) 2015 Van Dongen (bib0038) 2001 . Kanji (bib0017) 2006 Dubois, Prade (bib0010) 1990; 17 Dikaiakos, Stassopoulou, Papageorgiou (bib0006) 2005; 28 Atzori, Iera, Morabito (bib0004) 2010; 54 Huang, Xu, Lu (bib0014) 2016; 64 Pawlak (bib0023) 1982; 11 Verbiest, Cornelis, Herrera (bib0039) 2013; Vol. 8170 Stevanovic, An, Vlajic (bib0032) 2012; 39 Szilágyi, Medvés, Szilágyi (bib0036) 2010; 73 Tan, Kumar (bib0037) 2002; 6 Incapsula, 2015 bot traffic report: Humans take back the web, bad bots not giving any ground. (Dec. 2015). Tan (10.1016/j.eswa.2017.06.004_bib0037) 2002; 6 Jain (10.1016/j.eswa.2017.06.004_bib0016) 2011; 2 Zabihi (10.1016/j.eswa.2017.06.004_bib0041) 2014; 6 Parthasarathy (10.1016/j.eswa.2017.06.004_bib0022) 2011 Doran (10.1016/j.eswa.2017.06.004_bib0007) 2009 Kwon (10.1016/j.eswa.2017.06.004_bib0018) 2012; Vol. 4 Lourenço (10.1016/j.eswa.2017.06.004_bib0021) 2006; Vol. 263 Dutta (10.1016/j.eswa.2017.06.004_bib0011) 2015 Kanji (10.1016/j.eswa.2017.06.004_bib0017) 2006 Pawlak (10.1016/j.eswa.2017.06.004_bib0023) 1982; 11 Radzikowska (10.1016/j.eswa.2017.06.004_bib0025) 2002; 126 10.1016/j.eswa.2017.06.004_bib0003 10.1016/j.eswa.2017.06.004_bib0001 Doran (10.1016/j.eswa.2017.06.004_bib0009) 2013 Doran (10.1016/j.eswa.2017.06.004_bib0008) 2011; 22 Gržinić (10.1016/j.eswa.2017.06.004_bib0012) 2015 Bomhardt (10.1016/j.eswa.2017.06.004_bib0005) 2005 Stassopoulou (10.1016/j.eswa.2017.06.004_bib0031) 2009; 53 Stevanovic (10.1016/j.eswa.2017.06.004_bib0033) 2013; 13 Sun (10.1016/j.eswa.2017.06.004_bib0035) 2010; Vol. 1 Liao (10.1016/j.eswa.2017.06.004_bib0020) 2015; 42 Satuluri (10.1016/j.eswa.2017.06.004_bib0029) 2010 Stevanovic (10.1016/j.eswa.2017.06.004_bib0032) 2012; 39 Van Dongen (10.1016/j.eswa.2017.06.004_bib0038) 2001 Zabihi (10.1016/j.eswa.2017.06.004_bib0040) 2014 Lee (10.1016/j.eswa.2017.06.004_bib0019) 2009; 28 Rude (10.1016/j.eswa.2017.06.004_bib0027) 2015 Dikaiakos (10.1016/j.eswa.2017.06.004_bib0006) 2005; 28 Sisodia (10.1016/j.eswa.2017.06.004_bib0030) 2015; 3 Atzori (10.1016/j.eswa.2017.06.004_bib0004) 2010; 54 Dubois (10.1016/j.eswa.2017.06.004_bib0010) 1990; 17 Heinz (10.1016/j.eswa.2017.06.004_bib0013) 2015; 7 Szilágyi (10.1016/j.eswa.2017.06.004_bib0036) 2010; 73 Ruan (10.1016/j.eswa.2017.06.004_bib0026) 2013 Suchacka (10.1016/j.eswa.2017.06.004_bib0034) 2015 Verbiest (10.1016/j.eswa.2017.06.004_bib0039) 2013; Vol. 8170 Qian (10.1016/j.eswa.2017.06.004_bib0024) 2015; 258 Sadeghi (10.1016/j.eswa.2017.06.004_bib0028) 2016 Amigó (10.1016/j.eswa.2017.06.004_bib0002) 2013 10.1016/j.eswa.2017.06.004_bib0015 Huang (10.1016/j.eswa.2017.06.004_bib0014) 2016; 64 |
| References_xml | – start-page: 643 year: 2013 end-page: 652 ident: bib0002 article-title: A general evaluation measure for document organization tasks publication-title: Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval – volume: 7 start-page: 1628 year: 2015 end-page: 1643 ident: bib0013 article-title: Evolution of the translocation and assembly module (TAM) publication-title: Genome Biology and Evolution – start-page: 1 year: 2016 end-page: 12 ident: bib0028 article-title: Automatic support vector data description publication-title: Journal of Soft Computing – volume: 73 start-page: 2332 year: 2010 end-page: 2345 ident: bib0036 article-title: A modified Markov clustering approach to unsupervised classification of protein sequences publication-title: Neurocomputing – reference: Incapsula, 2015 bot traffic report: Humans take back the web, bad bots not giving any ground. (Dec. 2015). – start-page: 79 year: 2011 end-page: 113 ident: bib0022 article-title: Community discovery in social networks: Applications, methods and emerging trends publication-title: Social network data analytics – start-page: 267 year: 2015 end-page: 277 ident: bib0011 article-title: Enriching structured knowledge with open information publication-title: Proceedings of the 24th international conference on world wide web – start-page: 247 year: 2010 end-page: 256 ident: bib0029 article-title: Markov clustering of protein interaction networks with improved balance and scalability publication-title: Proceedings of the first ACM international conference on bioinformatics and computational biology – start-page: 23 year: 2014 end-page: 28 ident: bib0040 article-title: A density based clustering approach for web robot detection publication-title: Proceedings of 2014 4th international e-conference on computer and knowledge engineering – volume: 53 start-page: 265 year: 2009 end-page: 278 ident: bib0031 article-title: Web robot detection: A probabilistic reasoning approach publication-title: Computer Networks – volume: 39 start-page: 8707 year: 2012 end-page: 8717 ident: bib0032 article-title: Feature evaluation for web crawler detection with data mining techniques publication-title: Expert Systems with Applications – volume: Vol. 263 start-page: 265 year: 2006 end-page: 272 ident: bib0021 article-title: Catching web crawlers in the act publication-title: Proceedings of the 6th international conference on web engineering – volume: 11 start-page: 341 year: 1982 end-page: 356 ident: bib0023 article-title: Rough sets publication-title: International Journal of Computer & Information Sciences – volume: 258 start-page: 61 year: 2015 end-page: 78 ident: bib0024 article-title: Fuzzy-rough feature selection accelerator publication-title: Fuzzy Sets and Systems – start-page: 995 year: 2015 end-page: 1000 ident: bib0027 article-title: Request type prediction for web robot and internet of things traffic publication-title: Proceedings of 2015 IEEE 14th international conference on machine learning and applications – reference: Imam Reza International University. (Aug. 2015). – volume: 64 start-page: 1362 year: 2016 end-page: 1378 ident: bib0014 article-title: Some new inequalities for the Hadamard product of a nonsingular M-matrix and its inverse publication-title: Linear and Multilinear Algebra – start-page: 97 year: 2009 end-page: 102 ident: bib0007 article-title: Classifying web robots by K-means clustering publication-title: Proceedings of the international conference on software engineering and knowledge engineering – volume: 28 start-page: 795 year: 2009 end-page: 802 ident: bib0019 article-title: Classification of web robots: An empirical study based on over one billion requests publication-title: Computers & Security – volume: Vol. 4 start-page: 43 year: 2012 end-page: 48 ident: bib0018 article-title: Web robot detection based on monotonous behavior publication-title: Proceedings of the information science and industrial applications – volume: 13 start-page: 698 year: 2013 end-page: 708 ident: bib0033 article-title: Detection of malicious and non-malicious website visitors using unsupervised neural network learning publication-title: Applied Soft Computing – reference: Article Baz. (Nov. 2013). – volume: 3 start-page: 1 year: 2015 end-page: 10 ident: bib0030 article-title: Agglomerative approach for identification and elimination of web robots from web server logs to extract knowledge about actual visitors publication-title: Journal of Data Analysis and Information Processing – volume: 2 start-page: 45 year: 2011 end-page: 50 ident: bib0016 article-title: Min max normalization based data perturbation method for privacy protection publication-title: International Journal of Computer & Communication Technology – volume: 126 start-page: 137 year: 2002 end-page: 155 ident: bib0025 article-title: A comparative study of fuzzy rough sets publication-title: Fuzzy Sets and Systems – volume: 17 start-page: 191 year: 1990 end-page: 209 ident: bib0010 article-title: Rough fuzzy sets and fuzzy rough sets publication-title: International Journal of General System – volume: 6 start-page: 77 year: 2014 end-page: 89 ident: bib0041 article-title: A density based clustering approach to distinguish between web robot and human requests to a web server publication-title: The ISC International Journal of Information Security – volume: 54 start-page: 2787 year: 2010 end-page: 2805 ident: bib0004 article-title: The internet of things: A survey publication-title: Computer Networks – reference: . – start-page: 559 year: 2015 end-page: 568 ident: bib0012 article-title: Lino – An intelligent system for detecting malicious web-robots publication-title: Proceedings of Asian conference on intelligent information and database systems – year: 2006 ident: bib0017 article-title: 100 statistical tests – volume: Vol. 1 start-page: 668 year: 2010 end-page: 675 ident: bib0035 article-title: The ethicality of web crawlers publication-title: Proceedings of 2010 IEEE/WIC/ACM international conference on web intelligence and intelligent agent technology – volume: Vol. 8170 start-page: 180 year: 2013 end-page: 190 ident: bib0039 article-title: OWA-FRPS: A prototype selection method based on ordered weighted average fuzzy rough set theory publication-title: Proceedings of international workshop on rough sets, fuzzy sets, data mining, and granular computing – start-page: 1089 year: 2013 end-page: 1098 ident: bib0026 article-title: Efficient community detection in large networks using content and links publication-title: Proceedings of the 22nd international conference on world wide web – volume: 28 start-page: 880 year: 2005 end-page: 897 ident: bib0006 article-title: An investigation of web crawler behavior: Characterization and metrics publication-title: Computer Communications – volume: 22 start-page: 183 year: 2011 end-page: 210 ident: bib0008 article-title: Web robot detection techniques: Overview and limitations publication-title: Data Mining and Knowledge Discovery – volume: 6 start-page: 9 year: 2002 end-page: 35 ident: bib0037 article-title: Discovery of web robot sessions based on their navigational patterns publication-title: Data Mining and Knowledge Discovery – start-page: 365 year: 2015 end-page: 370 ident: bib0034 article-title: Detection of Internet robots using a Bayesian approach publication-title: Proceedings of 2015 IEEE 2nd international conference on cybernetics – year: 2001 ident: bib0038 article-title: Graph clustering by flow simulation – start-page: 1374 year: 2013 end-page: 1380 ident: bib0009 article-title: A comparison of web robot and human requests publication-title: Proceedings of the 2013 IEEE/ACM international conference on advances in social networks analysis and mining – start-page: 113 year: 2005 end-page: 124 ident: bib0005 article-title: Web robot detection-preprocessing web log files for robot detection publication-title: New developments in classification and data analysis – volume: 42 start-page: 5328 year: 2015 end-page: 5336 ident: bib0020 article-title: Approaches to manage hesitant fuzzy linguistic information based on the cosine distance and similarity measures for HFLTSs and their application in qualitative decision making publication-title: Expert Systems with Applications – volume: 28 start-page: 880 issue: 8 year: 2005 ident: 10.1016/j.eswa.2017.06.004_bib0006 article-title: An investigation of web crawler behavior: Characterization and metrics publication-title: Computer Communications doi: 10.1016/j.comcom.2005.01.003 – start-page: 995 year: 2015 ident: 10.1016/j.eswa.2017.06.004_bib0027 article-title: Request type prediction for web robot and internet of things traffic – volume: Vol. 1 start-page: 668 year: 2010 ident: 10.1016/j.eswa.2017.06.004_bib0035 article-title: The ethicality of web crawlers – start-page: 1089 year: 2013 ident: 10.1016/j.eswa.2017.06.004_bib0026 article-title: Efficient community detection in large networks using content and links – start-page: 1374 year: 2013 ident: 10.1016/j.eswa.2017.06.004_bib0009 article-title: A comparison of web robot and human requests – volume: 73 start-page: 2332 issue: 13 year: 2010 ident: 10.1016/j.eswa.2017.06.004_bib0036 article-title: A modified Markov clustering approach to unsupervised classification of protein sequences publication-title: Neurocomputing doi: 10.1016/j.neucom.2010.02.023 – start-page: 247 year: 2010 ident: 10.1016/j.eswa.2017.06.004_bib0029 article-title: Markov clustering of protein interaction networks with improved balance and scalability – volume: 42 start-page: 5328 issue: 12 year: 2015 ident: 10.1016/j.eswa.2017.06.004_bib0020 article-title: Approaches to manage hesitant fuzzy linguistic information based on the cosine distance and similarity measures for HFLTSs and their application in qualitative decision making publication-title: Expert Systems with Applications doi: 10.1016/j.eswa.2015.02.017 – volume: 2 start-page: 45 issue: 8 year: 2011 ident: 10.1016/j.eswa.2017.06.004_bib0016 article-title: Min max normalization based data perturbation method for privacy protection publication-title: International Journal of Computer & Communication Technology – volume: 39 start-page: 8707 issue: 10 year: 2012 ident: 10.1016/j.eswa.2017.06.004_bib0032 article-title: Feature evaluation for web crawler detection with data mining techniques publication-title: Expert Systems with Applications doi: 10.1016/j.eswa.2012.01.210 – start-page: 23 year: 2014 ident: 10.1016/j.eswa.2017.06.004_bib0040 article-title: A density based clustering approach for web robot detection – start-page: 643 year: 2013 ident: 10.1016/j.eswa.2017.06.004_bib0002 article-title: A general evaluation measure for document organization tasks – volume: 22 start-page: 183 issue: 1–2 year: 2011 ident: 10.1016/j.eswa.2017.06.004_bib0008 article-title: Web robot detection techniques: Overview and limitations publication-title: Data Mining and Knowledge Discovery doi: 10.1007/s10618-010-0180-z – volume: 17 start-page: 191 issue: 2–3 year: 1990 ident: 10.1016/j.eswa.2017.06.004_bib0010 article-title: Rough fuzzy sets and fuzzy rough sets publication-title: International Journal of General System doi: 10.1080/03081079008935107 – start-page: 267 year: 2015 ident: 10.1016/j.eswa.2017.06.004_bib0011 article-title: Enriching structured knowledge with open information – start-page: 1 year: 2016 ident: 10.1016/j.eswa.2017.06.004_bib0028 article-title: Automatic support vector data description publication-title: Journal of Soft Computing – volume: Vol. 4 start-page: 43 year: 2012 ident: 10.1016/j.eswa.2017.06.004_bib0018 article-title: Web robot detection based on monotonous behavior – start-page: 79 year: 2011 ident: 10.1016/j.eswa.2017.06.004_bib0022 article-title: Community discovery in social networks: Applications, methods and emerging trends – volume: 54 start-page: 2787 issue: 15 year: 2010 ident: 10.1016/j.eswa.2017.06.004_bib0004 article-title: The internet of things: A survey publication-title: Computer Networks doi: 10.1016/j.comnet.2010.05.010 – start-page: 559 year: 2015 ident: 10.1016/j.eswa.2017.06.004_bib0012 article-title: Lino – An intelligent system for detecting malicious web-robots – volume: Vol. 8170 start-page: 180 year: 2013 ident: 10.1016/j.eswa.2017.06.004_bib0039 article-title: OWA-FRPS: A prototype selection method based on ordered weighted average fuzzy rough set theory – volume: 13 start-page: 698 issue: 1 year: 2013 ident: 10.1016/j.eswa.2017.06.004_bib0033 article-title: Detection of malicious and non-malicious website visitors using unsupervised neural network learning publication-title: Applied Soft Computing doi: 10.1016/j.asoc.2012.08.028 – volume: 126 start-page: 137 issue: 2 year: 2002 ident: 10.1016/j.eswa.2017.06.004_bib0025 article-title: A comparative study of fuzzy rough sets publication-title: Fuzzy Sets and Systems doi: 10.1016/S0165-0114(01)00032-X – volume: 258 start-page: 61 year: 2015 ident: 10.1016/j.eswa.2017.06.004_bib0024 article-title: Fuzzy-rough feature selection accelerator publication-title: Fuzzy Sets and Systems doi: 10.1016/j.fss.2014.04.029 – volume: 53 start-page: 265 issue: 3 year: 2009 ident: 10.1016/j.eswa.2017.06.004_bib0031 article-title: Web robot detection: A probabilistic reasoning approach publication-title: Computer Networks doi: 10.1016/j.comnet.2008.09.021 – start-page: 113 year: 2005 ident: 10.1016/j.eswa.2017.06.004_bib0005 article-title: Web robot detection-preprocessing web log files for robot detection – volume: Vol. 263 start-page: 265 year: 2006 ident: 10.1016/j.eswa.2017.06.004_bib0021 article-title: Catching web crawlers in the act – volume: 3 start-page: 1 issue: 2 year: 2015 ident: 10.1016/j.eswa.2017.06.004_bib0030 article-title: Agglomerative approach for identification and elimination of web robots from web server logs to extract knowledge about actual visitors publication-title: Journal of Data Analysis and Information Processing doi: 10.4236/jdaip.2015.31001 – volume: 7 start-page: 1628 issue: 6 year: 2015 ident: 10.1016/j.eswa.2017.06.004_bib0013 article-title: Evolution of the translocation and assembly module (TAM) publication-title: Genome Biology and Evolution doi: 10.1093/gbe/evv097 – volume: 11 start-page: 341 issue: 5 year: 1982 ident: 10.1016/j.eswa.2017.06.004_bib0023 article-title: Rough sets publication-title: International Journal of Computer & Information Sciences doi: 10.1007/BF01001956 – volume: 28 start-page: 795 issue: 8 year: 2009 ident: 10.1016/j.eswa.2017.06.004_bib0019 article-title: Classification of web robots: An empirical study based on over one billion requests publication-title: Computers & Security doi: 10.1016/j.cose.2009.05.004 – year: 2001 ident: 10.1016/j.eswa.2017.06.004_bib0038 – ident: 10.1016/j.eswa.2017.06.004_bib0015 – volume: 64 start-page: 1362 issue: 7 year: 2016 ident: 10.1016/j.eswa.2017.06.004_bib0014 article-title: Some new inequalities for the Hadamard product of a nonsingular M-matrix and its inverse publication-title: Linear and Multilinear Algebra doi: 10.1080/03081087.2015.1083529 – volume: 6 start-page: 77 issue: 1 year: 2014 ident: 10.1016/j.eswa.2017.06.004_bib0041 article-title: A density based clustering approach to distinguish between web robot and human requests to a web server publication-title: The ISC International Journal of Information Security – year: 2006 ident: 10.1016/j.eswa.2017.06.004_bib0017 – start-page: 365 year: 2015 ident: 10.1016/j.eswa.2017.06.004_bib0034 article-title: Detection of Internet robots using a Bayesian approach – volume: 6 start-page: 9 issue: 1 year: 2002 ident: 10.1016/j.eswa.2017.06.004_bib0037 article-title: Discovery of web robot sessions based on their navigational patterns publication-title: Data Mining and Knowledge Discovery doi: 10.1023/A:1013228602957 – ident: 10.1016/j.eswa.2017.06.004_bib0003 – ident: 10.1016/j.eswa.2017.06.004_bib0001 – start-page: 97 year: 2009 ident: 10.1016/j.eswa.2017.06.004_bib0007 article-title: Classifying web robots by K-means clustering |
| SSID | ssj0017007 |
| Score | 2.411827 |
| Snippet | •We propose a method called SMART (Soft computing for MAlicious RoboT detection).•The method detects benign and malicious robots, and human visitors to a web... The accurate detection of web robot sessions from a web server log is essential to take accurate traffic-level measurements and to protect the performance and... |
| SourceID | proquest crossref elsevier |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 129 |
| SubjectTerms | Automation Clustering Cybersecurity File servers Fuzzy Rough Set Theory Heuristic methods Intrusion detection systems Malicious web agents Markov clustering algorithm Privacy Robots Set theory Soft computing State of the art Traffic information Web crawler Web Robot Detection |
| Title | A soft computing approach for benign and malicious web robot detection |
| URI | https://dx.doi.org/10.1016/j.eswa.2017.06.004 https://www.proquest.com/docview/1943623338 |
| Volume | 87 |
| WOSCitedRecordID | wos000407183900011&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1873-6793 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0017007 issn: 0957-4174 databaseCode: AIEXJ dateStart: 19950101 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1LbxMxELZCyoELb9SWgnzgFm20Xm9i-xhBq4KgQrRIua38WiVVu6mSTWn5AfxuxutHoIgKkLisolW8cTyfZ2bHM98g9EpqofnIkkwqIjLwiG2mwK3PTKmptHVRUtux679nR0d8OhUfe71vsRbm8ow1Db-6Ehf_VdRwD4TtSmf_QtzpoXADPoPQ4Qpih-sfCX4yWIFq7XLF111Oc6QN95mZtnGdOF24_BxccN2lwLp6xeVCLdqBsW2Xm9X8FLF3dMhtIH2O5XA_HHyn4LNU89n8XF6Hw6UPcmbmNsWbj6WxvocwCPVrMgef1p7593DoMqdnG7S-AXQGnbgMlMIhPAEmL9AixphZrJvZJCn54CPLSuL78wytV72c0WzMfL_EqJuDMfbKlYTYiLfTxNM8_WICfDTidGhXXxyvFGEdP2tebgxeSkM8dvNw0yCg50pCxnfQVsFGgvfR1uTt_vRdOo9iuS-8j_MO5Vc-U_DmL_3Oxblh7DsP5uQhuh9ePfDEQ-YR6tnmMXoQ23rgoOWfoIMJdgjCCUE4IggDgrBHEAYE4YQgDAjCHYJwQtBT9Plg_-T1YRbabWSaFrzNmClqoQr4U3lNOTW81iPLZcmtUFQyJsAZVZYY8PqodC--TJNaEqW1suDyEPoM9ZtFY7cRLgxVQpvc5DlseqUk4bmEwdbymqvReAeRuEKVDlz0riXKWRWTDk8rt6qVW9Wqy7wsd9AgjbnwTCy3fnsUF74KvqT3ESvAya3j9qKUqrCpVxURJfh5lFK--4-PfY7ubTbHHuq3y7V9ge7qy3a-Wr4MaPsOtB6k4w |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=A+soft+computing+approach+for+benign+and+malicious+web+robot+detection&rft.jtitle=Expert+systems+with+applications&rft.au=Zabihimayvan%2C+Mahdieh&rft.au=Sadeghi%2C+Reza&rft.au=Rude%2C+H.+Nathan&rft.au=Doran%2C+Derek&rft.date=2017-11-30&rft.pub=Elsevier+Ltd&rft.issn=0957-4174&rft.eissn=1873-6793&rft.volume=87&rft.spage=129&rft.epage=140&rft_id=info:doi/10.1016%2Fj.eswa.2017.06.004&rft.externalDocID=S0957417417304116 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0957-4174&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0957-4174&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0957-4174&client=summon |