The System Architecture for the Basic Information of Science and Technology Experts Based on Distributed Storage and Web Mining

In order to build an efficient basic information system of science and technology experts based on Web mining, a novel system architecture for application is proposed in this paper. The proposed system architecture integrates spider module, local distributed storage and Mongo-DB. The basic experts i...

Full description

Saved in:
Bibliographic Details
Published in:2012 International Conference on Computer Science and Service System pp. 527 - 530
Main Authors: Quanyin Zhu, Pei Zhou
Format: Conference Proceeding
Language:English
Published: IEEE 01.08.2012
Subjects:
Computer architecture
Data processing
distributed storage
Educational institutions
Information management
Information retrieval
Mongo-DB
science and technology experts
sharing cluster architecture
spider module
systen architecture
Web mining
ISBN:9781467307215, 1467307211
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In order to build an efficient basic information system of science and technology experts based on Web mining, a novel system architecture for application is proposed in this paper. The proposed system architecture integrates spider module, local distributed storage and Mongo-DB. The basic experts information of science and technology appeared in the Websites are synthesized as two format and using two strategies to deal with it respectively. The normalized texts which extracted from Web page by URLs are suggested. The extracted results include the name, sex, birth, hometown and professional title of science and technology experts respectively. The data stream flow, the information management model for the users and science and technology experts, the target website URLs and URLs management model, and data processing module are introduced in detailed. The synchronization of multiple databases and replica sets architecture for sharing cluster architecture is proposed in application system. Experiments show that the application system obtains a very high efficiency. The results show as by proposed system architecture can satisfy the application requirements for the customer.
ISBN:9781467307215
1467307211
DOI:10.1109/CSSS.2012.138