基于爬虫的招聘信息管理及可视化分析系统

北京理工大学珠海学院2020届本科生毕业设计Recruitment information management and visual analysis systembased on crawlerAbstractWith the development of Internet information technology,online recruitment has becomea bridge between enterprises and job seekers.But with the popularity of Internet recruitment,recruitment information on the site show explosive growth,companies recruit less than peopleat the same time,a lot of people also can not find work,appeared the phenomenon ofstructural unemployment,how to dig up information from a lot of hiring data potential value,reflect the enterprise to talented person's demand,find effective information,become acandidate in the job is more and more concern.Through the recruitment website of 5ljob,this system will adopt the keyword-baseddirectional crawling method,crawl the recruitment data of several popular positions,anddownload the sample data to the database MongoDB for storage.In order to ensure theaccuracy of the data,we must also use the third-party library of python to clean and clean thedata,such as deduplication,normalization,etc.Finally,we use pyecharts to visualize the datain charts,and display the charts in the recruitment data analysis platform in a beautiful andnovel way.In the exploratory analysis of the data,it is found that there is a certain linear relationshipbetween education and salary,work experience and salary.Therefore,the recruitment data isfinally used as sample data in this paper to train a linear model.Key Words:Web crawler;data analysis;visual presentation