An associated perception import method for big data - Huaiyu GONG; Jinsong XU; Pan WANG (Dianxin kexue)
Against the background of the existing database to the large data migration，Apache introduced the Sqoop as the main tool for the relational database to the big data migration.Sqoop simply cut the data table and randomly store it on diffe rent nodes.Being aimed at the problem of low efficiency of the query of the relationship between the Hadoop，a method of data importing and preprocessing was designed.To reduce the network transmission delay and improve the performance of the system，the high correlation degree was kept in the adjacent nodes.The contrast experiment shows that the performance of the data query can be improved greatly by the same or adjacent nodes.