Research on Compression Storage of Massive Agricultural Data Based on Cloud Environment

With the development of information technology, agriculture data show large amount of data, distributed, heterogeneous characteristics. It is difficult to access and management with the massive data are continuously increasing which affect the large-scale use of agricultural information data. In this paper, the method of compression algorithm is proposed which based on real-time and time space correlation characteristics. All data is divided into several categories by Huffman compression algorithm combines parallel processing cloud platform. Then, the massive agricultural data is compressed and reducing the data storage. Exprienment result show that cloud storage platform with dynamic scalability. Under the same experimental data, the method of this paper has higher compress ratio, and compression consuming less when a larger amount data, compared with the Huffman compress and dictionary-based data compression algorithm.

[1]  Abraham Lempel,et al.  A universal algorithm for sequential data compression , 1977, IEEE Trans. Inf. Theory.

[2]  K. J. Ray Liu,et al.  Anti-forensics of digital image compression , 2011, IEEE Transactions on Information Forensics and Security.

[3]  Sonali Aggarwal,et al.  Characterization of Hadoop Jobs Using Unsupervised Learning , 2010, 2010 IEEE Second International Conference on Cloud Computing Technology and Science.

[4]  Jianfei Cai,et al.  Content-Based Image Compression for Arbitrary-Resolution Display Devices , 2011, 2011 IEEE International Conference on Communications (ICC).

[5]  Qiao Wei-min Design of Heuristic Parallel Computation Model Based on PVM , 2005 .

[6]  Xiaoyan Sun,et al.  Cloud-Based Image Coding for Mobile Devices—Toward Thousands to One Compression , 2013, IEEE Transactions on Multimedia.