Teracomputing for mask data preparation
暂无分享,去创建一个
Current and future Mask Data Preparation continues to see larger file sizes and longer processing times. Distributed processing using multiple processors provides more compute power, but file Input/Output time remains a significant portion of MDP processing. Data compression, fast disk storage, and fast network hardware are shown to provide some benefit, but are not sufficient for unlimited scalability. Most MDP file formats store pattern data in a single disk file, which creates a performance obstacle in the process flow. Dividing data into multiple files is shown to improve writing speed, and to facilitate pipelined execution of multistage process flows. The advantages, disadvantages, and system management of distributed files in the terabyte era are described.