Efficient Processing of Voluminous EDI Documents

Business data is frequently exchanged between heterogeneous information systems using standard EDI formats like EDIFACT, X12 and in the future also XML/EDI. For inhouse use, data represented in these formats must be converted to inhouse data formats by message converters. With the growing usage of EDI, high volumes of data, like financial transactions, must be converted and processed efficiently and reliably. Considering the complex hierarchical structure of EDI messages, message conversion is a complex data transformation problem. For efficient processing, the different conversion steps have to be executed in an optimized manner exploiting the available, typically distributed, processing architecture. In addition, the processing has to take into account different optimization goals, like maximizing throughput, minimizing delays and keeping deadlines. We approach message conversion as a data management problem. First we show that the nested relational data model is adequate for representing the message structure and the transformation operations. Based on this we develop a cost model. This serves as input to an optimization strategy for the conversion processing that uses efficient scheduling strategies. We present a novel scheduling strategy and give simulation results that show that they outperform alternative strategies for typical workloads of EDI converters.