On Regression-Tree-Based Synthetic Data Methods for Business Data

This paper concerns the use of synthetic data for protecting the confidentiality of business data during statistical analysis. Synthetic data sets are traditionally constructed by replacing sensitive values in a confidential data set with draws from statistical models estimated on the confidential data set. Unfortunately, the process of generating effective statistical models can be a difficult and labour-intensive task. Recently, it has been proposed to use easily-implemented methods from machine learning instead of statistical model estimation in the data synthesis task. J. Drechsler and J.P. Reiter (2011) have conducted an evaluation of four such methods, and have found that regression trees could give rise to synthetic data sets which provide reliable analysis results as well as low disclosure risks. Their conclusion was based on simulations using a subset of the 2002 Uganda census public use file. It is an interesting question whether the same conclusion applies to other types of data with different characteristics, for example business data which have quite different characteristics from population census and survey data. In particular, business data generally have few variables that are mostly categorical, and often have highly skewed distributions with outliers. In this paper we investigate the applicability of regression-tree-based methods for constructing synthetic business data. We give a detailed example comparing exploratory data analysis and linear regression results under two variants of a regression-tree-based synthetic data approach. We also include an evaluation of the analysis results with respect to the results of analysis of the original data. We further investigate the impact of different stopping criteria on performance. While it is certainly true that any method designed to protect confidentiality introduces error, and may indeed give misleading conclusions, our analysis of the results for synthesisers based on CART models has provided some evidence that this error is not random but is due to the particular characteristics of business data. We conclude that more careful analysis needs to be done in applying these methods and end users certainly need aware of possible discrepancies.