论文信息 - Data Profiling Technology of Data Governance Regarding Big Data: Review and Rethinking

Data Profiling Technology of Data Governance Regarding Big Data: Review and Rethinking

Data profiling technology is very valuable for data governance and data quality control because people need it to verify and review the quality of structured, semi-structured, and unstructured data. In this paper, we first review relevant works and discuss their definitions of data profiling. Second, we offer a new definition and propose new classifications for data profiling tasks. Third, the paper presents several free and commercial profiling tools. Fourth, authors offer a new data quality metrics and data quality score calculation. Finally, authors discuss a data profiling tool framework for big data.

[1] Dan Suciu,et al. Adding Structure to Unstructured Data , 1997, ICDT.

[2] Peter Buneman,et al. Semistructured data , 1997, PODS.

[3] Richard Y. Wang,et al. A product perspective on total data quality management , 1998, CACM.

[4] Daniel L. Moody,et al. Metrics for Evaluating the Quality of Entity Relationship Models , 1998, ER.

[5] Giri Kumar Tayi,et al. Enhancing data quality in data warehouse environments , 1999, CACM.

[6] Erhard Rahm,et al. Data Cleaning: Problems and Current Approaches , 2000, IEEE Data Eng. Bull..

[7] Mario Piattini,et al. Towards Data Warehouse Quality Metrics , 2001, DMDW.

[8] Richard Y. Wang,et al. Data quality assessment , 2002, CACM.

[9] Richard Y. Wang,et al. Journey to Data Quality , 2006 .

[10] Carol V. Brown,et al. Designing data governance , 2010, CACM.

[11] Paul Zikopoulos,et al. Understanding Big Data: Analytics for Enterprise Class Hadoop and Streaming Data , 2011 .