Big data challenges and opportunities in financial stability monitoring

The exponential growth of machine-readable data to record and communicate activities throughout the financial system has significant implications for macroprudential monitoring. The central challenge is the scalability of institutions and processes in the face of the variety, volume, and rate of the “big data” deluge. This deluge also provides opportunities in the form of new, rapidly available, valuable streams of information with finer levels of detail and granularity. A difference in scale can become a difference in kind, as legacy processes are overwhelmed and innovative responses emerge.Despite the importance and ubiquity of data in financial markets, processes to manage this crucial resource must adapt. This need applies especially to financial stability or macroprudential analysis, where information must be assembled, cleaned, and integrated from regulators around the world to build a coherent view of the financial system to support policy decisions. We consider the key challenges for systemic risk supervision from the expanding volume and diversity of financial data. The discussion is organised around five broad supervisory tasks in the typical life cycle of supervisory data.

[1]  Richard Stanton,et al.  Handbook of Financial Data and Risk Information II: US residential-mortgage transfer systems: a data-management crisis , 2014 .

[2]  Jason W. Osborne,et al.  Best Practices in Data Cleaning: A Complete Guide to Everything You Need to Do Before and After Collecting Your Data , 2012 .

[3]  Jonathan Katz,et al.  Cryptography and the Economics of Supervisory Information: Balancing Transparency and Confidentiality , 2013 .

[4]  Francis X. Diebold,et al.  On the Origin(s) and Development of the Term 'Big Data' , 2012 .

[5]  Joann J. Ordille,et al.  Data integration: the teenage years , 2006, VLDB.

[6]  Jennifer McNabb,et al.  Uses of Administrative Data at the Social Security Administration , 2009, Social security bulletin.

[7]  J. Manyika.  Big data: The next frontier for innovation, competition, and productivity , 2011 .

[8]  Erhard Rahm,et al.  A survey of approaches to automatic schema matching , 2001, The VLDB Journal.

[9]  Louiqa Raschid,et al.  Exploiting Lists of Names for Named Entity Identification of Financial Institutions from Unstructured Documents , 2016, ArXiv.

[10]  J. Pattison.  Data-driven regulation and financial reform: one perspective from industry on the financial crisis , 2014 .

[11]  Victoria L. Lemieux,et al.  The Application of Visual Analytics to Financial Stability Monitoring , 2015 .

[12]  S. Claessens,et al.  Systemic Risks in Global Banking: What Available Data Can Tell Us and What More Data are Needed? , 2011, SSRN Electronic Journal.

[13]  Victoria Stodden,et al.  Breakdown Point of Model Selection When the Number of Variables Exceeds the Number of Observations , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[14]  Han Liu,et al.  Challenges of Big Data Analysis. , 2013, National science review.

[15]  Tobias Preis,et al.  Quantifying the Relationship Between Financial News and the Stock Market , 2013, Scientific Reports.

[16]  Maureen O'Hara,et al.  High frequency market microstructure. , 2015 .

[17]  Benjamin Munyan.  Regulatory Arbitrage in Repo Markets , 2015 .

[18]  David Bholat,et al.  Big Data and central banks , 2015, Big Data & Society.

[19]  Jignesh M. Patel,et al.  Big data and its technical challenges , 2014, CACM.

[20]  M. Cipriani,et al.  The U.S. Bilateral Repo Market: Lessons from a New Survey , 2016 .

[21]  J. Jacod,et al.  High-Frequency Financial Econometrics , 2014 .

[22]  Laura M. Haas,et al.  Information integration in the enterprise , 2008, CACM.

[23]  X. Sala-i-Martin,et al.  I Just Ran Two Million Regressions , 1997 .

[24]  H Eugene Stanley,et al.  Quantifying the semantics of search behavior before stock market moves , 2014, Proceedings of the National Academy of Sciences.

[25]  Hal R. Varian,et al.  Big Data: New Tricks for Econometrics , 2014 .

[26]  Jonathan Levin,et al.  Economics in the age of big data , 2014, Science.

[27]  Michael LaCour-Little,et al.  Does Regulatory Capital Arbitrage, Reputation, or Asymmetric Information Drive Securitization? , 2005 .

[28]  Divesh Srivastava,et al.  Big Data Integration , 2015, Synthesis Lectures on Data Management.

[29]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[30]  E. Horvitz.  From Data to Predictions and Decisions: Enabling Evidence-Based Healthcare , 2010 .

[31]  Joel Hasbrouck,et al.  Low-latency trading $ , 2013 .

[32]  A. Lo,et al.  Moore's Law vs. Murphy's Law: Algorithmic Trading and Its Discontents , 2013 .

[33]  Louiqa Raschid,et al.  Knowledge Representation and Information Management for Financial Risk Management: Report of a Workshop , 2010 .

[34]  Richard Y. Wang,et al.  Data quality assessment , 2002, CACM.

[35]  Natalya F. Noy,et al.  Semantic integration: a survey of ontology-based approaches , 2004, SGMD.

[36]  Michael A. Lombardi,et al.  Legal and Technical Measurement Requirements for Time and Frequency , 2006 .

[37]  Yong Zhao,et al.  Grid middleware services for virtual data discovery, composition, and integration , 2004, MGC '04.

[38]  Alex Nazaruk,et al.  Big data in capital markets , 2013, SIGMOD '13.

[39]  M. Flood.  Embracing change: financial informatics and risk analytics , 2006 .

[40]  Matthias Schunter,et al.  Fair Exchange , 2011, Encyclopedia of Cryptography and Security.

[41]  Chanley Howell,et al.  Privacy and Big Data , 2014 .

[42]  Vasant Dhar,et al.  Data science and prediction , 2012, CACM.

[43]  Mark D. Flood,et al.  Monitoring Financial Stability in a Complex World , 2012 .

[44]  H. Peyton Young,et al.  Gaming Performance Fees By Portfolio Managers , 2010 .

[45]  James Cheney,et al.  Provenance in databases , 2009, SIGMOD '07.

[46]  Rajasekar Krishnamurthy,et al.  Extracting, Linking and Integrating Data from Public Sources: A Financial Case Study , 2015, IEEE Data Eng. Bull..

[47]  Michael Bromwich,et al.  Following the Money: The Enron Failure and the State of Corporate Disclosure , 2003 .

[48]  R. Kitchin.  The opportunities, challenges and risks of big data for official statistics , 2015 .

[49]  A. Karr.  Exploratory Data Mining and Data Cleaning , 2006 .

[50]  Erhard Rahm,et al.  Data Cleaning: Problems and Current Approaches , 2000, IEEE Data Eng. Bull..

[51]  Pedro M. Domingos.  A few useful things to know about machine learning , 2012, Commun. ACM.

[52]  Tony Hey,et al.  The Fourth Paradigm: Data-Intensive Scientific Discovery , 2009 .