A Practical, Collaborative Approach for Modeling Big Data Analytics Application Requirements

Data analytics application development introduces many challenges including: new roles not in traditional software engineering practices – e.g. data scientists and data engineers; use of sophisticated machine learning (ML) model-based approaches; uncertainty inherent in the models; interfacing with models to fulfill software functionalities; deploying models at scale and rapid evolution of business goals and data sources. We describe our Big Data Analytics Modeling Languages (BiDaML) toolset to bring all stakeholders around one tool to specify, model and document big data applications. We report on our experience applying BiDaML to three real-world large-scale applications. Our approach successfully supports complex data analytics application development in industrial settings.