Navigating the development challenges in creating complex data systems