AI and data-driven infrastructures for workflow automation and integration in advanced research and industrial applications