Dynamic resource allocation for large-scale streaming data