Concave Utility Reinforcement Learning: the Mean-field Game viewpoint