On the Singular Local Limit for Conservation Laws with Nonlocal Fluxes