Primordial black holes from passive density fluctuations