Planetary systems in a star cluster II: intermediate-mass black holes and planetary systems