A model for AGN variability on multiple time-scales