The Evolution of Passive Galaxies since z=1: Major Mergers vs Secular Processes