On the Evolution of Harming and Recognition in Finite Panmictic and Infinite Structured Populations