Critical Properties of a Simple Spin Glass Model