A Model of the 2-35 micron Point Source Infrared Sky