Testing the Paradigm that Ultraluminous X-Ray Sources as a Class Represent Accreting Intermediate-Mass Black Holes