An extension of the interacting boson approximation model is proposed by allowing for two- and four-quasiparticle excitations out of the boson space. The formation of band patterns based on two- and four-quasiparticle states is investigated in the SU(3) limit of the model. For hole-type (particle-type) fermions coupled to the SU(3) prolate (oblate) core, it is shown that the algebraic {ital K}-representation basis, which is the analog of the strong-coupling basis of the geometrical model, provides an appropriate description of the low-lying two-quasiparticle bands. In the case of particle-type (hole-type) fermions coupled to the SU(3) prolate (oblate) core, a new algebraic decoupling basis is derived that is equivalent in the geometrical limit to Stephens' rotation-aligned basis. Comparing the wave functions that are obtained by diagonalization of the model Hamiltonian to the decoupling basis, several low-lying two-quasiparticle bands are identified. The effects of an interaction that conserves only the total nucleon number, mixing states with different number of fermions, are investigated in both the strong-coupling and decoupling limits. All calculations are performed for an SU(3) boson core and the {ital h}11/2 fermion orbital.