Superconducting qubits often show signatures of coherent coupling to microscopic two-level fluctuators (TLFs), which manifest themselves as avoided level crossings in spectroscopic data. In this work we study a phase qubit, in which we induce Rabi oscillations by resonant microwave driving. When the qubit is tuned close to the resonance with an individual TLF and the Rabi driving is strong enough (Rabi frequency of order of the qubit-TLF coupling), interesting four-level dynamics are observed. The experimental data show a clear asymmetry between biasing the qubit above or below the fluctuator's level splitting. Theoretical analysis indicates that this asymmetry is due to an effective coupling of the TLF to the external microwave field induced by the higher qubit levels.