Mode conversion and harmonic generation at the upper hybrid layer in toroidal plasmas