Organic-inorganic hybrid sol-gel materials have been widely used for the fabrication of optoelectronic devices due to advantages such as low cost, low loss, low temperature fabrication, ease of processing, good thermal and mechanical stability, and large refractive index tunability. However, the residual OH-like groups remaining after low temperature treatment have hindered the development of the erbium doped waveguide amplifiers working at ~1550 nm. A variety of means have been adopted to overcome this problem: (1) low vibration energy deuterated or fluorinated polymers have been applied as matrix materials; (2) erbium ions have been incorporated into organic or inorganic complexes for protection and the ligands of the complexes have been further modified to reduce the vibrational energy or increase the pump energy transfer efficiency; and (3) the erbium ions have been shielded in inorganic nanoparticles or other micro-structures. Doping erbium ions along with ytterbium ions to increase the pumping efficiency in lanthanum phosphate nanoparticles appears to be a very promising approach since (i) the emission lifetime is fairly long; (ii) at ~1550 nm the nanoparticles are transparent and have a refractive index very close to that of standard optical fiber; and (iii) the nanoparticles are highly dispersible in most organic media. Here we demonstrate how Er3+/Yb3+ containing lanthanum phosphate nanoparticles dispersed in an organically modified sol-gel can be used in a reverse mesa waveguide to achieve optical signal enhancement. Er3+/Yb3+ containing lanthanum phosphate nanoparticles have been synthesized with a molar ratio of La:Yb:Er = 76:21:3 and dispersed in cyclopentanone with 30wt% hydrolyzed MAPTMS. The composite has been incorporated into a 4mm reverse mesa waveguide prepared using an organic-inorganic hybrid sol-gel material based on MAPTMS and zirconium n-propoxide. An optical signal enhancement of ~2 dB/cm at 1527 nm has been obtained using a 200 mW 980 nm pump laser. Modeling analyses have shown that the overall performance can be further improved. The results indicate that such hybrid sol-gel reverse mesa waveguides using Er3+/Yb3+ containing lanthanum phosphate nanoparticles are very promising for optical amplifiers in integrated optical systems.