Wavelength demultiplexer operating over mode division multiplexed signals on rectangular core fiber