Spectral characteristics of a modified inverted-Y system beyond rotating wave approximation