This paper considers that Multi Protocol Label Switching (MPLS) and Differential Services (DiffServ) architectures are being applied to IP networks to provide more scalability, service differentiation. The traditional Diffserv mechanism provides classifier for edge router to examine incoming packets and mark packets with a code point that reflects the desired level of service, the core router responsibilities are to examine incoming packets for code point and forward incoming packets according to their code points. However, when multiple edge routers transmit data packets high speedily to a single core router, the core router will have bottleneck congestion because the core router has not enough buffers to cope with a large number of incoming packets. This paper proposes a scheme based on DiffServ and MPLS that edge routers are responsible for marking and dropping packets and the core routers are mainly responsible for forwarding the packets. This paper also illustrates its effectiveness by performing a simulation using Network Simulator (ns-2). The simulation results show that the proposed scheme can effectively alleviate the congestion of network and improve the QOS.