With the development of the multimedia and network technology, video applications such as VOD, video-phone etc. are used more and more widely in our life. Consider by the high performance and easy implementation, we select TMS320 c6416 DSP platform and implement the MPEG-4 decoder. Based on the architecture and instruction characteristic of DSP, structure-level, program-level, code-level optimization was conducted respectively. Experimental data show that the decoder implemented can decode video in real time efficiently.