The rupture of atherosclerotic plaques is the leading cause of acute coronary events, so accurate assessment of plaque is critical. A large lipid pool, thin fibrous cap, and inflammatory reaction are the crucial characteristics for identifying vulnerable plaques. In our study, a tri-modality imaging system for intravascular imaging was designed and implemented. The tri-modality imaging system with a 1-mm probe diameter is able to simultaneously acquire optical coherence tomography (OCT), intravascular ultrasound (IVUS), and fluorescence imaging. Moreover, for fluorescence imaging, we used the FDA-approved indocyanine green (ICG) dye as the contrast agent to target lipid-loaded macrophages. Firstly, IVUS is used as the first step for identifying plaque since IVUS enables the visualization of the layered structures of the artery wall. Due to low soft-tissue contrast, IVUS only provides initial identification of the lipid plaque. Then OCT is used for differentiating fibrosis and lipid pool based on its relatively higher soft tissue contrast and high sensitivity/specificity. Last, fluorescence imaging is used for identifying inflammatory reaction to further confirm whether the plaque is vulnerable or not. Ex vivo experiment of a male New Zealand white rabbit aorta was performed to validate the performance of our tri-modality system. H and E histology results of the rabbit aorta were also presented to check assessment accuracy. The miniature tri-modality probe, together with the use of ICG dye suggest that the system is of great potential for providing a more accurate assessment of vulnerable plaques in clinical applications.