Background Substantial falls in the mortality of people living with HIV (PLWH) have been observed since the introduction of antiretroviral therapy (ART) in sub-Saharan Africa. However, access and uptake of ART have been variable in many countries. We report the excess deaths observed in PLWH before and after the introduction of ART. We use data from five longitudinal studies in Malawi, South Africa, Tanzania, and Uganda, members of the network for Analysing Longitudinal Population-based HIV/AIDS data on Africa (ALPHA). Methods Individual data from five demographic surveillance sites that conduct HIV testing were used to estimate mortality attributable to HIV, calculated as the difference between the mortality rates in PLWH and HIV-negative people. Excess deaths in PLWH were standardized for age and sex differences and summarized over periods before and after ART became generally available. An exponential regression model was used to explore differences in the impact of ART over the different sites. Results 127,585 adults across the five sites contributed a total of 487,242 person years. Before the introduction of ART, HIV-attributable mortality ranged from 45 to 88 deaths per 1,000 person years. Following ART availability, this reduced to 14-46 deaths per 1,000 person years. Exponential regression modeling showed a reduction of more than 50% (HR =0.43, 95% CI: 0.32-0.58), compared to the period before ART was available, in mortality at ages 15-54 across all five sites. Discussion Excess mortality in adults living with HIV has reduced by over 50% in five communities in sub-Saharan Africa since the advent of ART. However, mortality rates in adults living with HIV are still 10 times higher than in HIV-negative people, indicating that substantial improvements can be made to reduce mortality further. This analysis shows differences in the impact across the sites, and contrasts with developed countries where mortality among PLWH on ART can be similar to that of the general population. Further research is urgently needed to establish why the different impacts on mortality were observed and how the care and treatment programmes in these countries can be more effective in reducing mortality further.