The Abdus Salam International Centre for Theoretical Physics (ICTP) is concluding a three-year project Special African Project with a component aimed at building Computational Research Infrastructures for Africa. The component, with funding from the Italian government, will setup a few Computational Physics/ High Performance Computing (HPC) Centres in sub-Saharan Africa. This component will foster the implementation of low cost commodity based Linux clusters of 5–32 nodes with an emphasis on building sustainable/self reliant capacity for implementing and maintaining Linux based commodity clusters, providing end-user support for HPC applications, porting applications to HPC environment and researching new HPC technologies. ICTP has a longstanding tradition in building scientific capacity in the developing world and champions the use of HPC and ICT tools in a supporting role to scientific development: even in the young area of HPC several capacity building activities have been organized in partnership with the DEMOCRITOS National Simulation Center/SISSA Joint Laboratory for e-Science. In this paper we will focus on the successful approach to capacity building used for the above subproject that has lead to the creation of an active team of HPC training instructors and computational scientists in the sub-Saharan region who are already making an impact on scientific development on the continent. We describe our methodology, the approach employed during the different training events and the lessons learnt from our training experiences that improved the capacity building aspect of the sub-project.