Testing linearity in semi-parametric functional data analysis