Role of matrix elements in the time-resolved photoemission signal