How Markovian is exciton dynamics in purple bacteria?