Towards “matter matters” in spin foam quantum gravity