Suppressed flavor violation in lepton flavored dark matter from an extra dimension