Theory of high-energy emission from the pulsar/Be star system PSR 1259-63. I. Radiation mechanisms and interaction geometry