Satellite optical communication has gained significant attention owing to its many quality features (e.g., a larger bandwidth, license free spectrum, higher data rate, and better security) compared to satellite microwave communication. Various experiments have been performed during many space missions to demonstrate and characterize inter-satellite links, downlinks, and uplinks. Korea has also planned to establish an experimental communication system using a geostationary earth orbit (GEO) satellite and the Geochang station as an optical ground station for low Earth orbit (LEO)-to-ground optical relay links. In this study, the performance of inter-satellite communication links and downlinks was investigated for the new Korean experimental communication system in terms of link margin, bit error rate (BER), and channel capacity. In particular, the performance of the inter-satellite links was analyzed based on the receiving apertures and the transmitting power, while that of the downlink was analyzed in terms of atmospheric turbulence conditions and transmitting power. Finally, we discussed two system parameters of receiving aperture and transmitting power to meet the three criteria of link margin, BER, and channel capacity.