With the rapid development of deep learning, the neural network becomes an efficient approach for eddy detection. However, previous work employs a traditional neural network with a focus on improving the detecting accuracy only using limited data under a single scenario. Meanwhile, the experience of detecting eddies from one experiment is not directly inherited from the detection model for other experiments. Therefore, a cross-domain submesoscale eddy detection neural network (CDEDNet) based on the high-frequency radar (HFR) data of the Nansan and Xuwen region is proposed in this paper. Firstly, a fundamental deep eddy detection architecture CDEDNet-0 is constructed with a fully convolutional network (FCN). Secondly, for solving the problem of insufficient labeled eddy data, an instance-based domain adaption method is adopted in CDEDNet-1 to increase training samples. Thirdly, for tackling the problem of unable to inherit previous detection experience, parameter-based transfer learning is incorporated in CDEDNet-2 for multi-scene eddy detection. The experiment results demonstrate CDEDNet-1 and CDEDNet-2 perform better than CDEDNet-0 in terms of accuracy. Meanwhile, eddy characteristics including eddy type, radius, occurring time, merger, and dynamic trajectory are analyzed for the Nansan and Xuwen regions.