This paper investigates the influence of concrete railway sleepers on the ground-penetrating radar (GPR) signal for the assessment of railway ballast conditions. The main aim of this study is to propose a data processing scheme capable to filter out the effects of concrete sleepers on GPR signals with compound information (i.e., containing multiple reflections from the concrete sleepers and the underlying ballast material). The method relies on spectral-based filtering, i.e., it is based on the minimisation of the difference between i) the frequency spectra of the GPR traces collected along the scanning line at the abscissae of the sleepers and ii) the spectra of the traces acquired at the abscissae of the material between two consecutive sleepers. To this purpose, a stretch of 9.8 km of railway line was investigated using two horn antenna systems with operating frequencies of 1000 and 2000 MHz. Each system was oriented in two different ways, i.e. longitudinally and transversely with respect to the orientation of the tracks. Results have proven the viability of the proposed data processing scheme and spectral-based filtering to reduce the effects exerted by the sleepers on the signal.