The reflectivity of a large LuFe2O4 single crystal has been measured with the radiation field either perpendicular or parallel to the c axis of its rhombohedral structure, from 10 to 500 K, and from 7 to 16 000 cm(-1). The transition between the two-dimensional and the three-dimensional charge order at T-CO=320 K is found to change dramatically the phonon spectrum in both polarizations. The number of the observed modes above and below T-CO, according to a factor-group analysis, is in good agreement with a transition from the rhombohedral space group R (3) over barm to the monoclinic C2/m. In the subterahertz region a peak becomes evident at low temperature, whose origin is discussed in relation with previous experiments.