The high-frequency permittivity of composites consisting of a lattice of ferromagnetic wires is investigated. Experimental results using free space or coaxial line microwave measurements are reported. It is shown that the dielectric response is strongly dependent on the magnetic properties of the wires. Negative real permittivity is observed over a wide frequency range for wires with circumferential magnetization, while a resonant behavior is observed on wires with an axially magnetized core. In addition, it is shown that a moderate external field can induce large changes in the dielectric response. We prove that the underlying physics of these composites made of oriented magnetic wires is basically the same as the giant magnetoimpedance (GMI) effect. A model based on GMI equations is proposed which predicts this unusual dielectric phenomenon.