We analyze the structure of the vacuum polarization tensor in the presence of a background electromagnetic field in a medium. We use various discrete symmetries and crossing symmetry to constrain the form factors obtained for the most general case. From these symmetry arguments, we show why the vacuum polarization tensor has to be even in the background field when there is no background medium. Taking then the background field to be purely magnetic, we evaluate the vacuum polarization to linear order in it. The result shows the phenomenon of Faraday rotation, i.e., the rotation of the plane of polarization of a plane polarized light passing through this background. We find that the usual expression for Faraday rotation, which is derived for a non-degenerate plasma in the non-relativistic approximation, undergoes substantial modification if the background is degenerate and/or relativistic. We give explicit expressions for Faraday rotation in completely degenerate and ultra-relativistic media.