Non-Drude THz conductivity of graphene due to structural distortions