An aminoacyl-tRNA synthetase that specifically activates pyrrolysine.