Recognition of digital modulations based on mathematical classifier