The mathematical structure of characters and modularity