Quantum and classical criticality in a dimerized quantum antiferromagnet