Large scale signal and information processing by simultaneous characterization of randomness and structures