Spin torque magnetic memory (ST-MRAM) is currently under intense academic and industrial development, as it features non-volatility, high write and read speed and high endurance. However, one of its great challenge is the probabilistic nature of programming magnetic tunnel junctions, which imposes significant circuit or energy overhead for conventional ST-MRAM applications. In this work, we show that in unconventional computing applications, this drawback can actually be turned into an advantage. First, we show that conventional magnetic tunnel junctions can be reinterpreted as stochastic “synapses” that can be the basic element of low-energy learning systems. System-level simulations on a task of vehicle counting highlight the potential of the technology for learning systems. We investigate in detail the impact of magnetic tunnel junctions’ imperfections. Second, we introduce how intentionally superparamagnetic tunnel junctions can be the basis for low-energy fundamentally stochastic computing schemes, which harness part of their energy in thermal noise. We give two examples built around the concepts of synchronization and Bayesian inference. These results suggest that the stochastic effects of spintronic devices, traditionally interpreted by electrical engineers as a drawback, can be reinvented as an opportunity for low energy circuit design.