Efficiency bottlenecks inherent to conventional computing in executing neural algorithms have spurred the development of novel devices capable of “in-memory” computing. Commonly known as “memristors,” a variety of device concepts including conducting bridge, vacancy filament, phase change, and other types have been proposed as promising elements in artificial neural networks for executing inference and learning algorithms. In this article, we review the recent advances in memristor technology for neuromorphic computing and discuss strategies for addressing the most significant performance challenges, including nonlinearity, high read/write currents, and endurance. As an alternative to two-terminal memristors, we introduce the three-terminal electrochemical memory based on the redox transistor (RT), which uses a gate to tune the redox state of the channel. Decoupling the “read” and “write” operations using a third terminal and storage of information as a charge-compensated redox reaction in the bulk of the transistor enables high-density information storage. These properties enable low-energy operation without compromising analog performance and nonvolatility. We discuss the RT operating mechanisms using organic and inorganic materials, approaches for array integration, and prospects for achieving the device density and switching speeds necessary to make electrochemical memory competitive with established digital technology.