An electrochemical device capable of manifesting reversible charge storage at the interface of an active layer offers formidable advantages, such as low switching energy and long retention time, in realizing synaptic behavior for ultralow power neuromorphic systems. Contrary to a supercapacitor-based field-effect device that is prone to low memory retention due to fast discharge, a solid electrolyte-gated ZnO thin-film device exhibiting a battery-controlled charge storage mechanism via mobile charges at its interface with tantalum oxide is demonstrated. Analysis via cyclic voltammetry and chronoamperometry uniquely distinguishes the battery behavior of these devices, with an electromotive force generated due to polarization of charges strongly dependent on the scan rate of the applied voltage. The Faradaic-type diffusion-controlled charge storage mechanism exhibited by these devices is capable of delivering robust enhancement in the channel conductance and leads to a superior ON–OFF ratio of 108–109. The nonvolatile behavior of the interface charge storage and slow diffusion of ions is utilized in efficiently emulating spike timing-dependent plasticity (STDP) at similar time scales of biological synapses and unveils the possibility of STDP behavior using multiple in-plane gates that alleviate additional requirement of waveform-shaping circuits.