Monitoring heart rate (HR) by a single wrist-worn accelerometer would provide many advantages over electrocardiogram (ECG) or photoplethysmography (PPG), such as wearing comfortability, saving power, tiny footprint and automatic motion artifact removal. However, like ECG, accelerometry was mostly implemented on the chest, named seismocardiogram (SCG), and PPG was dominating the wrist worn format. Pulse condition detection in Traditional Chinese Medicine or sphygmography was engineered into a wearable format in this work, and a wrist-worn HR monitor by a single accelerometer was demonstrated. A major limitation of wrist-worn device is that motion artifacts and noise severely corrupt the signal integrity. In this study, with raw data segments including hundreds of random finger or hand motions, several signal decomposition algorithms were compared, such as independent component analysis (ICA), variable mode decomposition (VMD), wavelet synchrosqueezed transform (WSST), and singular spectrum analysis (SSA), and Kalman smoothing was implemented to track HR continuously. Our method was tested on 20 subjects during random finger tapping or hand swinging. Without knowing properties of noise or motion beforehand or requirement of an extra sensor as reference, our method provides a significant removal of motion artifacts and noise from three-dimensional acceleration signals, with 95% of HR estimation within ±8.86 bpm, much longer battery life and better wearing comfort than PPG. It would broaden the working conditions, and thus provide a more holistic assessment of HR during everyday life.