A compact fiber Bragg grating (FBG) diaphragm accelerometer based on L-shaped rigid cantilever beam is proposed and experimentally demonstrated. The sensing system is based on the integration of a flat diaphragm and an L-shaped rigid cantilever beam. The FBG is pre-tensioned and the two side points are fixed, efficiently avoiding the unwanted chirp effect of grating. Dynamic vibration measurement shows that the proposed FBG diaphragm accelerometer provides a wide frequency response range (0–110 Hz) and an extremely high sensitivity (106.5 pm/g), indentifying it as a good candidate for embedding structural health monitoring and seismic wave measurement.