The Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2) has been collecting elevation measurements of the Earth’s surface since its launch in September 2018. Although ICESat-2 was not designed for urban applications, its excellent altimetry capabilities over the globe make it possible to obtain urban height information. However, few studies have been conducted to validate ICESat-2 height measurements in urban areas. In this letter, we evaluate the heights retrieved from 20 months of ATL03 and ATL08 data using airborne LiDAR data collected in New York City (NYC). The results indicated that the heights from ATL03 product have a moderate agreement with airborne LiDAR data with vertical errors around −1.49 m (root mean squared error [RMSE] = 2.89 m, mean absolute error [MAE] = 2.11 m, $R^{2} = 0.98$ , and observations = 910 497). The ATL08 product also performed a fine accuracy in terrain heights estimating (mean error [ME] = −0.01 m, RMSE = 3.63 m, MAE = 2.04 m, and $R^{2} = 0.94$ ). In the three categories of urban environments, ATL03 performs best in urban high-rise dense area, with an average residual error of −1.44 m, followed by urban non-high-rise dense area with an average residual error of −1.49 m. In urban forest area, ATL08 shows its performance in measuring terrain height with an RMSE of 1.78 m. We demonstrated that ICESat-2 can provide a useful source of urban heights and holds great potential to light up more urban applications related to urban change monitoring and 3-D morphology.