The study deals with an accurate cutting error prediction of end milling in the finishing operation. As it was indicated that the consideration of cutting edge roundness improves the accuracy of cutting force simulation, a new estimation method of cutting force has been proposed for end milling with the straight flute tools, based on the calculation of the actual depth of cut. The influence of cutting edge roundness was not considered in the conventional cutting error prediction. Then, the study proposes an extended estimation method of cutting error for the helical flute end mill. It is necessary to calculate the actual depth of cut with an accurate description method of workpiece surface since the amount of cutting error on workpiece surface usually changes with the height of a cutting flute. As a change in workpiece surface in machining is correctly estimated by the calculation of the actual depth of cut, the estimation of cutting error is attained by referring to the form of machined workpiece surface, which is left behind after the end mill passed. A prototype of simulation system is developed, based on the proposed method. The effectiveness of the proposed estimation method is demonstrated by making cutting experiments with helical flute end mill under the cutting condition of finishing operation. The predicted cutting error shows a good qualitative agreement with the measured one, thus the more accurate prediction of cutting error is realized by the proposed method.