This paper describes a control method for a power assist device used in factories. An adaptive control scheme is employed to control the power assist device and to estimate its dynamic parameters. Using the adaptive control, the maneuverability of the system is good in free space but it is very dangerous in the task of which an object supported by system contacts on a floor or a wall. Therefore, we propose an improved system controlled by an adaptive control in which the local control method changes to a feedback or a feedforward control in the contact condition. The improved system detects collisions based on the difference between the actual input torque to the power assist device and reference input torque, which is calculated based on the estimated parameters of the manipulator dynamics. Then, the effectiveness of the system is shown.