Experimental observation of the surface roughening transition in vapor phase growth