Purpose: Several automatic segmentation methods have been developed to aid physicians in drawing tumor contours from PETimages. Our goal is to compare the consistency of current methods for delineating in vivo tumors and uniform objects in phantom. Method and Materials: We compare three published methods, each based on a single threshold value per scan, for segmenting objects in experimental or Monte Carlo simulated PET scans of cylinders and spheres with uniform activity concentration in a phantom, and for segmenting tumor volumes in the torso from 20 patient PET scans. Results: For uniform activity objects in phantom, segmented volumes generated by the 3 methods differ from true values by more than a factor of 2. The segmented volumes are within a factor of 3 and 4 of the true volume when the objects are larger than twice the PET resolution. Between the methods, the segmented volumes differ by up to 78% and 93% for objects in zero and non‐zero background respectively. These differences are close to the volume change caused by adding a single voxel layer to the surface of the object. The discrepancies between the different segmentation methods are even larger for segmenting in vivo tumors where volume differences larger than a factor of 10 were observed, far larger than the single voxel enlargement effect. Conclusion: Threshold value based segmentation methods can be used only as a rough guide for tumor delineation and then only after adapting to each clinics PET scanner and procedures. Among the probable sources of inaccuracy are various patient dependent factors including tracer uptake non‐uniformity. This suggests that effort in analyzingPETimages should be shifted towards providing accurate quantitative information to the physician to improve confidence in target delineation amidst the various phenomena affecting the PETimage. Supported in part from NCI Grant P01‐CA59017.