The importance of data in the social and behavioral domains to biomedical research is increasing, but ensuring the reusability of such data through standardization is not a trivial task. To start addressing this challenge, we developed a semantic model of the physical activity domain by reviewing 302 physical activity questions collected from standardized questionnaires and public data repositories. Our semantic model is comprised of activity keywords, qualifiers, response measures and context. We identified three types of contexts: active lifestyle, physical capacity, and environment. The majority (94%) of the 204 activity keywords extracted from the 302 questions were mapped to the UMLS Metathesaurus. Preliminary evaluation of our model with 309 additional activity questions showed that the majority of the questions were related to one of the three context categories. We also noted the need to expand context categories to incorporate the questions assessing psychological aspects of dealing with physical activities.