A framework for developing semantic differentials in IS research: Assessing the meaning of electronic marketplace quality (EMQ)

Adequate usage of measurement instruments is key for solid research. In this study we focus on the semantic differential as general technique of measurement. Despite calls for methodological rigor in information systems (IS) research, many of the applications of the semantic differential in IS studies are characterized by flaws and weaknesses. Consequently, the findings of these studies demand cautious usage since validity problems are likely to exist. The aim of this study is to shed light on the semantic differential. Principles of semantic differentiation are discussed, and used as foundation to introduce a framework for developing and applying semantic differentials. The framework delineates the crucial role of linguistics and concept-scale interaction, and extends available guidelines for measurement validation with procedures to test wording credibility, linguistic contrast, psychological bipolarity, and contextual contamination. The framework is exemplified using a demonstration exercise, which centers on the assessment of the meaning of the concept electronic marketplace quality (EMQ). Using a mixture of qualitative and quantitative methods, the demonstration exercise clarifies the prerequisites for semantic differentiation and provides guidelines for researchers. The paper concludes with a discussion of implications for researchers, reviewers and practice.

[1]  Charles E. Osgood,et al.  The cross-cultural generality of visual-verbal synesthetic tendencies. , 2007 .

[2]  Tadasu Oyama,et al.  A cross-culture and cross-concept study of the generality of semantic spaces , 1963 .

[3]  Itamar Simonson,et al.  Anchoring Effects on Consumers' Willingness-to-Pay and Willingness-to-Accept , 2004 .

[4]  Jay F. Nunamaker,et al.  Affective Reward and the Adoption of Group Support Systems: Productivity Is Not Always Enough , 1995, J. Manag. Inf. Syst..

[5]  Izak Benbasat,et al.  Development of an Instrument to Measure the Perceptions of Adopting an Information Technology Innovation , 1991, Inf. Syst. Res..

[6]  Laku Chidambaram,et al.  How Much Bandwidth Is Enough? A Longitudinal Examination of Media Characteristics and Group Outcomes , 1999, MIS Q..

[7]  A. Parasuraman,et al.  The Behavioral Consequences of Service Quality , 1996 .

[8]  W. Foddy Constructing Questions for Interviews and Questionnaires: Theory and Practice in Social Research , 1993 .

[9]  R. Tourangeau,et al.  Cognitive Processes Underlying Context Effects in Attitude Measurement , 1988 .

[10]  Beat F. Schmid,et al.  Framework for Specifying, Building, and Operating Electronic Markets , 1998, Int. J. Electron. Commer..

[11]  C WetherbeJames,et al.  The relationship between user participation and user satisfaction , 1994 .

[12]  Uriel Weinreich,et al.  Travels through semantic space , 1958 .

[13]  Jerald Greenberg,et al.  The College Sophomore as Guinea Pig: Setting the Record Straight , 1987 .

[14]  Detmar W. Straub,et al.  Toward New Metrics for Net-Enhanced Organizations , 2002, Inf. Syst. Res..

[15]  E. Laird Landon,et al.  Order Bias, the Ideal Rating, and the Semantic Differential , 1971 .

[16]  Gerald Albaum,et al.  Stapel Scale or Semantic Differential in Marketing Research? , 1974 .

[17]  J. Cacioppo,et al.  Relationship between attitudes and evaluative space: A critical review, with emphasis on the separability of positive and negative substrates. , 1994 .

[18]  Jonathan W. Palmer,et al.  Web Site Usability, Design, and Performance Metrics , 2002, Inf. Syst. Res..

[19]  L GriffithTerri,et al.  Cognitive elements in the implementation of new technology , 1996 .

[20]  Ruey-Lin Hsiao,et al.  Technology fears: distrust and cultural persistence in electronic marketplace adoption , 2003, J. Strateg. Inf. Syst..

[21]  Yannis Bakos,et al.  A Strategic Analysis of Electronic Marketplaces , 1991, MIS Q..

[22]  Young Eun Lee,et al.  The Effects of Virtual Reality on Consumer Learning: An Empirical Investigation , 2005, MIS Q..

[23]  William A. Mindak,et al.  Fitting the Semantic Differential to the Marketing Problem , 1961 .

[24]  Prashant C. Palvia,et al.  Developing and validating an instrument for measuring user-perceived web quality , 2002, Inf. Manag..

[25]  W. Bearden,et al.  The use of expert judges in scale development: Implications for improving face validity of measures of unobservable constructs , 2004 .

[26]  Martin Grieger Electronic marketplaces: A literature review and a call for supply chain management research , 2003, Eur. J. Oper. Res..

[27]  David R. Heise,et al.  Some methodological issues in semantic differential research. , 1969 .

[28]  Marvin A. Jolson,et al.  Statement Polarity in Attitude Studies , 1974 .

[29]  Ming-Hui Huang,et al.  Web performance scale , 2005, Inf. Manag..

[30]  Richard G. Netemeyer,et al.  Handbook of Marketing Scales: Multi-Item Measures for Marketing and Consumer Behavior Research , 1993 .

[31]  Subhash Sharma,et al.  Sample Size Effects on Chi Square and Other Statistics Used in Evaluating Causal Models , 1982 .

[32]  Elena Karahanna,et al.  Time Flies When You're Having Fun: Cognitive Absorption and Beliefs About Information Technology Usage , 2000, MIS Q..

[33]  D. K. Darnell,et al.  Concept scale interaction in the semantic differential. , 1966, The Journal of communication.

[34]  Albert H. Segars,et al.  An Empirical Examination of the Concern for Information Privacy Instrument , 2002, Inf. Syst. Res..

[35]  H. Raghav Rao,et al.  On risk, convenience, and Internet shopping behavior , 2000, CACM.

[36]  Joseph A. Cote,et al.  Estimating Trait, Method, and Error Variance: Generalizing across 70 Construct Validation Studies , 1987 .

[37]  Neil F. Doherty,et al.  The relative success of alternative approaches to strategic information systems planning: an empirical analysis , 1999, J. Strateg. Inf. Syst..

[38]  R. Bagozzi,et al.  A cross-country construct validation of cognitive age , 2006 .

[39]  Hans van der Heijden,et al.  User Acceptance of Hedonic Information Systems , 2004, MIS Q..

[40]  Ingoo Han,et al.  The Impact of Customer Trust and Perception of Security Control on the Acceptance of Electronic Commerce , 2003, Int. J. Electron. Commer..

[41]  ChidambaramLaku,et al.  How much bandwidth is enough? a longitudinal examination of media characteristics and group outcomes , 1999 .

[42]  John P. Robinson,et al.  CHAPTER 1 – Criteria for Scale Selection and Evaluation , 1991 .

[43]  David M. Szymanski,et al.  E-satisfaction: an initial examination , 2000 .

[44]  M. Gilly,et al.  eTailQ: dimensionalizing, measuring and predicting etail quality , 2003 .

[45]  Adamantios Diamantopoulos,et al.  The effect of pretest method on error detection rates: Experimental evidence , 1998 .

[46]  Fatemeh Zahedi,et al.  Exploring the influence of perceptual factors in the success of web-based spatial DSS , 2007, Decis. Support Syst..

[47]  Wynne W. Chin Issues and Opinion on Structural Equation Modeling by , 2009 .

[48]  L. F. Lages,et al.  The Relationship between Buyer and a B2b E-Marketplace: Cooperation Determinants in an Electronic Market Context , 2006 .

[49]  Ho Geun Lee,et al.  Market Process Reengineering through Electronic Market Systems: Opportunities and Challenges , 1996, J. Manag. Inf. Syst..

[50]  Tibert Verhagen,et al.  Online store image: conceptual foundations and empirical measurement , 2004, Inf. Manag..

[51]  T. Skjøtt-Larsen,et al.  Electronic marketplaces and supply chain relationships , 2003 .

[52]  J. Hair Multivariate data analysis , 1972 .

[53]  C. Osgood,et al.  Semantic differential technique : a sourcebook , 1969 .

[54]  Anol Bhattacherjee,et al.  Influence Processes for Information Technology Acceptance: An Elaboration Likelihood Model , 2006, MIS Q..

[55]  P. Bentler,et al.  Significance Tests and Goodness of Fit in the Analysis of Covariance Structures , 1980 .

[56]  Barbara Bickart,et al.  Carryover and Backfire Effects in Marketing Research , 1993 .

[57]  Zheng Zhou,et al.  Development and validation of an instrument to measure user perceived service quality of information presenting Web portals , 2005, Inf. Manag..

[58]  M. Parry,et al.  A Cross-National Comparative Study of New Product Development Processes: Japan and the United States , 1997 .

[59]  Paul A. Pavlou,et al.  Building Effective Online Marketplaces with Institution-Based Trust , 2004, Inf. Syst. Res..

[60]  Brian S. Butler,et al.  Intermediaries and Cybermediaries: A Continuing Role for Mediating Players in the Electronic Marketplace , 1995, J. Comput. Mediat. Commun..

[61]  Timothy Paul Cronan,et al.  Modeling IT Ethics: A Study in Situational Ethics , 1998, MIS Q..

[62]  W. Bearden,et al.  Consumer Self‐Confidence: Refinements in Conceptualization and Measurement , 2001 .

[63]  William Foddy,et al.  The in-depth testing of survey questions: A critical appraisal of methods , 1996 .

[64]  Richard P. Bagozzi,et al.  Assessing Construct Validity in Organizational Research , 1991 .

[65]  A. Tversky,et al.  Judgment under Uncertainty , 1982 .

[66]  Norman Fletcher,et al.  Review of The Measurement of Meaning. , 1958 .

[67]  L. Stoel,et al.  Apparel retailers: website quality dimensions and satisfaction , 2004 .

[68]  T. A. Michalske,et al.  An integrated approach to chemical microsystems: A state-of-the-art assessment , 1997 .

[69]  Terry L. Childers,et al.  The measurement of intergenerational communication and influence on consumption: Development, validation, and cross-cultural comparison of the IGEN scale , 2000 .

[70]  Robert A Ping,et al.  On assuring valid measures for theoretical models using survey data , 2004 .

[71]  Peter P. Mykytyn,et al.  Effects of computer experience and task complexity on attitudes of managers , 1992, Inf. Manag..

[72]  Wynne W. Chin,et al.  Research Report: Better Theory Through Measurement-Developing a Scale to Capture Consensus on Appropriation , 2002, Inf. Syst. Res..

[73]  John P. Robinson,et al.  Measures Of Personality And Social Psychological Attitudes , 1991 .

[74]  Bodo B. Schlegelmilch,et al.  Pre-Testing in Questionnaire Design: A Review of the Literature and Suggestions for Further Research , 1993 .

[75]  Jos van Iwaarden,et al.  Perceptions about the quality of web sites: a survey amongst students at Northeastern University and Erasmus University , 2004, Inf. Manag..

[76]  Anol Bhattacherjee,et al.  Understanding Changes in Belief and Attitude Toward Information Technology Usage: A Theoretical Model and Longitudinal Test , 2004, MIS Q..

[77]  Beth H. Jones,et al.  Impact of Communication Medium and Computer Support on Group Perceptions and Performance: A Comparison of Face-to-Face and Dispersed Meetings , 1993, MIS Q..

[78]  Anol Bhattacherjee,et al.  Understanding Information Systems Continuance: An Expectation-Confirmation Model , 2001, MIS Q..

[79]  Andrew B. Whinston,et al.  Hope or Hype: On the Viability of Escrow Services as Trusted Third Parties in Online Auction Environments , 2004, Inf. Syst. Res..

[80]  V. Srinivasan,et al.  Note---The Factor Structure of Multidimensional Response to Marketing Stimuli: A Comparison of Two Approaches , 1989 .

[81]  Detmar W. Straub,et al.  Validation in Information Systems Research: A State-of-the-Art Assessment , 2001, MIS Q..

[82]  J Blair,et al.  An experimental comparison of alternative pretest techniques: A note on preliminary findings. , 1992 .

[83]  R. Peterson On the Use of College Students in Social Science Research: Insights from a Second‐Order Meta‐analysis , 2001 .

[84]  Richard G. Netemeyer,et al.  Scaling Procedures: Issues and Applications , 2003 .

[85]  Soyoung Kim,et al.  Dimensional hierarchy of retail website quality , 2004, Inf. Manag..

[86]  Tibert Verhagen,et al.  Perceived risk and trust associated with purchasing at electronic marketplaces , 2006, Eur. J. Inf. Syst..

[87]  Joseph J. Martocchio,et al.  Microcomputer playfulness: development of a measure with workplace implications , 1992 .

[88]  Monica Martinussen,et al.  Likert-based vs. semantic differential-based scorings of positive psychological constructs: A psychometric comparison of two versions of a scale measuring resilience. , 2006 .

[89]  Fred D. Davis Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[90]  Sammy W. Pearson,et al.  Development of a Tool for Measuring and Analyzing Computer User Satisfaction , 1983 .

[91]  Dennis F. Galletta,et al.  Social influence and end-user training , 1995, CACM.

[92]  Gilbert A. Churchill A Paradigm for Developing Better Measures of Marketing Constructs , 1979 .

[93]  Tor Guimaraes,et al.  The Relationship Between User Participation and User Satisfaction: An Investigation of Four Contingency Factors , 1994, MIS Q..

[94]  Nitish Singh,et al.  Culture and web communications , 2007 .

[95]  Marvin R. Goldfried,et al.  On the Bipolarity of Semantic Space , 1965 .

[96]  Uma Sekaran,et al.  Methodological and Theoretical Issues and Advancements in Cross-Cultural Research , 1983 .

[97]  Stephanie Watts,et al.  Straight Talk: Delivering Bad News through Electronic Communication , 1999, Inf. Syst. Res..

[98]  Suci Gj,et al.  A comparison of semantic structures in American Southwest culture groups. , 1960 .

[99]  G J SUCI A comparison of semantic structures in American Southwest culture groups. , 1960, Journal of abnormal and social psychology.

[100]  C. Osgood,et al.  Factor analysis of meaning. , 1955, Journal of experimental psychology.

[101]  J. M. Kittross The measurement of meaning , 1959 .

[102]  Theodore Clevenger,et al.  Measurement of Corporate Images by the Semantic Differential , 1965 .

[103]  C. Osgood The nature and measurement of meaning. , 1952, Psychological bulletin.

[104]  Chi-Bin Cheng,et al.  Intelligent agents for e-marketplace: Negotiation with issue trade-offs by fuzzy inference systems , 2006, Decis. Support Syst..

[105]  Xianggui Qu,et al.  Multivariate Data Analysis , 2007, Technometrics.

[106]  Terri L. Griffith,et al.  Cognitive Elements in the Implementation of New Technology: Can Less Information Provide More Benefits? , 1996, MIS Q..

[107]  Henry Clay Lindgren,et al.  Review of The Measurement of Meaning. , 1958 .

[108]  Naresh K. Malhotra,et al.  A Scale to Measure Self-Concepts, Person Concepts, and Product Concepts , 1981 .

[109]  J. Deese The associative structure of some common english adjectives , 1964 .

[110]  Fred D. Davis,et al.  Developing and Validating an Observational Learning Model of Computer Software Training and Skill Acquisition , 2003, Inf. Syst. Res..

[111]  C. Fornell,et al.  Evaluating Structural Equation Models with Unobservable Variables and Measurement Error , 1981 .

[112]  Chester A. Schriesheim,et al.  Fiedler's Least Preferred Coworker (LPC) Instrument: An Investigation of its True Bipolarity , 1991 .

[113]  Louis Raymond,et al.  Determinants of EIS use: Testing a behavioral model, , 1995, Decis. Support Syst..

[114]  Ronald K. Hambleton,et al.  Advances in Criterion-Referenced Measurement , 1991 .

[115]  N. Cliff,et al.  Adverbs as multipliers. , 1959, Psychological review.

[116]  T. Brashear,et al.  Development and validation of scales to measure attitudes influencing monetary donations to charitable organizations , 2000 .

[117]  Michael Tow Cheung,et al.  Internet-based e-banking and consumer attitudes: an empirical study , 2002, Inf. Manag..

[118]  Jagdip Singh,et al.  Consumer Trust, Value, and Loyalty in Relational Exchanges , 2002 .

[119]  Subhash Sharma,et al.  Consumer Ethnocentrism: Construction and Validation of the CETSCALE , 1987 .

[120]  Jochen Wirtz,et al.  An Examination of the Quality and Context-Specific Applicability of Commonly Used Customer Satisfaction Measures , 2003 .

[121]  R. F. Kelly,et al.  The Semantic Differential: An Information Source for Designing Retail Patronage Appeals , 1967 .

[122]  Fatemeh Zahedi,et al.  The Measurement of Web-Customer Satisfaction: An Expectation and Disconfirmation Approach , 2002, Inf. Syst. Res..

[123]  Albert H. Segars,et al.  Assessing the unidimensionality of measurement : a paradigm and illustration within the context of information systems research , 1997 .

[124]  J. Aaker,et al.  Dimensions of Brand Personality , 1997 .

[125]  Wynne W. Chin,et al.  Advancing the Theory of Adaptive Structuration: The Development of a Scale to Measure Faithfulness of Appropriation , 1997, Inf. Syst. Res..

[126]  William J. Doll,et al.  A Confirmatory Factor Analysis of the End-User Computing Satisfaction Instrument , 1994, MIS Q..

[127]  W. Foddy An Empirical Evaluation of In-Depth Probes Used to Pretest Survey Questions , 1998 .

[128]  Stuart Van Auken,et al.  An Assessment of the Trait Validity of Cognitive Age Measures , 1995 .

[129]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[130]  Shintaro Okazaki,et al.  What do we know about mobile Internet adopters? A cluster analysis , 2006, Inf. Manag..

[131]  Richard A. Spreng,et al.  A Reexamination of the Determinants of Consumer Satisfaction , 1996 .

[132]  Suzanne Rivard,et al.  An Integrative Contingency Model of Software Project Risk Management , 2001, J. Manag. Inf. Syst..

[133]  R. Oliver A Cognitive Model of the Antecedents and Consequences of Satisfaction Decisions , 1980 .

[134]  Marcel Creemers,et al.  Understanding online purchase intentions: contributions from technology and trust perspectives , 2003, Eur. J. Inf. Syst..

[135]  DeGross Errata Notes: User Heterogeneity and Its Impact on Electronic Auction Market Design: An Empirical Exploration , 2004 .

[136]  Dennis P. Slevin,et al.  The Use of Semantic Differential Scales in Studying The Innovation Boundary , 1975 .

[137]  D. Sandy Staples,et al.  The use of collaborative electronic media for information sharing: an exploratory study of determinants , 2000, J. Strateg. Inf. Syst..

[138]  Robert D. St. Louis,et al.  Convincing DSS users that complex models are worth the effort , 2004, Decis. Support Syst..

[139]  France Bélanger,et al.  Trustworthiness in electronic commerce: the role of privacy, security, and site attributes , 2002, J. Strateg. Inf. Syst..

[140]  Younghwa Lee,et al.  Investigating the effect of website quality on e-business success: An analytic hierarchy process (AHP) approach , 2006, Decis. Support Syst..

[141]  Detmar W. Straub,et al.  Validating Instruments in MIS Research , 1989, MIS Q..

[142]  Raj Mehta,et al.  An Investigation into the Antecedents of Organizational Participation in Business-to-Business Electronic Markets , 2001 .

[143]  Yannis Bakos,et al.  The emerging role of electronic marketplaces on the Internet , 1998, CACM.

[144]  Abraham Seidmann,et al.  Managing Online Auctions: Current Business and Research Issues , 2003, Manag. Sci..

[145]  P. Bahr,et al.  Sampling: Theory and Applications , 2020, Applied and Numerical Harmonic Analysis.

[146]  Heiner Evanschitzky,et al.  E-satisfaction: a re-examination , 2004 .

[147]  Joseph O. Rentz,et al.  A measure of service quality for retail stores: Scale development and validation , 1996 .

[148]  Jack M. Feldman,et al.  Self-generated validity and other effects of measurement on belief, attitude, intention, and behavior. , 1988 .

[149]  Christopher D. Allport,et al.  A Research Note Regarding the Development of the Consensus on Appropriation Scale , 2003, Inf. Syst. Res..

[150]  Carol Stoak Saunders,et al.  Electronic window dressing: impression management with Websites , 2003, Eur. J. Inf. Syst..

[151]  Chester A. Schriesheim,et al.  Development and Application of a New Approach to Testing the Bipolarity of Semantic Differential Items , 1994 .

[152]  Howard B. Lee,et al.  Foundations of Behavioral Research , 1973 .

[153]  Eldon Y. Li,et al.  Marketing Information Systems in the Fortune 500 Companies: Past, Present, and Future , 1993, J. Manag. Inf. Syst..

[154]  Nikunj P. Dalal,et al.  Web home pages as advertisements , 1999, CACM.

[155]  Robert J. Bigg Some Methodological Issues , 1990 .

[156]  Gerald Albaum,et al.  A Method for Developing Tailormade Semantic Differentials for Specific Marketing Content Areas , 1977 .

[157]  R. Berk Handbook of methods for detecting test bias , 1982 .

[158]  V. Mittal,et al.  Satisfaction, Repurchase Intent, and Repurchase Behavior: Investigating the Moderating Effect of Customer Characteristics , 2001 .

[159]  David W. Gerbing,et al.  An Updated Paradigm for Scale Development Incorporating Unidimensionality and Its Assessment , 1988 .

[160]  Samuel Messick,et al.  Metric Properties of the Semantic Differential , 1957 .

[161]  A. Burns,et al.  The antecedents of preventive health care behavior: An empirical study , 1998 .

[162]  Paul A. Pavlou,et al.  Institution-Based Trust in Interorganizational Exchange Relationships: The Role of Online B2B Marketplaces on Trust Formation , 2002, J. Strateg. Inf. Syst..

[163]  William R. King,et al.  Antecedents of Knowledge Transfer from Consultants to Clients in Enterprise System Implementations , 2005, MIS Q..

[164]  D. Gerbing,et al.  Viability of exploratory factor analysis as a precursor to confirmatory factor analysis , 1996 .