An Analysis of More Than 1, 400 Articles, 900 Scales, and 17 Years of Research: The State of Scales in Cyberpsychology, Behavior, and Social Networking

Cyberpsychology is a recently emergent field that examines the impact of technology upon human cognition and behavior. Given its infancy, authors have rapidly created new measures to gauge their constructs of interest. Unfortunately, few of these authors have had the opportunity to test their scales' psychometric properties and validity. This is concerning, as many theoretical assumptions may be founded upon scales with inadequate attributes. If this were found to be true, then previous findings in cyberpsychology studies would need to be retested, and future research would need to shift its focus to creating psychometrically sound and valid measures. To provide inferences on this concern, the current study examines the article reporting, scale creation, and scale reliabilities of every article published in Cyberpsychology, Behavior, and Social Networking from its inception to July 2014. The final data set encompassed the coding of 1,478 individual articles, including 921 scales, and spanning 17 years. The results demonstrate that the simple survey methodology has become more popular over time. Authors are gradually applying empirically tested scales. However, self-created measures are still the most popular, leading to concerns about the measures' validity. Also, the use of multi-item measures has increased over time, but many articles still fail to report adequate information to assess the reliability of the applied scales. Lastly, the average scale reliability is 0.81, which barely meets standard cutoffs. Overall, these results are not overly concerning, but suggestions are given on methods to improve the reporting of measures, the creation of scales, and the state of cyberpsychology.

[1]  Jinwoo Kim,et al.  Why People Continue to Play Online Games: In Search of Critical Design Factors to Increase Customer Loyalty to Online Contents , 2004, Cyberpsychology Behav. Soc. Netw..

[2]  William Gibson,et al.  Prevalence of Problematic Mobile Phone Use in British Adolescents , 2014, Cyberpsychology Behav. Soc. Netw..

[3]  Eric Rice,et al.  Online Social Networking Technologies, HIV Knowledge, and Sexual Risk and Testing Behaviors Among Homeless Youth , 2010, AIDS and Behavior.

[4]  Héctor Fuster,et al.  Relationship Between Passion and Motivation for Gaming in Players of Massively Multiplayer Online Role-Playing Games , 2014, Cyberpsychology Behav. Soc. Netw..

[5]  Ercan Dalbudak,et al.  Relationship of Internet Addiction Severity with Depression, Anxiety, and Alexithymia, Temperament and Character in University Students , 2013, Cyberpsychology Behav. Soc. Netw..

[6]  John P. Wanous,et al.  Estimating the Reliability of a Single-Item Measure , 1996 .

[7]  Hans-Peter Tossmann,et al.  A Controlled Trial of an Internet-Based Intervention Program for Cannabis Users , 2011, Cyberpsychology Behav. Soc. Netw..

[8]  L. Hedges,et al.  The Handbook of Research Synthesis and Meta-Analysis , 2009 .

[9]  Monica T. Whitty,et al.  Pushing the Wrong Buttons: Men's and Women's Attitudes toward Online and Offline Infidelity , 2003, Cyberpsychology Behav. Soc. Netw..

[10]  Mark D. Griffiths,et al.  Social Responsibility Tools in Online Gambling: A Survey of Attitudes and Behavior among Internet Gamblers , 2009, Cyberpsychology Behav. Soc. Netw..

[11]  Rachel Grieve,et al.  Cyberfaking: I Can, So I Will? Intentions to Fake in Online Psychological Testing , 2013, Cyberpsychology Behav. Soc. Netw..

[12]  Jonathan Steuer,et al.  Defining virtual reality: dimensions determining telepresence , 1992 .

[13]  Matt C. Howard.  Creation of a Computer Self-Efficacy Measure: Analysis of Internal Consistency, Psychometric Properties, and Validity , 2014, Cyberpsychology Behav. Soc. Netw..

[14]  Gilbert A. Churchill.  A Paradigm for Developing Better Measures of Marketing Constructs , 1979 .

[15]  Neil Matthews,et al.  A Pilot Study of Problem Gambling among Student Online Gamblers: Mood States as Predictors of Problematic Behavior , 2009, Cyberpsychology Behav. Soc. Netw..

[16]  TengChing-I.  How Do Challenges Increase Customer Loyalty to Online Games , 2013 .

[17]  Mira Lee,et al.  Social Network Games Uncovered: Motivations and Their Attitudinal and Behavioral Outcomes , 2012, Cyberpsychology Behav. Soc. Netw..

[18]  Stephen Lacy,et al.  Analyzing Media Messages , 2019 .

[19]  Jeffrey M. Stanton,et al.  Beyond Online Surveys: Internet Research Opportunities for Industrial‐Organizational Psychology , 2008 .

[20]  H. G. Osburn.  Coefficient alpha and related internal consistency reliability coefficients. , 2000 .

[21]  Cem Oktay Güzeller,et al.  Development of a Problematic Mobile Phone Use Scale for Turkish Adolescents , 2012, Cyberpsychology Behav. Soc. Netw..

[22]  Erin E. Hollenbaugh.  Motives for Maintaining Personal Journal Blogs , 2011, Cyberpsychology Behav. Soc. Netw..

[23]  Mustafa Koc,et al.  Facebook Addiction Among Turkish College Students: The Role of Psychological Health, Demographic, and Usage Characteristics , 2013, Cyberpsychology Behav. Soc. Netw..

[24]  Jesse Fox,et al.  Romantic Relationship Development in the Age of Facebook: An Exploratory Study of Emerging Adults' Perceptions, Motives, and Behaviors , 2013, Cyberpsychology Behav. Soc. Netw..

[25]  Michael Chan,et al.  Microblogging, Online Expression, and Political Efficacy Among Young Chinese Citizens: The Moderating Role of Information and Entertainment Needs in the Use of Weibo , 2012, Cyberpsychology Behav. Soc. Netw..

[26]  Adolfo J. Cangas,et al.  Early Detection of Drug Use and Bullying in Secondary School Children by Using a Three-Dimensional Simulation Program , 2012, Cyberpsychology Behav. Soc. Netw..

[27]  Thorsten Quandt,et al.  Unpopular, Overweight, and Socially Inept: Reconsidering the Stereotype of Online Gamers , 2014, Cyberpsychology Behav. Soc. Netw..

[28]  Alessandro Gabbiadini,et al.  Virtual Users Support Forum: Do Community Members Really Want to Help You? , 2013, Cyberpsychology Behav. Soc. Netw..

[29]  Nazir S. Hawi.  Arabic Validation of the Internet Addiction Test , 2013, Cyberpsychology Behav. Soc. Netw..

[30]  Rosanna E. Guadagno,et al.  Don't It Make My Brown Eyes Green? An Analysis of Facebook Use and Romantic Jealousy , 2013, Cyberpsychology Behav. Soc. Netw..

[31]  Dishan Kamdar,et al.  Paying a Price: Culture, Trust, and Negotiation Consequences , 2011, The Journal of applied psychology.

[32]  Sabrina C. Eimler,et al.  Men Are from Mars, Women Are from Venus? Examining Gender Differences in Self-Presentation on Social Networking Sites , 2012, Cyberpsychology Behav. Soc. Netw..

[33]  Pamela A. Moss,et al.  Can There Be Validity Without Reliability? , 1994 .

[34]  Wenliang Su,et al.  Internet-Based Intervention for the Treatment of Online Addiction for College Students in China: A Pilot Study of the Healthy Online Self-Helping Center , 2011, Cyberpsychology Behav. Soc. Netw..

[35]  Sean D. Young,et al.  The Influence of Social Networking Photos on Social Norms and Sexual Health Behaviors , 2013, Cyberpsychology Behav. Soc. Netw..

[36]  Ching-I Teng,et al.  How Do Challenges Increase Customer Loyalty to Online Games? , 2013, Cyberpsychology Behav. Soc. Netw..

[37]  U. Josefsson,et al.  Patients creating self-help on the internet -lessons for future design of internet based healthcare resources , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[38]  Sonia Livingstone,et al.  Taking risky opportunities in youthful content creation: teenagers' use of social networking sites for intimacy, privacy and self-expression , 2008, New Media Soc..

[39]  J. P. Peter.  Reliability: A Review of Psychometric Basics and Recent Marketing Practices , 1979 .

[40]  James R. Baker,et al.  Creation and Validation of the Personal Blogging Style Scale , 2011, Cyberpsychology Behav. Soc. Netw..

[41]  Jason S. Zack,et al.  Online therapy: review of relevant definitions, debates, and current empirical support. , 2004, Journal of clinical psychology.

[42]  Hyunmin Lee,et al.  Show Us You Are Real: The Effect of Human-Versus-Organizational Presence on Online Relationship Building Through Social Networking Sites , 2013, Cyberpsychology Behav. Soc. Netw..

[43]  Werner Wirth,et al.  The Development of Video Game Enjoyment in a Role Playing Game , 2013, Cyberpsychology Behav. Soc. Netw..

[44]  Robert M. Guion,et al.  Validity and Reliability , 2008 .

[45]  Paula M. Caligiuri,et al.  Testing a theoretical model for examining the relationship between family adjustment and expatriates' work adjustment. , 1998, The Journal of applied psychology.

[46]  N. Lackey,et al.  Making Sense of Factor Analysis: The Use of Factor Analysis for Instrument Development in Health Care Research , 2003 .

[47]  Daejoong Kim,et al.  The Integrated Model of Smartphone Adoption: Hedonic and Utilitarian Value Perceptions of Smartphones Among Korean College Students , 2012, Cyberpsychology Behav. Soc. Netw..

[48]  Andrea Gaggioli,et al.  From Psychotherapy to e-Therapy: The Integration of Traditional Techniques and New Communication Tools in Clinical Settings , 2003, Cyberpsychology Behav. Soc. Netw..

[49]  Allen I. Kraut,et al.  Organizational surveys : tools for assessment and change , 1996 .

[50]  Brian Hobbs,et al.  Developing a Virtual Reality-Based Methodology for People with Dementia: A Feasibility Study , 2003, Cyberpsychology Behav. Soc. Netw..

[51]  Yoojung Kim,et al.  Networking for Philanthropy: Increasing Volunteer Behavior via Social Networking Sites , 2014, Cyberpsychology Behav. Soc. Netw..

[52]  Carlo Galimberti,et al.  Three-Dimensional Virtual Environments for Cybertherapy: A Psychosocial Approach to Effective Usability , 2003, Cyberpsychology Behav. Soc. Netw..

[53]  Seth A. Kaplan,et al.  The benefits of flexible team interaction during crises. , 2009, The Journal of applied psychology.

[54]  S. Parker.  Content Analysis for the Social Sciences and Humanities , 1970 .

[55]  Elly A. Konijn,et al.  Cyberbullying Behavior and Adolescents' Use of Media with Antisocial Content: A Cyclic Process Model , 2014, Cyberpsychology Behav. Soc. Netw..

[56]  N. Lackey,et al.  Making Sense of Factor Analysis , 2003 .

[57]  Bruce H. Thomas,et al.  Virtual Reality as a Pediatric Pain Modulation Technique: A Case Study , 2003, Cyberpsychology Behav. Soc. Netw..

[58]  Linda Loos Scarth,et al.  The Corsini Encyclopedia of Psychology and Behavioral Science , 2002 .

[59]  Xiao Zhang,et al.  The Role of Self-Construal in Predicting Self-Presentational Motives for Online Social Network Use in the UK and Japan , 2014, Cyberpsychology Behav. Soc. Netw..