AI for social science and social science of AI: A Survey

Recent advancements in artificial intelligence, particularly with the emergence of large language models (LLMs), have sparked a rethinking of artificial general intelligence possibilities. The increasing human-like capabilities of AI are also attracting attention in social science research, leading to various studies exploring the combination of these two fields. In this survey, we systematically categorize previous explorations in the combination of AI and social science into two directions that share common technical approaches but differ in their research objectives. The first direction is focused on AI for social science, where AI is utilized as a powerful tool to enhance various stages of social science research. While the second direction is the social science of AI, which examines AI agents as social entities with their human-like cognitive and linguistic capabilities. By conducting a thorough review, particularly on the substantial progress facilitated by recent advancements in large language models, this paper introduces a fresh perspective to reassess the relationship between AI and social science, provides a cohesive framework that allows researchers to understand the distinctions and connections between AI for social science and social science of AI, and also summarized state-of-art experiment simulation platforms to facilitate research in these two directions. We believe that as AI technology continues to advance and intelligent agents find increasing applications in our daily lives, the significance of the combination of AI and social science will become even more prominent.

[1]  Jia Liu,et al.  Body size as a metric for the affordable world , 2023, bioRxiv.

[2]  Mrinmaya Sachan,et al.  Agents: An Open-source Framework for Autonomous Language Agents , 2023, ArXiv.

[3]  Wayne Xin Zhao,et al.  A Survey on Large Language Model based Autonomous Agents , 2023, Frontiers Comput. Sci..

[4]  Valdemar Pinho Neto,et al.  More human than human: measuring ChatGPT political bias , 2023, SSRN Electronic Journal.

[5]  Qianqian Xie,et al.  PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance , 2023, ArXiv.

[6]  Justin F. Rousseau,et al.  Less Likely Brainstorming: Using Language Models to Generate Alternative Hypotheses , 2023, ACL.

[7]  Chan Young Park,et al.  From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models , 2023, ACL.

[8]  Leland Bybee.  Surveying Generative AI's Economic Expectations , 2023, SSRN Electronic Journal.

[9]  Guijin Son,et al.  Beyond Classification: Financial Reasoning in State-of-the-Art Language Models , 2023, FINNLP.

[10]  Xiaozhi Wang,et al.  ChatLog: Recording and Analyzing ChatGPT Across Time , 2023, ArXiv.

[11]  Yuehua Tang,et al.  Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models , 2023, SSRN Electronic Journal.

[12]  Markus Pauly,et al.  The Self-Perception and Political Biases of ChatGPT , 2023, Human Behavior and Emerging Technologies.

[13]  Omar Shaikh,et al.  Can Large Language Models Transform Computational Social Science? , 2023, CL.

[14]  Qianqian Xie,et al.  The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges , 2023, ArXiv.

[15]  Michael S. Bernstein,et al.  Generative Agents: Interactive Simulacra of Human Behavior , 2023, UIST.

[16]  K. Chamari,et al.  From human writing to artificial intelligence generated text: examining the prospects and potential threats of ChatGPT in academic writing. , 2023, Biology of sport.

[17]  Haowei Xu,et al.  Can ChatGPT be used to generate scientific hypotheses? , 2023, Journal of Materiomics.

[18]  Percy Liang,et al.  Whose Opinions Do Language Models Reflect? , 2023, ICML.

[19]  Bishal Lamichhane.  Evaluation of ChatGPT for NLP-based Mental Health Applications , 2023, ArXiv.

[20]  Jacob Andreas,et al.  Language Models Trained on Media Diets Can Predict Public Opinion , 2023, ArXiv.

[21]  Meysam Alizadeh,et al.  ChatGPT outperforms crowd workers for text-annotation tasks , 2023, Proceedings of the National Academy of Sciences of the United States of America.

[22]  S. S. Gill,et al.  Mind meets machine: Unravelling GPT-4's cognitive psychology , 2023, BenchCouncil Transactions on Benchmarks, Standards and Evaluations.

[23]  Steffen Staab,et al.  The Role of Large Language Models in the Recognition of Territorial Sovereignty: An Analysis of the Construction of Legitimacy , 2023, ArXiv.

[24]  Kimberly T. Mai,et al.  Susceptibility to Influence of Large Language Models , 2023, ArXiv.

[25]  C. Miao,et al.  Can ChatGPT Assess Human Personalities? A General Evaluation Framework , 2023, EMNLP.

[26]  Margarita Rodríguez-Ibáñez,et al.  A review on sentiment analysis from social media platforms , 2023, Expert Syst. Appl..

[27]  C. Bylund,et al.  Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information , 2023, JNCI cancer spectrum.

[28]  Tzeng-Ji Chen.  ChatGPT and other artificial intelligence applications speed up scientific writing. , 2023, Journal of the Chinese Medical Association : JCMA.

[29]  Haewoon Kwak,et al.  Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech , 2023, WWW.

[30]  B. Koopman,et al.  Can ChatGPT Write a Good Boolean Query for Systematic Review Literature Search? , 2023, SIGIR.

[31]  J. Horton.  Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus? , 2023, SSRN Electronic Journal.

[32]  D. Katz,et al.  GPT as Knowledge Worker: A Zero-Shot Evaluation of (AI)CPA Capabilities , 2023, SSRN Electronic Journal.

[33]  Bowen Zhang,et al.  How would Stance Detection Techniques Evolve after the Launch of ChatGPT? , 2022, ArXiv.

[34]  Shafiq R. Joty,et al.  Does GPT-3 Demonstrate Psychopathy? Evaluating Large Language Models from a Psychological Perspective , 2022, 2212.10529.

[35]  U. Kang,et al.  Accurate Stock Movement Prediction with Self-supervised Learning from Sparse Noisy Tweets , 2022, 2022 IEEE International Conference on Big Data (Big Data).

[36]  Markus Leippold.  Thus Spoke GPT-3: Interviewing a Large-Language Model on Climate Finance , 2022, SSRN Electronic Journal.

[37]  Alexander M. Rush,et al.  BLOOM: A 176B-Parameter Open-Access Multilingual Language Model , 2022, ArXiv.

[38]  Ronan Le Bras,et al.  Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs , 2022, EMNLP.

[39]  William Yang Wang,et al.  ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering , 2022, EMNLP.

[40]  B. Schölkopf,et al.  When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment , 2022, NeurIPS.

[41]  Bennett Kleinberg,et al.  Who is GPT-3? An exploration of personality, values and demographics , 2022, NLPCSS.

[42]  Doug Beeferman,et al.  CommunityLM: Probing Partisan Worldviews from Language Models , 2022, COLING.

[43]  Haewoon Kwak,et al.  Chain of Explanation: New Prompting Method to Generate Quality Natural Language Explanation for Implicit Hate Speech , 2022, WWW.

[44]  A. Kalai,et al.  Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies , 2022, ICML.

[45]  Michael S. Bernstein,et al.  Social Simulacra: Creating Populated Prototypes for Social Computing Systems , 2022, UIST.

[46]  Kazi Sinthia Kabir,et al.  A Disability Lens towards Biases in GPT-3 Generated Open-Ended Languages , 2022, ArXiv.

[47]  Jessica B. Hamrick,et al.  Towards Understanding How Machines Can Learn Causal Overhypotheses , 2022, ArXiv.

[48]  R. Grasman,et al.  Putting GPT-3's Creativity to the (Alternative Uses) Test , 2022, ICCC.

[49]  Martine B. Powell,et al.  Synthesizing a Talking Child Avatar to Train Interviewers Working with Maltreated Children , 2022, Big Data Cogn. Comput..

[50]  Debanjan Ghosh,et al.  FLUTE: Figurative Language Understanding through Textual Explanations , 2022, EMNLP.

[51]  J. Tenenbaum,et al.  Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks , 2022, ArXiv.

[52]  H. A. Schwartz,et al.  Natural language analyzed with AI-based transformers predict traditional subjective well-being measures approaching the theoretical upper limits in accuracy , 2021, Scientific Reports.

[53]  Ryan J. Lowe,et al.  Training language models to follow instructions with human feedback , 2022, NeurIPS.

[54]  J. Steinhardt,et al.  Capturing Failures of Large Language Models via Human Cognitive Biases , 2022, NeurIPS.

[55]  Mai ElSherief,et al.  Latent Hatred: A Benchmark for Understanding Implicit Hate Speech , 2021, EMNLP.

[56]  Li Lucy,et al.  Gender and Representation Bias in GPT-3 Generated Stories , 2021, NUSE.

[57]  Matthew Louis Mauriello,et al.  SAD: A Stress Annotated Dataset for Recognizing Everyday Stressors in SMS-like Conversational Systems , 2021, CHI Extended Abstracts.

[58]  Soroush Vosoughi,et al.  Mitigating Political Bias in Language Models Through Reinforced Calibration , 2021, AAAI.

[59]  Yejin Choi,et al.  Misinfo Reaction Frames: Reasoning about Readers’ Reactions to News Headlines , 2021, ACL.

[60]  D. Klein,et al.  Calibrate Before Use: Improving Few-Shot Performance of Language Models , 2021, ICML.

[61]  Laurel Smith‐Doerr,et al.  Toward a Sociology of Artificial Intelligence: A Call for Research on Inequalities and Structural Change , 2021, Socius: Sociological Research for a Dynamic World.

[62]  Ankur Sinha,et al.  Impact of News on the Commodity Market: Dataset and Results , 2020, Advances in Intelligent Systems and Computing.

[63]  Mark Chen,et al.  Language Models are Few-Shot Learners , 2020, NeurIPS.

[64]  Kathleen McKeown,et al.  Dreaddit: A Reddit Dataset for Stress Analysis in Social Media , 2019, EMNLP.

[65]  Peter J. Liu,et al.  Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer , 2019, J. Mach. Learn. Res..

[66]  Guangxu Xun,et al.  Hypothesis Generation From Text Based On Co-Evolution Of Biomedical Concepts , 2019, KDD.

[67]  A. Zeilinger,et al.  Predicting research trends with semantic and neural networks with an application in quantum physics , 2019, Proceedings of the National Academy of Sciences.

[68]  Iyad Rahwan,et al.  The evolution of citation graphs in artificial intelligence research , 2019, Nature Machine Intelligence.

[69]  Wei Zhang,et al.  Hybrid Deep Sequential Modeling for Social Text-Driven Stock Prediction , 2018, CIKM.

[70]  Çağrı Çöltekin,et al.  Identifying Depression on Reddit: The Effect of Training Data , 2018, EMNLP 2018.

[71]  Angela D. Wilkins,et al.  Automated literature mining and hypothesis generation through a network of Medical Subject Headings , 2018, bioRxiv.

[72]  Shay B. Cohen,et al.  Stock Movement Prediction from Tweets and Historical Prices , 2018, ACL.

[73]  Dario Amodei,et al.  AI safety via debate , 2018, ArXiv.

[74]  Emiel Krahmer,et al.  Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation , 2017, J. Artif. Intell. Res..

[75]  Saif Mohammad,et al.  SemEval-2016 Task 6: Detecting Stance in Tweets , 2016, *SEMEVAL.

[76]  Randy S Hebert,et al.  The Stanford Prison Experiment , 2016, The American journal of hospice & palliative care.

[77]  Mark Dredze,et al.  Shared Task : Depression and PTSD on Twitter , 2015 .

[78]  Philip Resnik,et al.  Political Ideology Detection Using Recursive Neural Networks , 2014, ACL.

[79]  Pekka Korhonen,et al.  Good debt or bad debt: Detecting semantic orientations in economic texts , 2013, J. Assoc. Inf. Sci. Technol..

[80]  Charu C. Aggarwal,et al.  A Survey of Text Classification Algorithms , 2012, Mining Text Data.

[81]  Anol Bhattacherjee.  Social Science Research: Principles, Methods, and Practices , 2012 .

[82]  Ben Goertzel,et al.  Artificial General Intelligence: Concept, State of the Art, and Future Prospects , 2009, J. Artif. Gen. Intell..

[83]  H. Klein,et al.  The Social Construction of Technology: Structural Considerations , 2002 .

[84]  Hillel J. Kieval.  Pursuing the Golem of Prague: Jewish Culture and the Invention of a Tradition , 1997 .

[85]  S. Woolgar.  Why not a Sociology of Machines? The Case of Sociology and Artificial Intelligence , 1985 .

[86]  J. Wallwork.  Language and linguistics , 1981, Language Teaching.

[87]  B. R. Pollin.  Philosophical and Literary Sources of Frankenstein , 1965 .

[88]  D. Easton,et al.  The Political System: An Inquiry into the State of Political Science. , 1953 .

[89]  G. Catlin.  An Introduction to Sociology , 1942, Nature.

[90]  B. McCarl,et al.  Economics , 1870, The Indian medical gazette.

[91]  Kadir Uludag.  ChatGPT Can Distinguish Paranoid Thoughts in Patients with Schizophrenia , 2023, SSRN Electronic Journal.

[92]  Meena Andiappan,et al.  A Manager and an AI Walk into a Bar: Does ChatGPT Make Biased Decisions Like We Do? , 2023, SSRN Electronic Journal.

[93]  Donald Ngwe,et al.  Using GPT for Market Research , 2023, SSRN Electronic Journal.

[94]  K. Kalinin.  Geopolitical Forecasting Analysis of the Russia-Ukraine War Using the Expert's Survey, Predictioneer's Game and GPT-3 , 2023, SSRN Electronic Journal.

[95]  Robert W. McGee.  Is Chat Gpt Biased Against Conservatives? An Empirical Study , 2023, SSRN Electronic Journal.

[96]  S. Ananiadou,et al.  On the Evaluations of ChatGPT and Emotion-enhanced Prompting for Mental Health Analysis , 2023, ArXiv.

[97]  Paweł Niszczota,et al.  Gpt as a Financial Advisor , 2023, SSRN Electronic Journal.

[98]  Timothy Baldwin,et al.  Domain Adaption of Named Entity Recognition to Support Credit Risk Assessment , 2015, ALTA.

[99]  Robert S. Pindyck,et al.  Microeconomics , 2014 .

[100]  Richard A. Demers,et al.  A linguistics workbook : companion to linguistics , 2010 .

[101]  David Willer,et al.  Building Experiments: Testing Social Theory , 2007 .

[102]  Nicholas Walliman,et al.  Social research methods , 2006 .

[103]  Cli McMahon,et al.  Machines Who Think : A Personal Inquiry into the History and Prospects of Artificial Intelligence , 2004 .

[104]  B. Chadwick,et al.  Social Science Research Methods , 1984 .

[105]  Todd Donovan,et al.  The Elements of Social Scientific Thinking , 1980 .