PKU A-LAB · AI for Social Science Series

Generative AI

• Artificial Intelligence (AI), particularly Generative AI, is rapidly penetrating every aspect of the world, including social science research.
• Generative AI: Deep-learning models that can generate high-quality text, images, and other content based on the data they were trained on.
  • Text: ChatGPT, Gemini, LLaMA, Claude, Bloom, Jasper,….
  • Image: DALL-E, Midjourney, Stable Diffusion, Imagen, Parti, ….
  • Speech: Vertex AI, ElevenLabs, Murf AI, Resemble AI, ….
  • Video: Sora, Synthesia, Make-A-Video, Phenaki, Amazon Rekognition, ….
  • Data: SMOTE, ADASYN, Augmenter, ….
• AI tools and AI-generated data are increasingly being integrated into social science research.

GPT in Political Science

Overview

• Questions:
  • How generative AIs can assist and extend social science research?
  • How we can properly use generative AIs for social science research?
• Today
  • Fundementals of Generative AI
  • Simulation
  • Verification
  • Interaction
• Disclosure

How Does AI Become So Good?

• It didn’t happen overnight.
• There are three major breakthroughs worth highlighting:
  • Neural network
  • Transformer
  • Training and tuning

Neural Network

• The Statistical Paradigm Ante 1990
  • Pick a functional form based on domain and nature of the data.
  • Find parameters that (best) fit the data.
  • Gain understanding from examining those parameters

Neural Network

• Neural networks are flexible predictors
  • Neural networks are highly parameterized functions.
  • We can get any other function we want by setting the neural net parameters the right way.
  • Rather, we put data into the neural net, compare the function it represents with the one the data implicitly represents, and update the parameters.
  • Neural networks are built hierarchically, and plug together like Lego blocks.
  • In particular, neural nets are built compositionally.
    • Depth = levels of compositionality.

Neural Network

• E.g., the most advanced GPT-3 model was trained in a model with 170B parameters and 96 layers.

Neural Network

• Neural Netowrok Playground (https://playground.tensorflow.org/)