publications

publications by categories in reversed chronological order.

2025

  1. grammaticality-probability.jpeg
    What Can String Probability Tell Us About Grammaticality?
    Jennifer Hu, Ethan Gotlieb Wilcox, Siyuan Song, Kyle Mahowald, and Roger P. Levy
    to appear in TACL, 2025
    arXiv PDF
  2. babybabelmap.jpg
    BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data
    Jaap Jumelet, Abdellah Fourtassi, Akari Haga, Bastian Bunzeck, Bhargav Shandilya, Diana Galvan-Sosa, Faiz Ghifari Haznitrama, Francesca Padovani, Francois Meyer, Hai Hu, Julen Etxaniz, Laurent Prévot, Linyang He, María Grandury, Mila Marcheva, Negar Foroutan, Nikitas Theodoropoulos, Pouya Sadeghi, Siyuan Song, Suchir Salhan, Susana Zhou, Yurii Paniv, Ziyin Zhang, Arianna Bisazza, Alex Warstadt, and Leshem Choshen
    2025
    arXiv PDF Website
  3. introspection2.png
    Privileged Self-Access Matters for Introspection in AI
    Siyuan Song, Harvey Lederman , Jennifer Hu, and Kyle Mahowald
    arXiv preprint, 2025
    arXiv PDF Code
  4. zhoblimp.png
    ZhoBLiMP: a Systematic Assessment of Language Models with Linguistic Minimal Pairs in Chinese
    Yikang Liu, Yeting Shen, Hongao Zhu, Lilong Xu, Zhiheng Qian, Siyuan Song, Kejia Zhang, Jialong Tang, Pei Zhang, Baosong Yang, Rui Wang, and Hai Hu
    to appear in TACL, 2025
    arXiv PDF Code
  5. introspectionfig1.jpeg
    Language Models Fail to Introspect About Their Knowledge of Language
    Siyuan Song , Jennifer Hu, and Kyle Mahowald
    COLM, 2025
    arXiv PDF Code

2024

  1. wlwz.webp
    Do large language models understand conversational implicature–a case study with a Chinese sitcom
    Shisen Yue, Siyuan Song, Xinyuan Cheng, and Hai Hu
    In China National Conference on Chinese Computational Linguistics (Highlight Paper), 2024
    arXiv PDF Code