TwiBot-22: Towards Graph-Based Twitter Bot Detection
Twitter bot detection has become an increasingly important task to combat misinformation, facilitate social media moderation, and preserve the integrity of the online discourse. State-of-the-art bot detection methods generally leverage the graph structure of the Twitter network, and they exhibit promising performance when confronting novel Twitter bots that traditional methods fail to detect.
However, very few of the existing Twitter bot detection datasets are graph-based, and even these few graph-based datasets suffer from limited dataset scale, incomplete graph structure, as well as low annotation quality. In fact, the lack of a large-scale graph-based Twitter bot detection benchmark that addresses these issues has seriously hindered the development and evaluation of novel graph-based bot detection approaches.
In this paper, we propose TwiBot-22, a comprehensive graph-based Twitter bot detection benchmark that presents the largest dataset to date, provides diversified entities and relations on the Twitter network, and has considerably better annotation quality than existing datasets.
In addition, we re-implement 35 representative Twitter bot detection baselines and evaluate them on 9 datasets, including TwiBot-22, to promote a fair comparison of model performance and a holistic understanding of research progress.
To facilitate further research, we consolidate all implemented codes and datasets into the TwiBot-22 evaluation framework, where researchers could consistently evaluate new models and datasets.
TwiBot-22 Paper » TwiBot-22 Dataset and Code » TwiBot-20 Paper » TwiBot-20 Repo » Previous Papers »TwiBot-22 Paper:
@misc{https://doi.org/10.48550/arxiv.2206.04564,
doi = {10.48550/ARXIV.2206.04564},
url = {https://arxiv.org/abs/2206.04564},
author = {Feng, Shangbin and Tan, Zhaoxuan and Wan, Herun and Wang, Ningnan and Chen, Zilong and Zhang, Binchi and Zheng, Qinghua and Zhang, Wenqian and Lei, Zhenyu and Yang, Shujie and Feng, Xinshun and Zhang, Qingyue and Wang, Hongrui and Liu, Yuhan and Bai, Yuyang and Wang, Heng and Cai, Zijian and Wang, Yanbo and Zheng, Lijing and Ma, Zihan and Li, Jundong and Luo, Minnan},
keywords = {Social and Information Networks (cs.SI), Artificial Intelligence (cs.AI), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {TwiBot-22: Towards Graph-Based Twitter Bot Detection},
publisher = {arXiv},
year = {2022},
copyright = {arXiv.org perpetual, non-exclusive license}
}
What's next?
How do we identify bot clusters and their coordination campaigns?
While existing works study Twitter bot detection through individual analysis, novel Twitter bots increasingly act in groups and launch coordinated attacks. We aim to complement the scarce literature by proposing temporal and subgraph-level bot detection approaches to address this issue.
How do we incorporate multi-modal user features for bot detection?
In addition to text and graph, Twitter users and bots generate multi-modal user information such as images and videos. Since TwiBot-22 provides user media while none of the 35 baselines leverage these modalities, we aim to further explore Twitter bot detection with the help of images and videos.
How do we evaluate the generalization ability of bot detection methods?
Existing works mainly focus on bot detection performance while generalization is essential in ensuring that bot detection research generates real-world impact. We aim to complement the scarce literature by proposing measures to quantitatively evaluate bot detection generalization.
Authors
This work was done by a team of reseachers at Xi'an Jiaotong University, specifically in the LUD Lab.
Leading Authors
Project Members
Project Advisors
Contact
Questions about Twitter bot detection, or want to get in touch? Contact Shangbin at bunsenfeng.github.io.
Design and CSS courtesy: Rowan Zellers
