The success of foundation models in language has inspired a new wave of general-purpose models for human mobility. However, existing approaches struggle to scale effectively due to two fundamental limitations: a failure to use meaningful basic units to represent movement, and an inability to capture the vast diversity of patterns found in large-scale data. In this work, we develop MoveGPT, a large-scale foundation model specifically architected to overcome these barriers. MoveGPT is built upon two key innovations: (1) a unified location encoder that maps geographically disjoint locations into a shared semantic space, enabling pre-training on a global scale; and (2) a Spatially-Aware Mixture-of-Experts Transformer that develops specialized experts to efficiently capture diverse mobility patterns. Pre-trained on billion-scale datasets, MoveGPT establishes a new state-of-the-art across a wide range of downstream tasks, achieving performance gains of up to 35% on average. It also demonstrates strong generalization capabilities to unseen cities. Crucially, our work provides empirical evidence of scaling ability in human mobility, validating a clear path toward building increasingly capable foundation models in this domain.
翻译:基础模型在语言领域的成功激发了面向人类移动性的新一代通用模型的发展。然而,现有方法因两个根本性局限而难以有效扩展:一是未能使用有意义的基本单元来表示移动行为,二是无法捕捉大规模数据中存在的巨大多样性模式。本研究开发了MoveGPT,这是一个专门设计用于突破这些障碍的大规模基础模型。MoveGPT基于两项关键创新构建:(1)统一位置编码器,将地理上离散的位置映射到共享语义空间,实现全球尺度的预训练;(2)空间感知专家混合Transformer,通过训练专业化专家模块高效捕捉多样化的移动模式。基于十亿级数据集预训练的MoveGPT在广泛的下游任务中确立了新的最优性能,平均性能提升最高达35%。该模型还展现出对未见城市的强大泛化能力。重要的是,我们的工作为人类移动性领域的扩展能力提供了实证依据,验证了在该领域构建更强大基础模型的清晰路径。