← Back to all models Foundation Model

scGPT

Cui H, Wang C, Maan H, Pang K, Luo F, Duan N, Wang B · 2024-02-26 · Nature Methods

Generative pre-trained transformer for single-cell biology. Pre-trained on 33M+ cells from CELLxGENE.

Overview

scGPT is a foundation model for single-cell biology built upon a generative pre-trained transformer architecture. It is trained on over 33 million human cells from the CELLxGENE database and can be fine-tuned for various downstream tasks including cell type annotation, gene perturbation prediction, multi-omics integration, and batch correction.

Publication

scGPT: Toward Building a Foundation Model for Single-Cell Multi-Omics Using Generative AI

DOI: 10.1038/s41592-024-02201-0

Links

📄 Read Paper 💻 GitHub

Specifications

  • ArchitectureTransformer (GPT-style)
  • Parameters~300M
  • Pretraining DataCELLxGENE (33M cells)
  • ModalityscRNA-seq

Tags