Description
TabFM 1.0.0 (PyTorch) is a zero-shot tabular foundation model developed by Google Research. It is designed to handle classification and regression tasks on structured data, specifically tabular data with mixed numerical and categorical columns. The model operates without the need for fine-tuning or hyperparameter search, allowing users to pass training examples as context and make predictions in a single forward pass.
Read More
The model architecture employs alternating row and column attention to capture feature interactions and row-level patterns. Column attention uses Set Transformer to embed each cell with Fourier features and aggregates across rows via induced self-attention. Row compression is achieved through CLS tokens that summarize each row into a dense vector using row-level attention with Rotary Position Embedding (RoPE). The ICL Transformer, a 24-block causal transformer, operates over these compressed row vectors, treating training rows as context and outputting predictions for test rows.
TabFM is optimized for binary and multiclass classification (up to 10 classes) and regression on continuous targets. It supports data in the form of pandas DataFrames or numpy arrays. The model was trained on hundreds of millions of synthetic datasets generated using structural causal models (SCMs), which encode inductive biases about causal structures typical in tabular tasks.
Performance evaluations on TabArena across 51 datasets show that TabFM outperforms heavily-tuned supervised baselines, including gradient-boosted trees, in zero-shot mode. However, it has limitations, such as a maximum of 10 classes for classification and memory usage that scales with the number of training rows. It is not intended for commercial use, images, audio, video, raw text, or non-tabular structured data.
The model weights are released under the TabFM Non-Commercial License v1.0, and the source code is available under the Apache 2.0 license.
- CategoryAI Models & LLMs
- Pricing ModelOpen source
- DeploymentCloud
- PlatformsWeb
TabFM 1.0.0 (PyTorch) Highlights
Zero-shot tabular model
Supports classification and regression
Handles mixed numerical and categorical data
No fine-tuning required
Alternating row and column attention
Trained on synthetic datasets
Maximum 10 classes for classification
Non-commercial license
Getting Started with TabFM 1.0.0 (PyTorch)
Access model: Visit Hugging Face page
Authenticate: Use Hugging Face credentials
Set up environment: Install PyTorch
Integrate via API: Use Hugging Face Hub API
Optimize: Evaluate on representative data
TabFM 1.0.0 (PyTorch)'s Use Cases
- Binary Classification
- Multiclass Classification
- Regression Analysis
- DataFrame Integration
- Numpy Array Support
