Description
This repository serves as the official implementation for the research paper "Revisiting Deep Learning Models for Tabular Data," presented at NeurIPS 2021. The project delves into the effectiveness of various deep learning architectures when applied to tabular datasets, a domain where traditional methods like gradient-boosted decision trees (GBDTs) often excel.
Read More
The core findings of the paper, as reflected in this repository, indicate that simple, well-tuned Multi-Layer Perceptrons (MLPs) remain highly competitive, often performing on par with or even surpassing more complex deep learning models. The ResNet architecture, an MLP variant incorporating skip connections and batch normalization, further reinforces the strength of MLP-like structures for tabular data. However, the research also introduces the FT-Transformer, a novel architecture that adapts the Transformer model for tabular data. FT-Transformer demonstrates superior average performance across benchmarks compared to other deep models and significantly narrows the performance gap between deep learning and GBDTs on datasets where GBDTs traditionally dominate.
The repository is structured to facilitate reproduction of the paper's results and further research. It includes a Python package for practical use, detailed metrics and hyperparameters for various models and datasets, and clear instructions on setup, data download, and execution of tuning, evaluation, and ensembling scripts. The code is organized into directories for training, ensembling, tuning, analysis, and common tools, with outputs including detailed statistics and configurations.
This project is valuable for researchers and practitioners in machine learning, particularly those working with tabular data. It provides a robust framework for experimenting with and deploying deep learning models, offering insights into which architectures perform best and under what conditions. The inclusion of FT-Transformer suggests a promising direction for advancing deep learning on structured data.
- CategoryAI Models & LLMs
- Pricing ModelOpen source
- DeploymentCloud, Self-hosted
- PlatformsWeb
Deep Learning Models for Tabular Data's Core Features
Official implementation of NeurIPS 2021 paper 'Revisiting Deep Learning Models for Tabular Data'
Explores MLP-like models as strong baselines for tabular data
Introduces FT-Transformer, an adaptation of the Transformer architecture for tabular data
Provides Python package for practical application and future research
Includes detailed metrics and tuned hyperparameters for various models and datasets
Codebase for reproducing reported results
Scripts for hyperparameter tuning, model evaluation, and ensembling
Supports PyTorch and TensorFlow environments for experimentation
Facilitates comparison between deep learning models and GBDTs on tabular datasets
Offers insights into the performance characteristics of different deep learning architectures on tabular data
Getting Started with Deep Learning Models for Tabular Data
Set up environment: Install necessary dependencies using Conda and pip.
Download data: Obtain the dataset archive and unpack it into the repository root.
Reproduce results: Follow tutorial steps for tuning, evaluation, and ensembling.
Run scripts: Execute Python scripts from the repository root, providing configuration files.
Integrate via API: Utilize the Python package for practical application.
Explore metrics: Analyze `stats.json` files to understand model performance.
Configure models: Modify TOML configuration files for custom experiments.
Deep Learning Models for Tabular Data's Use Cases
- Tabular Data Analysis
- Model Benchmarking
- Research Implementation
- Feature Engineering Exploration
- Predictive Modeling
