Description
Neural Oblivious Decision Ensembles (NODE) is a supplementary code repository for the research paper on Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data. This project focuses on learning deep ensembles of oblivious, differentiable decision trees specifically designed for tabular datasets. The core innovation lies in applying deep learning techniques to structured data, a domain traditionally dominated by gradient boosting machines and simpler tree-based models.
NODE's architecture allows for the creation of complex, ensemble-based models that can capture intricate relationships within tabular data. By utilizing differentiable decision trees, the system can be trained end-to-end using gradient descent, similar to standard neural networks. This approach offers a unique blend of the interpretability and structure of decision trees with the powerful learning capabilities of deep neural networks.
The project provides a Python implementation that can be run on machines with a CPU and preferably one or more GPUs. While running without a GPU is possible, it is significantly slower. The implementation is noted to be memory inefficient and may require substantial GPU memory for convergence. It is tested on Linux x64 distributions like Ubuntu 16.04 and is expected to work on other popular Linux distributions and macOS. For Windows and x32 systems, significant modifications might be necessary, and using Docker, especially a GPU-enabled version like nvidia-docker, is recommended.
To run NODE, users need to clone or download the repository, set up a Python environment (Anaconda is suggested), and install the required packages from `requirements.txt`. A critical dependency is `torch >= 1.1`. Users will also need Jupyter Notebook or a similar tool to work with the provided `.ipynb` files. Before executing the first cell in a notebook, the `CUDA_VISIBLE_DEVICES` environment variable must be set to the desired GPU index. The notebooks download datasets from Dropbox, requiring 1-5 GB of disk space. The library showcases typical learning scenarios for both classification and regression tasks, with detailed training information available in the original research paper.
Neural Oblivious Decision Ensembles Highlights
Deep learning on tabular data
Ensembles of oblivious differentiable decision trees
End-to-end training via gradient descent
Supports classification tasks
Supports regression tasks
GPU acceleration for faster training
Python-based implementation
Supplementary code for research paper
Requires PyTorch version 1.1 or later
Getting Started with Neural Oblivious Decision Ensembles
Clone or download repository: Obtain the project files from GitHub.
Set up Python environment: Create and activate a Python environment (e.g., using Anaconda).
Install dependencies: Run `pip install -r requirements.txt` to install necessary packages, ensuring PyTorch version is >= 1.1.
Configure GPU usage: Set the `CUDA_VISIBLE_DEVICES` environment variable to your desired GPU index before running notebooks.
Run notebooks: Open and execute the provided Jupyter notebooks in the `./notebooks/` directory.
Download datasets: Notebooks will automatically download required datasets (1-5GB).
Neural Oblivious Decision Ensembles's Use Cases
- Tabular Data Classification
- Tabular Data Regression
- Deep Learning Research
- Feature Engineering Alternatives
- Model Interpretability Research
