Description
Vision GNN (ViG) represents a significant advancement in applying Graph Neural Networks (GNNs) to computer vision tasks. Developed by Huawei Noah's Ark Lab, this project provides PyTorch implementations of efficient AI backbones, focusing on the ViG and Pyramid ViG architectures. These models treat images as graphs of nodes, enabling novel approaches to feature extraction and representation learning.
Read More
The project includes pre-trained models for various ViG configurations, such as ViG-Ti, ViG-S, ViG-B, Pyramid ViG-Ti, Pyramid ViG-S, Pyramid ViG-M, and Pyramid ViG-B. These models are benchmarked by their parameter count (M) and FLOPs (B), alongside their Top-1 accuracy on image classification tasks. For instance, ViG-Ti boasts 7.1M parameters and 1.3B FLOPs with a 73.9% Top-1 accuracy, while Pyramid ViG-B offers 82.6M parameters and 16.8B FLOPs with an 83.7% Top-1 accuracy.
Detailed instructions are provided for data preparation, evaluation, and training. Users can evaluate models using provided scripts, and training commands are outlined for both ViG and Pyramid ViG on datasets like ImageNet. The training commands specify hyperparameters such as learning rate, optimizer, epochs, and augmentation techniques, allowing for customization and further experimentation. The repository also includes links to download pre-trained models, with some requiring specific passwords for access.
ViG is built upon PyTorch 1.7.0 and requires libraries like timm 0.3.2, torchprofile 0.0.4, and apex. The project acknowledges its partial use of code from deep_gcns_torch and timm. This initiative aims to push the boundaries of vision models by leveraging graph-based representations, offering a powerful alternative to traditional convolutional neural networks for a range of computer vision applications.
- CategoryAI Models & LLMs
- Pricing ModelOpen source
- DeploymentCloud, Self-hosted
- PlatformsWeb
Vision GNN (ViG)'s Core Features
PyTorch implementation of Vision GNN (ViG) and Pyramid ViG architectures
Includes pre-trained models for various ViG configurations
Provides model performance metrics (Parameters, FLOPs, Top-1 Accuracy)
Offers training scripts for ViG and Pyramid ViG models
Supports ImageNet dataset for evaluation and training
Requires PyTorch 1.7.0, timm 0.3.2, torchprofile 0.0.4, and apex
Codebase partially utilizes deep_gcns_torch and timm
Models treat images as graphs of nodes for feature learning
Includes detailed instructions for data preparation and evaluation
Downloadable pre-trained models available with some password protection
Getting Started with Vision GNN (ViG)
Access Model: Clone the GitHub repository to your local machine.
Set up Environment: Install required libraries including PyTorch 1.7.0, timm, torchprofile, and apex.
Prepare Data: Organize your image dataset (e.g., ImageNet) according to the specified directory structure.
Evaluate Model: Utilize the provided `train.py` script with the `--evaluate` flag and path to pre-trained weights.
Train Model: Execute the training commands for ViG or Pyramid ViG, specifying model, dataset, and training hyperparameters.
Integrate Model: Load pre-trained weights into your PyTorch model for inference or fine-tuning on custom tasks.
Vision GNN (ViG)'s Use Cases
- Image Classification
- Feature Extraction
- Computer Vision Research
- Model Development
- Transfer Learning
