Skip to main content
ToolPotion
AI Advisor
Submit a Tool (Free)

Forms That Read Themselves

AI AppsComputer Vision ToolsGPU Rental Cost · Web

Forms That Read Themselves presents a small, fine-tuned vision model for scanned form extraction, benchmarked against frontier vision APIs and Azure Document Intelligence. It demonstrates that a self-owned model can achieve comparable accuracy to leading commercial solutions on various form types, including ad-buy forms and receipts, at a significantly lower cost, while offering data privacy benefits.

Visit Website
Share
Forms That Read Themselves screenshot
Forms That Read Themselves screenshot

Forms That Read Themselves introduces a comprehensive evaluation of twelve different readers for scanned forms, ranging from state-of-the-art vision APIs to a custom 2B parameter model fine-tuned specifically for this purpose. The core offering is a small vision model that users can own and operate, providing a cost-effective alternative to commercial solutions like Azure Document Intelligence.

Read More

The platform meticulously grades each reader field by field on public datasets with known answers, including NIST Special Database 2 for 1988 US tax forms, Foreign Agents Registration Act filings, FCC political ad-buy orders, and CORD v2 receipts. This rigorous benchmarking reveals that the fine-tuned model outperforms Azure Document Intelligence on half of the tested form types and matches its performance on the remainder, all while operating at a fraction of the price. For instance, on ad-buy forms, the fine-tuned model achieves 59.6% accuracy against Azure's 40.5%, and on receipts, 77.6% against 66.6%.

A key finding is the transformative impact of fine-tuning on smaller models. Untuned models like Gemma 4 E2B show significantly lower accuracy (e.g., 15.0% on tax forms), which dramatically improves to 73.6% after fine-tuning. This process, which can take as little as 8 minutes to 4.9 hours on a rented GPU, makes these smaller models highly competitive. The fine-tuned model also demonstrates a lower rate of inventing values compared to frontier models on certain form types, such as registration forms and receipts, which is crucial for applications where a blank field is preferable to a confident but incorrect entry.

While frontier vision models like GPT-6.1 Sol and Gemini 3.8 Flash generally achieve higher accuracy, the fine-tuned model offers a compelling balance of performance and cost. For example, on receipts, it achieves 77.6% accuracy for US$0.16 per thousand pages, compared to Claude Haiku 5.5's 76.7% for US$0.38. A significant advantage of owning and running the model on a consumer GPU is the enhanced data privacy, as no pages need to leave the user's network.

The project also highlights the robustness of vision models against scan degradation. Unlike OCR-based pipelines, which suffer drastic accuracy drops with poor scan quality, vision models reading pixels directly maintain high performance. However, it's important to note that no reader achieved 99% accuracy on all fields, meaning human verification remains necessary for critical applications. The models primarily automate the typing process, not the final checking.

The architecture involves training open-weight Gemma 4 E2B and E4B models using QLoRA, freezing base model weights in 4 bits while training small adapter matrices. These adapters are served with vLLM on an RTX 3090 GPU, allowing for efficient, on-premise deployment. The site provides a calculator to determine the break-even point between API costs and GPU rental, emphasizing that the true cost of an owned model depends on its utilization.

  • CategoryComputer Vision Tools
  • Pricing ModelGPU Rental Cost
  • DeploymentOn-Premise
  • PlatformsWeb

  • Fine-tuned vision models for scanned form extraction

  • Benchmarking against frontier vision APIs (GPT-6.1 Sol, Claude Sonnet, Gemini Flash)

  • Comparison with Azure Document Intelligence

  • Support for various form types: registration, ad-buy, receipts, tax forms

  • Field-by-field accuracy grading on public datasets

  • Cost-effective alternative to commercial APIs

  • On-premise deployment capability on consumer GPUs (e.g., RTX 3090)

  • Reduced invention of values on certain form types

  • Robustness against scan degradation (noise, blur, skew, lost contrast)

  • Detailed performance metrics including fields correct and cost per thousand pages

  • Open-weight base models (Gemma 4 E2B, E4B) used for fine-tuning

  • QLoRA fine-tuning recipe for efficient training on single GPUs

  • vLLM for serving fine-tuned models

  • Calculator for API vs. owned GPU cost break-even analysis

  • Data privacy by keeping pages within your network

  1. Select a Reader: Choose from various vision models or OCR pipelines to compare their performance.

  2. Adjust Scan Quality: Simulate different degradation levels (noise, blur, skew, lost contrast) to observe impact on accuracy.

  3. Review Field-by-Field Results: See which fields each reader correctly extracts, marked green, and which are incorrect, marked red and dashed.

  4. Analyze Accuracy and Cost: Compare the share of fields right and the cost per thousand pages for different readers and form types.

  5. Understand Fine-tuning Impact: Observe how fine-tuning significantly improves the accuracy of smaller models on specific form types.

  6. Evaluate Deployment Costs: Use the provided calculator to determine the economic viability of renting a GPU versus using commercial APIs based on your volume.

  7. Access Adapters: Download available fine-tuned adapters for receipts, ad-buy forms, and tax forms from Hugging Face.

  • Automated Form Data Extraction
  • Cost-Effective Document Processing
  • On-Premise Data Processing
  • Benchmarking AI Vision Models
  • Improving OCR Accuracy
  • Custom Model Development

Forms That Read Themselves Reviews

Loading...

Popular AI Tools Like Forms That Read Themselves

Azure Document Intelligence, part of Azure Content Understanding in Foundry Tools, automates the extraction of text, structure, and fields from various documents using AI models,…

AI Document & PDF ToolsGovernment & Public Sector
View Tool

Part of Azure Content Understanding in Foundry Tools, automates the extraction of text, structure, and fields from various documents using AI models,…Azure Document Intelligence, part of Azure Content Understanding in Foundry Tools, automates the extraction of text, structure, and fields from various documents using AI models,…

AI Document & PDF ToolsGovernment & Public Sector
See the Top 250 AI Apps

Azure Vision in Foundry Tools enables developers to integrate computer vision capabilities into applications. It offers image analysis, OCR, and facial recognition without…

Computer Vision ToolsManufacturing & Industrial
View Tool

Enables developers to integrate computer vision capabilities into applications. It offers image analysis, OCR, and facial recognition without…Azure Vision in Foundry Tools enables developers to integrate computer vision capabilities into applications. It offers image analysis, OCR, and facial recognition without…

Computer Vision ToolsManufacturing & Industrial
See the Top 250 AI Apps

Instafill.ai automates PDF form filling, saving hours of manual work. Upload any PDF or Word document, provide data, and let AI accurately complete forms. Your data is secured and…

AI Document & PDF Tools

Ranked #72 in AI Document & PDF Tools

View Tool

ReceiptUp's advanced OCR API precisely extracts data from receipts and invoices, transforming them into structured JSON. It offers high accuracy, multilingual support, and handles…

Computer Vision Tools
View Tool

ReceiptUp's advanced OCR API precisely extracts data from receipts and invoices, transforming them into structured JSON. It offers high accuracy, multilingual support, and handles…

Computer Vision Tools
See the Top 250 AI Apps

Mindee offers an AI-powered document processing API that automates data extraction from various document types, including invoices and receipts. With high accuracy and speed, it…

AI Document & PDF Tools
View Tool

Veryfi offers AI-powered OCR APIs, mobile SDKs, and agents for secure, real-time data extraction from documents like invoices, receipts, and checks. It transforms unstructured…

AI Document & PDF ToolsFinance & Banking

Ranked #23 for Extracting text from images

View Tool

Offers AI-powered OCR APIs, mobile SDKs, and agents for secure, real-time data extraction from documents like invoices, receipts, and checks. It transforms unstructured…Veryfi offers AI-powered OCR APIs, mobile SDKs, and agents for secure, real-time data extraction from documents like invoices, receipts, and checks. It transforms unstructured…

FeaturedAI Document & PDF ToolsFinance & Banking
Ranked #23 for Extracting text from images

Doc2Form

AI Apps

Doc2Form converts PDFs, Word documents, and scanned forms into native Google Forms using AI. It extracts questions, options, and sections, creating a Google Form directly in your…

AI Document & PDF ToolsEducation & E-learning

Ranked #136 in AI Document & PDF Tools

View Tool

AI Apps

Converts PDFs, Word documents, and scanned forms into native Google Forms using AI. It extracts questions, options, and sections, creating a Google Form directly in your…Doc2Form converts PDFs, Word documents, and scanned forms into native Google Forms using AI. It extracts questions, options, and sections, creating a Google Form directly in your…

AI Document & PDF ToolsEducation & E-learning
Ranked #136 in AI Document & PDF Tools

api4ai offers cloud-native, AI-powered APIs for image processing. Access subscription-based solutions for background removal, OCR, content moderation, and face recognition. They…

Computer Vision Tools
View Tool

Offers cloud-native, AI-powered APIs for image processing. Access subscription-based solutions for background removal, OCR, content moderation, and face recognition. They…api4ai offers cloud-native, AI-powered APIs for image processing. Access subscription-based solutions for background removal, OCR, content moderation, and face recognition. They…

Computer Vision Tools
See the Top 250 AI Apps
1 of 8