Description
GLM-OCR is a sophisticated multimodal OCR model developed for complex document understanding. Built on the GLM-V encoder-decoder architecture, it introduces innovative techniques such as Multi-Token Prediction (MTP) loss and stable full-task reinforcement learning. These advancements significantly enhance training efficiency, recognition accuracy, and generalization capabilities.
The model integrates the CogViT visual encoder, pre-trained on extensive image-text data, and a lightweight cross-modal connector with efficient token downsampling. It also features a GLM-0.5B language decoder. Together, these components form a two-stage pipeline of layout analysis and parallel recognition based on PP-DocLayout-V3, ensuring robust and high-quality OCR performance across various document layouts.
GLM-OCR achieves state-of-the-art results, scoring 94.62 on OmniDocBench V1.5 and ranking #1 overall. It excels in major document understanding benchmarks, including formula recognition, table recognition, and information extraction. The model is optimized for real-world scenarios, maintaining robust performance on complex tables, code-heavy documents, seals, and other challenging layouts.
With only 0.9B parameters, GLM-OCR supports deployment via vLLM, SGLang, and Ollama, reducing inference latency and compute costs. This makes it ideal for high-concurrency services and edge deployments. The model is fully open-sourced, equipped with a comprehensive SDK and inference toolchain, offering simple installation, one-line invocation, and smooth integration into existing production pipelines.
The official SDK is recommended for document parsing tasks, integrating PP-DocLayoutV3 for layout analysis and structured output generation. This reduces the engineering overhead required to build end-to-end document intelligence systems. The GLM-OCR model is released under the MIT License, with the complete OCR pipeline integrating PP-DocLayoutV3, licensed under the Apache License 2.0.
GLM-OCR Highlights
Multimodal OCR model
GLM-V encoder-decoder architecture
Multi-Token Prediction loss
Stable reinforcement learning
CogViT visual encoder
GLM-0.5B language decoder
Two-stage pipeline
State-of-the-art performance
Getting Started with GLM-OCR
Access page: Visit the GLM-OCR page on Hugging Face
Load model: Download the GLM-OCR model
Configure environment: Set up the required environment for the model
Integrate: Use the SDK for seamless integration
GLM-OCR's Use Cases
- Document Parsing
- Information Extraction
- Table Recognition
- Formula Recognition
- Layout Analysis
