Welcome the NVIDIA Llama Nemotron Nano VLM to Hugging Face Hub

NVIDIA Llama Nemotron Nano VL is a state-of-the-art 8B Vision Language Model (VLM) designed for intelligent document processing, offering high accuracy and multimodal understanding. Available on Hugging Face, it excels in extracting and understanding information from complex documents like invoices, receipts, contracts, and more. With its powerful OCR capabilities and efficient performance on the OCRBench v2 benchmark , this model delivers industry-leading accuracy for text and table extraction, as well as chart, diagram, and table parsing. Whether you’re automating financial document processing or improving business intelligence workflows, Llama Nemotron Nano VL is optimized for fast, scalable deployments.

Check out the tutorial below to start building your own intelligent document processing solutions with Llama Nemotron Nano VL! Users can also post-train the model further with their own datasets using NVIDIA NeMo .

Llama Nemotron Nano VL , the latest addition to the NVIDIA Nemotron family of models, is a vision language model (VLM) designed to push the boundaries of intelligent document processing (IDP) and optical character recognition (OCR). With its high accuracy, low model footprint, and multimodal capabilities, Llama Nemotron Nano VL enables the seamless extraction and understanding of information from complex documents. This includes PDFs, images, tables, charts, formulas, and diagrams, making it an ideal solution for automating document workflows across various industries like finance, healthcare, legal, and government.

Llama Nemotron Nano VL builds upon Llama-3.1-8B-Instruct and C-RADIOv2-VLM-H , a Vision Transformer (ViT) that serves as the backbone for visual feature extraction. This allows the model to handle a wide variety of visual elements in documents, including charts, graphs, and other complex visual representations.

C-RADIOv2-VLM-H Vision Transformer (ViT) : The core visual understanding component of the model, C-RADIO allows for high-resolution processing of documents containing complex visual elements. This serves as a vision backbone, excelling across visual domains and enabling the model to process multi-image understanding with high resolution. This technology underpins the model’s ability to handle complex documents containing visual elements such as images, diagrams, charts, and tables.

C-RADIO is trained on multi-resolution data using multiple distillation techniques. Multiplicative noise was applied to our weights during training to improve generalization.

Llama Nemotron Nano VL was trained using NVIDIA Megatron and utilizes efficient Transformer implementations in NVIDIA Transformer Engine . For multimodal dataloading, we use Megatron Energon . We provide example Megatron training and inference scripts along with hyperparameters and other instructions to enable custom training of VLMs.

Llama Nemotron Nano VL demonstrates exceptional accuracy on the OCRBench v2 benchmark , which tests models on real-world OCR and document understanding tasks. These tasks include text recognition, table extraction, and element parsing across various document types. The model’s advanced capabilities enable it to deliver better performance than current leading VLMs in real-world enterprise scenarios.

This strong performance is underpinned by high-quality in-domain data combined with a diverse training distribution across document types, languages, and layouts. A robust data strategy ensures coverage across challenging use cases through selective curation, targeted augmentation, and formatting techniques that clarify task intent and reduce ambiguity—resulting in models that generalize effectively to real-world applications.

OCRBench v2 is an advanced benchmark designed to evaluate OCR models across a diverse range of real-world document types and layouts. It includes over 10,000 human-verified question-answer pairs to rigorously assess a model’s capabilities in visual text localization, table parsing, diagram reasoning and key-value extraction.

Llama Nemotron VL further adopts a design that dynamically aggregates encoded patch features, enabling support for high-resolution input without sacrificing spatial continuity. This strategy efficiently processes documents with arbitrary aspect ratios while preserving both local detail and global context. It enables fine-grained analysis of dense visual content—such as small fonts, multi-column layouts, and intricate charts—without compromising computational efficiency or coverage. The model can also preserve the information better and have less distortion, due to the innovation in high-resolution tiling.

The document understanding capabilities of Llama Nemotron Nano VL can be largely attributed to the OCR-focused SFT data blend. Besides simple OCR, many of the datasets involve tasks such as predicting the correct reading order, reconstructing markdown formatting along with semantic classes (such as Captions, Titles, Section headers) and bounding boxes of individual text blocks. The model is also trained to parse mathematical formulas in LaTeX format and to extract tables in LaTeX, HTML, or markdown formats, depending on the prompt.

By empowering Llama-3.1 8B LLM with this strong vision foundation, Llama Nemotron Nano VL delivers unparalleled accuracy in parsing and interpreting documents.

Llama Nemotron Nano VL was trained using several OSS datasets along with data from NVIDIA’s VLM-based OCR solution, NeMo Retriever Parse . This provides capabilities in text and table parsing, along with grounding, enabling the Llama Nemotron Nano VL to perform at an industry-leading level in document understanding tasks. Synthetic table extraction datasets that were used for training this OCR solution were also used for training Llama Nemotron Nano VL 8B VLM to enable more optimal table understanding and extraction.

Llama Nemotron Nano VL excels in tasks like text recognition and visual reasoning, and demonstrates advanced chart and diagram understanding capabilities. Llama Nemotron Nano VL allows for predicting bounding box coordinates in normalized space to enable grounding like tasks and text-referring.