NuExtract: A Foundation Model for Structured Extraction

We introduce NuExtract, a lightweight text-to-JSON LLM. NuExtract allows to extract arbitrarily complex information from text and turns it into structured data. This model can be directly used in a zero-shot setting or fine-tuned to solve a specific extraction problem. As usual, we open-source it under MIT license for everyone to use.

TLDR

We trained language models from 0.5B to 7B parameters on an LLM-generated structured-extraction dataset. The resulting models — NuExtract-tiny, NuExtract, and NuExtract-large — achieve similar or higher extraction performance than popular LLMs that are 100 times larger.

Structured Extraction

Structured Extraction is the most general and versatile information extraction task: extract all kinds of information from a document — entities, quantities, dates, and so on — and identify their (potentially hierarchical) relationships. The extracted information is structured as a tree following a template (a.k.a. schema), almost always expressed in JSON.

Even simple examples (5 entity kinds, 2 relation types, depth 4) are a headache for traditional information extraction methods. Real applications involve multi-page documents and deeper trees, challenging even for modern LLMs.

We encountered two main families of applications: parsing technical documents (medical, legal, financial — increasingly to power RAG knowledge bases), and chatbot conversations (extracting the right info to make API calls in real time).

In a sense, structured extraction is the holy grail of information extraction.

Why Not Just Use GPT-4?

GPT-4 with a good prompt can do structured extraction, but:

• Performance saturates with in-context learning (as we showed in our NuNER paper).
• It is massive, expensive, and requires sharing data.

To solve all this, we need a compact task-specific foundation model.

Task-Specific Foundation Models

A task-specific foundation model is specialized for a generic task — sentiment analysis, entity recognition, structured extraction — but agnostic in terms of data domain and specific problem. They are small, usable in private settings, and often better than much larger generic models at the task.

The recipe:

1. Take a diverse corpus (e.g. C4).
2. Annotate it using a modern LLM with a proper prompt — annotations don't have to be perfect.
3. Fine-tune a compact generic foundation model on this synthetic data.

The result can be used zero-shot, with examples, or fine-tuned for a specific problem.

Template Representation

We represent the schema with a sort of empty JSON:

{
  "reactants" : [{"name" : "" , "quantity" : ""}],
  "time" : [""]
}

Each array contains an element template; empty strings indicate fields to extract. Only string values — numbers can always be returned as strings. The format is simple, easy to read, and we believe examples are more informative than descriptions.

Dataset Creation

We use 300k English texts from the C4 dataset as base — diverse enough that we'll find something interesting to extract in most documents.

We then prompt an LLM (Llama 3 70B) to generate a template from each text, with hand-crafted few-shot examples in the prompt.

Once we have templates, we use the LLM again to extract information according to each template. For half the examples, we extract from the full text; for the other half, we remove parts of the text to teach the model that it's acceptable to return empty strings — a form of negative sampling that fights hallucinations.

The extraction prompt forces the LLM to copy-paste from the text rather than generate new values — another anti-hallucination tradeoff.

After filtering for template compliance and value-in-text checks, we end up with 50k annotated examples.

Word counts mostly stay below 200, with a tail up to 1200 (2-3 page documents). Extraction-tree depths span 3-5 (vs. depth 1 for classification/NER, depth 2 for relation extraction). The LLM produced 200k+ unique field names, with healthy domain coverage (dates, contact info, dimensions, nutrition, health, …).

We also add a hybrid few-shot setting: 0-3 output examples (no input) added to the prompt. NuExtract can be used pure zero-shot or with output-only examples.

Base Models

Structured extraction has a large complex output space, so we need to generate the output. We use pure decoder LLMs:

• Phi-3-mini (3.8B) → NuExtract
• Phi-3-small (7B) → NuExtract-large
• Qwen1.5-0.5B (0.5B) → NuExtract-tiny

Evaluation

We built a dedicated benchmark from a set of "problems" (e.g. resume parsing) with hand-extracted ground truths. We use a tree-matching metric that aligns extracted leaves recursively, scoring exact value matches between 0 (different) and 1 (perfect). Benchmark to be released publicly when finalized.

Zero-shot results:

• NuExtract-tiny is better than GPT-3.5 while at least 100× smaller.
• NuExtract outperforms Llama3-70B while 35× smaller.
• NuExtract-large reaches GPT-4o levels while at least 100× smaller.

Small specialized models 100× smaller than frontier LLMs bring three benefits: lower inference cost, local/private deployment, and easy fine-tuning.

After fine-tuning (5-fold cross-validation on a 50-example chemistry problem from Iktos.ai):

• NuExtract-tiny (0.5B) becomes a bit better than GPT-4o.
• NuExtract and NuExtract-large reach a different level entirely.

These results show the benefits of using small, fine-tuned, task-specific models for structured extraction.

Let's Use It!

Structured extraction is one of the main use cases of modern LLMs. NuExtract reaches similar or higher performance than the largest LLMs while being orders of magnitude cheaper to use. MIT-licensed, available for everyone. For even higher performance, talk to us 🙂.