receipt-vlm

Qwen3-VL-2B-Instruct fine-tuned with QLoRA on one T4 to extract receipts (CORD) and French invoices as JSON. This page shows results precomputed on a T4; run the live demo locally with the commands below.

Code · Adapter · Invoice dataset

Results

100 test documents per set, greedy decoding. Field F1 is the micro F1 over (field path, normalized value) pairs; exact match requires every field. Brackets: 95% bootstrap intervals.

Test setModelValid JSONExact matchField F1Truncated
CORDbase0.890.00 [0.00, 0.00]0.351 [0.322, 0.381]11%
fine-tuned0.990.35 [0.26, 0.45]0.832 [0.789, 0.873]0%
Invoicesbase0.990.13 [0.07, 0.21]0.891 [0.866, 0.912]0%
fine-tuned1.000.63 [0.53, 0.72]0.979 [0.972, 0.985]0%
Invoices, unseen templates (50)fine-tuned1.000.54 [0.40, 0.68]0.974 [0.963, 0.985]0%

Latency

SettingReceiptInvoice
T4 GPU, fp16, batch size 1 (mean, 20 documents)5.8 s14.3 s
T4 GPU, peak memory4.5 GB4.2 GB
CPU, fp32, 2 threads (one document)about 40 sone to three minutes

Examples

Images from the test sets and the JSON returned by the fine-tuned model on a T4.

Synthetic invoice, training template (classic)

invoice example
{
  "invoice_number": "F20240286",
  "date": "2024-03-25",
  "seller": {
    "name": "Klein Charpentier S.A.",
    "address": "30, rue Roland Bailly, 30976 Fabre",
    "siret": "31439091598402",
    "vat_number": "FR47314390915"
  },
  "client": {
    "name": "Mercier Gallet S.A.R.L.",
    "address": "658, rue Leclercq, 09743 Andreboeuf"
  },
  "items": [
    {
      "description": "Prestation de conseil premium",
      "quantity": 3,
      "unit_price": 155.32,
      "amount": 465.96
    },
    {
      "description": "Installation réseau premium",
      "quantity": 10,
      "unit_price": 760.96,
      "amount": 7609.6
    },
    {
      "description": "Prestation de conseil",
      "quantity": 1,
      "unit_price": 1527.42,
      "amount": 1527.42
    },
    {
      "description": "Support technique premium",
      "quantity": 10,
      "unit_price": 1425.21,
      "amount": 14252.1
    },
    {
      "description": "Câble HDMI",
      "quantity": 2,
      "unit_price": 2359.18,
      "amount": 4718.36
    }
  ],
  "total_ht": 28573.44,
  "vat_rate": 5.5,
  "vat_amount": 1571.54,
  "total_ttc": 30144.98
}

Synthetic invoice, template never seen in training (serif)

invoice example
{
  "invoice_number": "F20234427",
  "date": "2023-04-24",
  "seller": {
    "name": "Baudry Texier et Fils",
    "address": "43, chemin Julien, 97419 Sainte Sébastien",
    "siret": "87014182716586",
    "vat_number": "FR02870141827"
  },
  "client": {
    "name": "Charlotte Lesage",
    "address": "7, rue Guilbert, 50604 Evrard"
  },
  "items": [
    {
      "description": "Clavier sans fil premium",
      "quantity": 2,
      "unit_price": 2423.62,
      "amount": 4847.24
    },
    {
      "description": "Développement web premium",
      "quantity": 5,
      "unit_price": 730.43,
      "amount": 3652.15
    }
  ],
  "total_ht": 8499.39,
  "vat_rate": 10,
  "vat_amount": 849.94,
  "total_ttc": 9349.33
}

CORD test receipt 0 (field F1 0.91: the item subtotal is returned as a unit price)

cord example
{
  "menu": [
    {
      "nm": "-TICKET CP",
      "num": "901016",
      "unitprice": "60.000",
      "cnt": "2",
      "price": "60.000"
    }
  ],
  "sub_total": {
    "subtotal_price": "60.000",
    "discount_price": "-60.000",
    "tax_price": "5.455"
  },
  "total": {
    "total_price": "60.000",
    "menuqty_cnt": "2",
    "creditcardprice": "60.000"
  }
}

CORD test receipt 1

cord example
{
  "menu": [
    {
      "nm": "J.STB PROMO",
      "price": "17500"
    },
    {
      "nm": "Y.B.BAT",
      "price": "46000"
    },
    {
      "nm": "Y.BASO PROM",
      "price": "27500"
    }
  ],
  "total": {
    "total_price": "91000",
    "cashprice": "91000"
  }
}

Run the demo

git clone https://github.com/N-Mehdi/receipt-vlm && cd receipt-vlm
pip install -r app/requirements.txt && pip install -e .
python -m app.app                      # Gradio demo with these examples

docker build -t receipt-vlm-api .      # or the HTTP API
docker run -p 8000:8000 receipt-vlm-api
curl -F file=@invoice.jpg "localhost:8000/extract?kind=invoice"

Limits

Licenses

Code and adapter: Apache-2.0. Base model Qwen3-VL-2B-Instruct: Apache-2.0. The receipt images come from the CORD v2 test set (naver-clova-ix/cord-v2, CC-BY-4.0, Park et al., 2019). The invoices are synthetic (Apache-2.0).