Datasheet Parser

Parse manufacturer PDF datasheets into structured wiki markdown. Your role is orchestrator + reviewer, not extractor. The heavy lifting — rendering pages, running OCR, detecting layout/tables/figures, enumerating figure bboxes from the PDF object tree, computing confidence signals — happens on the ds-extract service. You only get called on the bbox crops the service couldn't resolve on its own.

Architecture

PDF --> POST /extract (ds-extract)
          |
          |-> 1141 blocks typed by docling
          |-> table cells from pdfplumber (fallback when docling empty)
          |-> figure bboxes from PyMuPDF object tree
          |-> cross-extractor agreement (containment)
          |-> rules engine -> escalation_queue (~40 bboxes, not 67 pages)
          |
You <-----+
  |
  |-> Batch escalation_queue crops via /extract-region -> single vision call
  |-> Merge answers back into blocks
  |-> Generate wiki markdown from the merged structured data
      |-> adom-wiki page publish + asset upload

Everything before you is deterministic and runs on the service in a few minutes of CPU. Vision tokens are only spent on the ~40 bbox crops the service's confidence routing couldn't resolve.

Service URL

https://ds-extract-fa4sdo7pnkrl.adom.cloud/

Queue Integration

A shared queue at https://wtqihf5e8fsv.adom.cloud coordinates parsing across agents.

ds-queue list                             # what needs parsing
ds-queue claim --by $(hostname)           # claim next item (returns id + pdf_url + part)
ds-queue complete <id> --by $(hostname) --wiki-slug "datasheets/<part>"
ds-queue fail <id> --by $(hostname) --reason "<what broke>"

Claim FIRST to prevent duplicate work when parsing from the queue.

Workflow

Step 1: Acquire the PDF

Three sources:

• User URL -> curl -sL -o /tmp/<part>.pdf "$URL"
• Local path -> use directly
• Queue item — check the pdf_url field:
  • http* -> download via curl
  • /uploads/* -> curl -sL -o /tmp/<part>.pdf "https://wtqihf5e8fsv.adom.cloud<upload_path>"
• No source -> WebSearch for the manufacturer's official PDF. Prefer ti.com, bosch-sensortec.com, st.com, nxp.com, microchip.com, analog.com over aggregators.

Step 2: Extract via ds-extract service

curl -sS -F pdf=@/tmp/<part>.pdf https://ds-extract-fa4sdo7pnkrl.adom.cloud/extract \
  -o /tmp/<part>-extract.json

This takes ~5-7 minutes on CPU for a 60-page datasheet (cached by sha256; re-requests are instant).

Abridged response shape:

{
  "pdf_hash": "sha256...",
  "page_count": 67,
  "pages": [
    {
      "page": 1, "width_pt": 612, "height_pt": 792,
      "page_png_url": "/artifact/<hash>/p1.png",
      "blocks": [
        {
          "block_id": "p1_b4", "type": "title",
          "bbox_pt": [x0, y0, x1, y1],
          "text": "BQ76920, BQ76930, BQ76940",
          "confidence": 0.9,
          "cross_agreement": {"pdftotext": 1.0, "pdfplumber": 1.0},
          "findings": [...]
        }
      ],
      "object_figures": [...]
    }
  ],
  "escalation_queue": [
    {
      "block_id": "p8_b4",
      "page": 8, "bbox_pt": [...], "block_type": "table",
      "priority": 20, "rule": "table.has_cells",
      "question": "Extract this table as markdown..."
    }
  ],
  "rules_summary": {
    "block_findings": {"info": 113, "warn": 145, "escalate": 40},
    "escalations": 40
  }
}

Step 3: Process the escalation queue

For every entry in escalation_queue, fetch a tight bbox crop and answer the rule-specific question using vision.

curl -sS -X POST https://ds-extract-fa4sdo7pnkrl.adom.cloud/extract-region \
  -H 'Content-Type: application/json' \
  -d '{"pdf_hash": "...", "page": 8, "bbox_pt": [x0, y0, x1, y1], "dpi": 300}' \
  -o /tmp/crop-<block_id>.png

Then Read each crop PNG and answer the question field. Batch where possible — reading 5-10 small crops in one turn is cheap.

For tables: expect a markdown table back. For figures: a "complete / partial / phantom" verdict plus a brief description. For low-agreement text: the verbatim text.

Write each verdict back into the corresponding block's text or table_cells field in memory.

Step 4: Assemble the wiki markdown

Walk the merged pages[].blocks and group by docling's semantic section structure.

Key rules:

• Key Specifications must be a 2-column table (Parameter | Value)
• Pin Configuration must have a Name column; optional Pin, Type, Description
• Formulas use LaTeX: $$...$$ (display) or $...$ (inline)
• Features use - unordered lists
• Do NOT include ## Diagrams — that tab is populated from screenshot assets

Save the body to /home/adom/project/project-content/datasheets/<part>/<part>.md plus a temp copy at /tmp/<part>-body.md.

Step 5: Publish

adom-wiki page publish "datasheets/<part>" \
  --title "<Part> -- <one-line description>" \
  --brief "<2-3 sentence summary>" \
  --body-md /tmp/<part>-body.md \
  --changelog "Parsed from <manufacturer> <doc number>" \
  --sample-prompt "Show me the datasheet for <part>" \
  --sample-prompt "Parse the <family> datasheet" \
  --sample-prompt "<3-5 trigger phrases total>"

Step 6: Upload hero + screenshot assets

For each validated figure block, fetch its crop and upload:

adom-wiki asset upload datasheets/<part> --asset-type hero-image \
  --file <best-figure>.png --caption "<description>"

adom-wiki asset upload datasheets/<part> --asset-type screenshot \
  --file <figure>.png --caption "<figure caption>"

Target 5-20 screenshots. Always explicitly set hero_asset_id after uploading.

Step 7: Store artifacts + complete the queue

cp /tmp/<part>.pdf /home/adom/project/project-content/datasheets/<part>/
mv /tmp/<part>-extract.json /home/adom/project/project-content/datasheets/<part>/
ds-queue complete <id> --by $(hostname) --wiki-slug "datasheets/<part>"

Troubleshooting

Dependencies

The ds-extract service handles all Python/OCR deps. You only need:

• curl
• adom-wiki CLI
• ds-queue CLI