kuzco-extract-text

Extract text visible in an image (OCR) with a local vision LLM. Wraps kuzco's tool_llm_image_extract_text() ellmer tool. Use when the task needs reading text from screenshots, documents, signs, labels

SKILL.md

GitHub

kuzco-extract-text

Extract text visible in an image (OCR) using a local vision language model, exposed as an ellmer tool by the kuzco R package.

When to use

The user wants the text content of an image: screenshots, scanned documents, photos of signs or labels, memes, charts with annotations. If the task is about the image as a whole rather than its text, use kuzco-classification instead.

Setup

library(kuzco)
library(ellmer)

chat <- chat_ollama(model = "qwen2.5vl")
chat$register_tool(tool_llm_image_extract_text())

Tool arguments (LLM-controlled)

  • image (required): local file path to a jpeg, jpg, or png image.
  • additional_prompt (optional): extra instructions, e.g. "only transcribe the headline".
  • language (optional): output language, e.g. "English".

Result

Compact JSON text with the extracted text and a confidence score. Reproduce the transcribed text faithfully; flag low confidence to the user rather than silently "fixing" unclear passages.

Example

User: "What does the sign say in img/sign.jpg?"

  1. Call kuzco_image_extract_text with image = "img/sign.jpg".
  2. Quote the extracted text and note the confidence.