nimble-extract-reference

Reference for nimble extract command. Load when fetching URLs or scraping pages. Contains: render tiers 1-3, all flags, browser actions, network capture.

SKILL.md

GitHub

nimble extract — reference

Fetches a URL and returns its content. The workhorse command — use for any URL where no agent exists.

Table of Contents


Parameters

Parameter CLI flag Type Default Description
url --url string — Target URL (required)
render --render bool false Enable headless browser (JS rendering)
driver --driver string vx6 Engine: vx6 · vx8 · vx8-pro · vx10 · vx10-pro — see Drivers table
formats --format array ["html"] Output format(s): "html", "markdown". CLI: string (--format markdown). SDK: array (formats=["markdown"]).
parse --parse bool false Enable parser (use with parser)
parser --parser JSON — Extraction schema — see parsing-schema.md
browser_actions --browser-action JSON — Browser actions sequence — see browser-actions.md
network_capture --network-capture JSON — XHR intercept rules — see network-capture.md
is_xhr --is-xhr bool false Direct API call — no browser, no render
country --country string — ISO Alpha-2 geo proxy (e.g. US, GB)
state --state string — State-level geo targeting
city --city string — City-level geo targeting
locale --locale string — LCID locale (e.g. en-US, fr-FR) — pair with country
method --method string GET HTTP method: GET, POST, PUT, PATCH, DELETE
tag --tag string — Request tag for analytics

Drivers

Driver Description Render Best for
vx6 Fast HTTP (no JS) No Static HTML, APIs, high volume
vx8 Headless browser Yes Dynamic sites, SPAs
vx8-pro Headful browser Yes Complex interactions
vx10 Stealth headless Yes Bot-protected sites
vx10-pro Stealth headful Yes Most protected sites

CLI

# Markdown output (default for most tasks)
nimble --transform "data.markdown" extract \
  --url "https://example.com/page" --format markdown

# Save to file
nimble --transform "data.markdown" extract \
  --url "https://example.com/page" --format markdown > .nimble/page.md

Python SDK

from nimble_python import Nimble

nimble = Nimble()
resp = nimble.extract(url="https://example.com/page", formats=["markdown"])
print(resp["data"]["markdown"])

Render tiers — escalate on failure

Failure signals: status 500 · empty data.html / data.markdown · "captcha" / "verify you are human" in content · login wall instead of target page

Tier CLI When
1 extract --url "..." Static pages, docs, news, GitHub, Wikipedia, HN
2 extract --url "..." --render SPAs, dynamic content, JS-rendered pages
2b --render --render-options '{"render_type":"idle2","timeout":60000}' Slow SPAs, wait for network idle
3 --render --driver vx10-pro E-commerce, social, job boards — max stealth
# Tier 1 — no render
nimble --transform "data.markdown" extract --url "https://example.com" --format markdown

# Tier 2 — render
nimble --transform "data.markdown" extract --url "https://example.com" --render --format markdown

# Tier 3 — stealth
nimble --transform "data.markdown" extract --url "https://example.com" --render --driver vx10-pro --format markdown
# Tier 2 — render
resp = nimble.extract(url="https://example.com", render=True, formats=["markdown"])

# Tier 3 — stealth
resp = nimble.extract(url="https://example.com", render=True, driver="vx10-pro", formats=["markdown"])

Browser actions

For interacting with a page before extracting — clicks, scrolls, form fills, infinite scroll. Requires render=True.

See browser-actions.md for all action types and params.

nimble --transform "data.markdown" extract \
  --url "https://example.com/product" --render \
  --browser-action '[
    {"type": "click", "selector": ".tab-reviews", "required": false},
    {"type": "wait_for_element", "selector": ".review-list"}
  ]' --format markdown
resp = nimble.extract(
    url="https://example.com/product",
    render=True,
    browser_actions=[
        {"type": "click", "selector": ".tab-reviews", "required": False},
        {"type": "wait_for_element", "selector": ".review-list"},
    ],
    formats=["markdown"],
)

Network capture

When page data comes from XHR/AJAX calls, or to call a known API endpoint directly with --is-xhr.

See network-capture.md for filter syntax and --is-xhr mode.

# Intercept an API call triggered by the page
nimble extract \
  --url "https://example.com/products" --render \
  --network-capture '[{"url": {"type": "contains", "value": "/api/products"}, "resource_type": ["xhr", "fetch"]}]' \
  > .nimble/products-api.json

# Known public API endpoint — use --is-xhr (no browser, fastest)
nimble --transform "data.markdown" extract \
  --url "https://api.example.com/v1/markets?q=elections&limit=50" \
  --is-xhr --format markdown
# Intercept via render
resp = nimble.extract(
    url="https://example.com/products",
    render=True,
    network_capture=[{"url": {"type": "contains", "value": "/api/products"}, "resource_type": ["xhr", "fetch"]}],
)
captures = resp["data"]["network_capture"]

# Direct API call — no browser
resp = nimble.extract(
    url="https://api.example.com/v1/markets?q=elections&limit=50",
    is_xhr=True,
)

Note: is_xhr and render are mutually exclusive.


Parser schemas — structured extraction

When markdown doesn't contain fields cleanly. Results land in data.parsing.

See parsing-schema.md for selector types, extractors, and post-processors.

nimble extract --url "https://example.com/product" --render --parse \
  --parser '{
    "type": "schema",
    "fields": {
      "title": {"type": "terminal", "selector": {"type": "css", "css_selector": "h1"}, "extractor": {"type": "text"}},
      "price": {"type": "terminal", "selector": {"type": "css", "css_selector": "[data-price]"}, "extractor": {"type": "text"}}
    }
  }'
resp = nimble.extract(
    url="https://example.com/product",
    render=True,
    parse=True,
    parser={
        "type": "schema",
        "fields": {
            "title": {"type": "terminal", "selector": {"type": "css", "css_selector": "h1"}, "extractor": {"type": "text"}},
            "price": {"type": "terminal", "selector": {"type": "css", "css_selector": "[data-price]"}, "extractor": {"type": "text"}},
        },
    },
)
print(resp["data"]["parsing"])

Geo targeting

# Country
nimble --transform "data.markdown" extract --url "https://example.com" --country US --format markdown

# City-level
nimble --transform "data.markdown" extract --url "https://example.com" --country US --state CA --city los_angeles --format markdown

# Localized (pair --locale with --country)
nimble --transform "data.markdown" extract --url "https://example.com/fr" --country FR --locale fr-FR --format markdown
resp = nimble.extract(url="https://example.com", country="US", formats=["markdown"])

resp = nimble.extract(url="https://example.com", country="US", state="CA", city="los_angeles", formats=["markdown"])

Async extract

For batch processing or long-running extractions. Returns immediately with a task ID; poll for results.

Additional async-only params:

Parameter CLI flag Type Description
storage_type --storage-type string Cloud provider: s3 or gs
storage_url --storage-url string Destination (e.g. s3://my-bucket/path/)
compress --compress bool GZIP compress results before storing
custom_name --custom-name string Custom filename (default: task ID)
callback_url --callback-url string Webhook URL — called on completion

Task states: pending → running → success / failed

# Submit async
nimble extract-async --url "https://example.com/page" --render --format markdown

# Poll status
nimble tasks get --task-id <task_id>

# Fetch results
nimble tasks results --task-id <task_id>
import asyncio
from nimble_python import AsyncNimble

async def extract():
    nimble = AsyncNimble()
    task = await nimble.extract_async(url="https://example.com/page", render=True, formats=["markdown"])
    task_id = task["task"]["id"]
    while True:
        status = await nimble.tasks.get(task_id=task_id)
        state = status["task"]["state"]
        if state in ("success", "failed"):
            break
        await asyncio.sleep(5)
    result = await nimble.tasks.results(task_id=task_id)
    print(result["data"]["markdown"])

asyncio.run(extract())

Batch extract

Submit up to 1,000 URLs in a single request. Uses an inputs + shared_inputs pattern — shared config applies to all items, per-item values override.

Parameters:

Parameter CLI flag Type Default Description
inputs --input array required Array of per-URL requests, each with at least url
shared_inputs --shared-inputs JSON — Defaults applied to all items (render, format, driver, delivery)

shared_inputs fields:

  • Extraction defaults (overridable per item): render, driver, formats, country, locale, parse, parser
  • Delivery params (batch-wide, not overridable): storage_type, storage_url, storage_compress, storage_object_name, callback_url

CLI:

nimble extract-batch \
  --shared-inputs 'render: true' --shared-inputs 'format: markdown' \
  --input '{"url": "https://example.com/page-1"}' \
  --input '{"url": "https://example.com/page-2"}' \
  --input '{"url": "https://example.com/page-3"}'

Python SDK:

resp = nimble.extract_batch(
    inputs=[
        {"url": "https://example.com/page-1"},
        {"url": "https://example.com/page-2"},
        {"url": "https://example.com/page-3"},
    ],
    shared_inputs={"render": True, "formats": ["markdown"]},
)
batch_id = resp["batch_id"]

Node SDK:

const resp = await nimble.extractBatch({
  inputs: [
    { url: "https://example.com/page-1" },
    { url: "https://example.com/page-2" },
    { url: "https://example.com/page-3" },
  ],
  sharedInputs: { render: true, formats: ["markdown"] },
});
const batchId = resp.batch_id;

Response:

{
  "batch_id": "b7e1a2f3-...",
  "batch_size": 3,
  "tasks": [
    { "id": "task-001-uuid", "state": "pending", "batch_id": "b7e1a2f3-..." }
  ]
}

Polling: Use nimble batches progress --batch-id <batch_id> to check completion, then nimble batches get --batch-id <batch_id> to get all task IDs, then nimble tasks results --task-id <id> for each successful task. See nimble-tasks reference for the full polling flow.

Delivery options:

  • Polling — check status with batch/task IDs (default)
  • Webhooks — pass callback_url in shared_inputs; Nimble POSTs on completion
  • Cloud storage — set storage_type + storage_url in shared_inputs

Parallelization

mkdir -p .nimble
nimble --transform "data.markdown" extract --url "https://example.com/1" --format markdown > .nimble/1.md &
nimble --transform "data.markdown" extract --url "https://example.com/2" --format markdown > .nimble/2.md &
nimble --transform "data.markdown" extract --url "https://example.com/3" --format markdown > .nimble/3.md &
wait

Response

Field Type Description
data.html string Extracted HTML content
data.markdown string Content as markdown (if format=markdown)
data.parsing object Structured data (if parse=True)
data.network_capture array Captured network requests (if network_capture set)
status_code number HTTP status code from target
task_id string Unique request identifier
metadata.driver string Driver used (e.g. vx6, vx10-pro)
metadata.query_duration number Extraction time in ms