kortix-computer
How to reach a user's CONNECTED COMPUTER (their laptop, desktop, or server) from a Kortix session — read/write files, run shell commands, and drive the desktop (click/type/screenshot) on that machine.
SKILL.mdpackages/starter/templates/managed/skills/kortix-computer/SKILL.md
GitHub<skill name="kortix-computer">
<which-desktop>
"My desktop" can mean three places:
- Their Drive folder,
~/Desktop(=/drives/me). Kortix mounts it as their desktop in every session. It holds files: the user sees them in Files, and they last (thekortix-drivesskill). - Your session desktop. A workspace agent's microVM runs a Linux screen
the user watches live in the session's Desktop panel. A program runs
there: install it, then open it on display
:1so the window shows (/MACHINE.md→ Desktop). - Their own computer (Mac, PC, server), reached only through this connector.
Pick by what they ask for:
- A file (a report, a download to keep):
~/Desktop. - A program to use ("install QGIS and put it on my desktop"): run
kortix connectors accounts computerfirst. No computer connected: install it on your session desktop, open it, and say in one line that it is in the Desktop panel and that connecting their computer puts it there instead. A computer connected and online: ask once which they mean, naming both. - "My laptop", "my Mac", a path like
~/…or/Users/…: their computer.
Software for their computer must match its platform (status → platform).
A Linux package on your session desktop is not on their computer, and an
installer file in ~/Desktop is not installed anywhere.
</which-desktop>
- A private computer (the default) follows its owner: it is an account in every workspace they belong to, reachable only in their own private sessions.
- A shared computer (shared with the workspace) is reachable by everyone in the workspace, including unattended runs (triggers, schedules, Slack).
- The owner decides on the computer who may use it: Ask each time (a prompt on their screen, allowed for up to 24 hours; the default for a computer connected in the desktop app), Always allowed (the default for a computer connected with the CLI alone, which has no app to show a prompt), or Off. You cannot change this from the sandbox.
Tools on the computer connector:
status— call it first. The selected computer's name,online, platform, capabilities,home_dir,allowed_paths, andaccess({ mode, granted_until }:ask,always, oroff;nullon older agents). File tools only work insideallowed_paths(by default the user's home folder); start there, never at/or/Users. Withmode: 'off', stop and tell the human. Withmode: 'ask'and no currentgranted_until, your first call shows them a prompt; say so before you make it.- filesystem —
fs.read/fs.write/fs.list/fs.stat/fs.delete - shell —
shell.exec(stdout / stderr / exitCode) - desktop —
desktop.cua.list_windows/get_window_state(one window's screenshot plus its elements) /get_desktop_state(the whole screen) /click/double_click/right_click/type_text/press_key/hotkey/scroll/drag/set_value/launch_app/list_apps/get_screen_size/health_report, plusdesktop.cua.callfor any other driver tool (desktop.cua.list_toolslists them)
This is for a connected, external computer — not this sandbox. </overview>
<usage> **1. List the computers you can reach.** Each account label is a machine name.kortix connectors accounts computer
The slug is computer in most workspaces. An older workspace may use another
slug for its computer connector; kortix connectors ls shows it.
If there is no account, no computer is connected yet. This sandbox cannot see the user's disk, so never guess at a local path or pretend to read it. Say so in one line and tell the human how to connect it:
- In the Kortix desktop app: click "Connect this computer" at the bottom of the workspace sidebar. One click; macOS then asks for the permissions it needs.
- In the browser: click "Connect your computer" at the bottom of the
workspace sidebar, then install the desktop app or run the command it shows
(
npx @kortix/agent-tunnel@latest connect) on that computer.
Then ask them to tell you when it is connected, and continue from step 2. There is no feature flag to turn on, and nothing to set up per workspace. For a single file, uploading it into the chat is the faster path; offer both.
2. Check the machine before real work. Read online and access.
kortix connectors call computer status '{}' --account "Studio Mac"
3. Call a tool on the machine you mean. Pass --account "<name>" (or
me for your own default). With one reachable computer you may omit it.
kortix connectors call computer fs.read '{"path":"/Users/me/notes.md"}' --account "Studio Mac"
kortix connectors call computer shell.exec '{"command":"git","args":["status"],"cwd":"/Users/me/proj"}' --account "Studio Mac"
kortix connectors show computer.<action> prints a tool's input schema.
4. Drive an app: find it, look at it, act on an element, look again.
Desktop tools act on one app (pid) and window (window_id), in the
background, without taking over the screen.
kortix connectors call computer desktop.cua.list_windows '{"on_screen_only":true}' --account "Studio Mac"
# → pick the window: its pid and window_id
kortix connectors call computer desktop.cua.get_window_state '{"pid":4242,"window_id":17}' --account "Studio Mac"
# → screenshot + elements, each with an element_token
kortix connectors call computer desktop.cua.click '{"pid":4242,"element_token":"<token>"}' --account "Studio Mac"
kortix connectors call computer desktop.cua.type_text '{"pid":4242,"text":"hello"}' --account "Studio Mac"
kortix connectors call computer desktop.cua.hotkey '{"pid":4242,"keys":["cmd","s"]}' --account "Studio Mac"
Prefer element_token over pixel x/y. A key, chord, or text without a
pid goes to the frontmost app only with "scope":"desktop". Check the
result with get_window_state again: a keystroke is never verified by the
driver itself. When desktop calls fail for no clear reason, call
desktop.cua.health_report and report its failing checks.
</usage>
account_required— several computers are reachable and none was named or pinned. Ask which one, then pass--account.account_not_found—--account(or a legacycomputerargument) names no reachable computer. Use a label fromkortix connectors accounts computer.computer_access_pending— the owner has not answered the access prompt on their computer yet. Tell them: "Allow Kortix on your computer (the prompt on your screen), then tell me to continue." Retry once, after they confirm. Never loop-retry.computer_access_denied— the owner chose Deny. Do not retry; the prompt is held back for 10 minutes. Ask them to allow it from the Kortix menu-bar icon or the "Your computer" menu, then continue when they say so.computer_access_off— the owner turned access off on the computer. Tell them; only they can turn it back on ("Your computer" → Access in Kortix, ornpx @kortix/agent-tunnel@latest access askon that computer). Do not retry.computer_offline— the machine is paired but not connected. Ask the human to wake it or resume access in the Kortix desktop app.computer_unpaired— the account's machine was disconnected. The human must connect the computer again.computer_capability_not_approved— the human approved this computer without that capability (filesystem, shell, or desktop). They must connect the computer again and select it.computer_desktop_permission_missing— macOS has not given Kortix Accessibility or Screen Recording on that computer, so it cannot see or use apps. Tell the human: "Open the Kortix app → Your computer → Allow all." Retry once after they confirm; Kortix applies a new grant within 30 seconds.Not allowed on this computer: …— the call is outside the computer's approved scope (for example a path outsideallowed_paths). Change the call; do not ask for re-pairing.connector_not_connected— no computer is reachable from this session. None is connected, or every connected computer is private to someone else (an unattended run reaches only computers shared with the workspace). With--account, the error listsavailable_accounts. Say so; do not look for another way in.denied— a connector policy or a grant blocks this call. Report the reason; do not retry.- A call that needs approval waits for a human under the connector's policy. Tell them what you are doing and why.</errors>