detect-utf8
Detects whether the terminal, console, or device supports UTF-8 encoding. Checks code page, .NET encoding defaults, environment locale, and verifies with a round-trip test of Unicode characters.
SKILL.mdskills/detect-utf8/SKILL.md
GitHubDetect UTF-8
Problem
Scripts and tools that output or read non-ASCII text (emojis, accented characters, CJK, special symbols) fail silently or produce garbled output when the terminal or device doesn't support UTF-8. This can corrupt logs, break cross-platform file reads, and introduce subtle bugs in text processing.
Detection triggers
Activate when:
- You're about to write a script that outputs non-ASCII characters
- You see garbled text in terminal output or log files
- Working across Windows and Unix environments with different default encodings
- You need to decide whether to use
-Encoding utf8or-Encoding utf8BOM - The task involves emoji, Unicode symbols, or internationalized text
Detection checks
The bundled script checks these in order:
- PowerShell output encoding —
$OutputEncodingand[Console]::OutputEncoding - Windows code page — via
chcp(e.g. 65001 = UTF-8, 437 = US-ASCII, 932 = Shift-JIS) - Locale environment —
LANG,LC_ALL,LC_CTYPEon Unix;$env:VSLANGon Windows - .NET default encoding —
[System.Text.Encoding]::Default - Round-trip test — writes and reads a Unicode string (✨🔬✅) to verify end-to-end UTF-8 fidelity
- UTF-8 BOM awareness — distinguishes between UTF-8 with BOM and without BOM
Protocol
- Run the detection script before creating or modifying files that contain non-ASCII text
- If UTF-8 is not supported, set the console to UTF-8 mode before proceeding:
- Windows:
chcp 65001or[Console]::OutputEncoding = [System.Text.Encoding]::UTF8 - Unix: set
LANG=en_US.UTF-8orLC_ALL=en_US.UTF-8
- Windows:
- If round-trip fails, use ASCII-safe alternatives or explicit byte encoding
- Apply UTF-8 to all subsequent operations when supported:
- Set
$PSDefaultParameterValues['*:Encoding'] = 'utf8'for all file/cmdlet operations - Use
-Encoding utf8explicitly on everyOut-File,Set-Content,Export-Csv, and similar cmdlets - Pass
[System.Text.UTF8Encoding]::new($false)to .NET file APIs for BOM-free UTF-8 - On Unix, ensure
LANGorLC_ALLends with.UTF-8before running scripts
- Set
- For scripts generated in this session, include an explicit encoding declaration:
- PowerShell:
#Requires -Version 7+ use-Encoding utf8on all file writes - Python:
# -*- coding: utf-8 -*-orsys.stdout.reconfigure(encoding='utf-8') - Node.js:
// encoding: utf-8or usefs.writeFileSync(path, data, 'utf8')
- PowerShell:
- Log the encoding state alongside other environment metadata
- If
-Applywas used, re-run detection after any shell/code-page reset to confirm UTF-8 is still active
When NOT to use
- Pure ASCII-only content (no Unicode characters at all)
- Binary file operations where encoding doesn't apply
- The encoding is already known from a prior detection in the same session
Bundled Script
Copy check-utf8.ps1 to your project's .ai_scripts/ directory:
# From skill source
cp <skill-path>/detect-utf8/scripts/check-utf8.ps1 .ai_scripts/
Run the detection:
.ai_scripts\check-utf8.ps1 # human-readable report
.ai_scripts\check-utf8.ps1 -Json # JSON for programmatic use
.ai_scripts\check-utf8.ps1 -Quiet # exit code only (0 = UTF-8 OK)
.ai_scripts\check-utf8.ps1 -Apply # detect + apply UTF-8 to session
Cross-references
- os-awareness — UTF-8 detection is part of OS-aware environment probing; OS type determines which encoding checks apply.
- portable-self-contained — Project-local tooling must be encoding-aware; UTF-8 detection ensures scripts produce portable output.
- audit-project — Audit can include encoding health as part of environment portability checks.