llm-boxed-answer-extraction
Extract final numeric answers from LaTeX \boxed{} notation in LLM math reasoning output, scanning matches in reverse for robustness
SKILL.mdskills/llm/boxed-answer-extraction/SKILL.md
GitHubBoxed Answer Extraction
Overview
Math-reasoning LLMs typically wrap their final answer in \boxed{}. However, chain-of-thought outputs may contain multiple \boxed{} occurrences (intermediate results, corrections). Scan all matches in reverse order and take the last non-empty one — this corresponds to the model's final answer after any self-corrections.
Quick Start
import re
def extract_boxed_answer(text):
pattern = r'\\boxed\{([^{}]*(?:\{[^{}]*\}[^{}]*)*)\}'
matches = re.findall(pattern, text)
if not matches:
return None
for match in reversed(matches):
match = match.strip()
if match:
nums = re.findall(r'-?\d+', match)
if nums:
return int(nums[-1])
return None
answers = [extract_boxed_answer(resp) for resp in responses]
valid = [a for a in answers if a is not None]
Workflow
- Generate reasoning chain from math LLM (DeepSeek-R1, QwQ, etc.)
- Find all
\boxed{...}matches via regex - Iterate matches in reverse (last = final answer)
- Parse numeric content, handling nested braces and formatting
- Apply domain constraints (e.g., modulo 1000 for competition)
Key Decisions
- Reverse scan: models self-correct — intermediate \boxed{} may be wrong, last is final
- Nested braces: use non-greedy matching with brace balancing for
\boxed{\frac{1}{2}} - Fallback: if no \boxed{} found, try extracting last number from the response
- Normalization: strip LaTeX commands (
\frac,\text) before numeric parsing