openscientist-hypothesis
Use when running or wiring OpenScientist for AI Gene Review gene-function hypotheses, especially blinded comparisons against local bioinformatics analyses.
SKILL.md.claude/skills/openscientist-hypothesis/SKILL.md
GitHubOpenScientist Hypothesis Workflow
Use this skill for AIGR work that runs, stages, or maintains OpenScientist hypothesis jobs for gene function review.
Core Rules
- Treat OpenScientist as an independent bioinformatics scientist exploring a
hypothesis: gene
Ghas functionF. - For existing annotations, prefer the neutral function-assignment mode:
--as-function-hypothesis. - Do not feed existing local
*-bioinformatics/RESULTS.mdanalyses into OpenScientist prompts. Those are holdout evidence for post-run comparison. - It is acceptable to pass bounded non-holdout review context, especially the selected term, original evidence reference, and a small number of relevant literature/deep-research references.
- Avoid sending every function for genes with large reviews; select the specific hypothesis being tested.
Running Jobs
- Use the
justrecipes, not rawdeep-research-client, for AIGR hypothesis jobs. The justfile carries project defaults such as OpenScientist iteration count. - Be patient with OpenScientist. Jobs may run quietly for a long time before the local report is written.
- Do not interrupt a real OpenScientist run merely because the local wrapper is silent or the child process is sleeping.
- Check completed and running jobs at https://www.openscientist.io/jobs before deciding a job is stuck.
- One-iteration OpenScientist runs are smoke tests only. Real hypothesis jobs should use the project default of at least 3 iterations unless the user explicitly asks otherwise.
Timeouts
- Give analyses ample time. Real 3-iteration runs commonly take ~50-70 minutes, and structural / fold-discovery jobs can run longer; they are cancelled mid-analysis if the timeout is too short (observed: a job cancelled at the upstream 3600s default after producing nothing).
- The
justrecipes now inject the maximum OpenScientist job timeout (--param timeout=7200, i.e. 2h) alongside--param max_iterations=3, and the wrapper's subprocess wall (--timeout-seconds, default 8100s) is kept above the job timeout so the child process is not killed first. If you ever bypassjustand call the script directly, pass both: e.g.-- --param max_iterations=3 --param timeout=7200plus--timeout-seconds 8100. The two timeouts are independent — raising one without the other still cancels the run. - 7200s is the API ceiling:
OpenScientistParams.timeoutis validatedle=7200, so a larger value fails fast with a pydanticless_than_equalerror and writes nothing. Do not exceed it. - Do not lower these to "save time"; a too-short timeout wastes the whole run.
- When a job still hits the 7200s ceiling (common for human proteins asked a
multi-faceted question), the fix is scope, not time: re-run with a single
decisive analysis in the hypothesis (e.g. "Foldseek fold assignment" alone,
not fold + disorder + motif + targeting) and/or
--param max_iterations=2. Observed: C18orf21 7200s timeout -> 2708s when narrowed to Foldseek-only; HSPA12A/DNAJC28 likewise completed once narrowed. The paralog of a job that did finish (e.g. HSPA12B at 5208s) will usually finish on a plain retry — variance, not a hard limit. - Scoping each hypothesis to one computable question up front (rather than a broad "characterize this gene") both avoids timeouts and gives cleaner, more decisive provenance.
Failure Modes
- A job can exit 0 yet write no report. Always confirm
openscientist.md(and theopenscientist_artifacts/dir) actually exist before treating a run as done; an empty hypothesis directory means it failed. - Two distinct failure signatures seen, both leaving an empty output dir:
detail=... timed out after Ns ... has been cancelled-> raise the timeout (see above) and re-run.detail=... Request ID: req_...-> a transient OpenScientist-internal LLM error (their server-side agent call failed; the id is for their debugging). Re-run; if the same gene fails this way repeatedly, treat it as a persistent upstream issue, move on, and report it rather than retrying indefinitely.
- Remove the empty
GENE-hypotheses/<slug>/directory before re-running so a failed attempt does not masquerade as a completed one.
After Completion
- Verify the generated report does not include withheld local bioinformatics
paths or
RESULTS.mdcontent. - Compare the OpenScientist conclusion against the held-out local bioinformatics result after the run.
- Keep operational lessons in skills or agent instructions, not in user-facing project pages.
Wiring Verdicts Into Reviews
- A refuted proposal must be deleted, not set to
REMOVE. Validation enforces that annotations absent from GOA carryaction: NEW(they are AI proposals, identifiable by having nooriginal_reference_id). If a run refutes such a proposal, delete the annotation block and anycore_functionsentry built on it, and record the refutation in thedescription+ an OpenScientistfile:reference.action: REMOVEis only for real GOA annotations and will fail validation on a not-in-GOA term ("Found N annotations not in GOA"). - Gate findings that depend on uncacheable references. When a run's
conclusion rests on very recent papers not in
publications/(so theirsupporting_textcannot be verbatim-verified) and that you cannot otherwise confirm, record it as a clearly-labelled lead — keep the conservative existing annotation, note "not yet verified / PMIDs not cached", and cite the OpenScientistfile:report — rather than asserting a new annotation. Verify, don't trust. (Example: C18orf21 -> RMP24 / RNase MRP, left as a lead pending primary-literature caching.) - Don't overstate disagreements. Before wiring "the run overturns the prior
review", check the primary data: the two analyses often agree on the facts
(e.g. residues present) and differ only on interpretation (e.g. a PROSITE
signature failure). Wire genuine standoffs as
UNDECIDEDwith both lines of evidence and the decisive experiment, and do not import the run's secondary claims (e.g. a named degenerate paralog) unless corroborated by repo data. - Verbatim
supporting_textgotchas: plain (unquoted) YAML scalars cannot contain": "— single-quote any quote containing a colon-space (e.g. "Foldseek: zero ..."). The pre-edit hook catches these.
Job Hygiene
- Background jobs are killed on container restart; the harness reports which.
Check the
GENE-hypotheses/<slug>/dir — if empty, clean it and re-launch. - Check whether the API key is already set before assuming it is not:
printenv OPENSCIENTIST_API_KEY(it is exported in some environments, so thejustrecipes work directly). Only if it is absent, passOPENSCIENTIST_API_KEYinline when calling the script directly, and wire it into the environment/setup config for reproduciblejustruns.