HexCore Binary Analysis

HexCore Binary Analysis Skill - v3.8.4 (Analysis Contract)

Scope

HexCore is a VS Code-based reverse-engineering environment. Use it only for binaries the user owns or is authorized to analyze, including CTF/HTB challenges, controlled game research, malware triage, and bug-bounty artifacts within scope.

The automation source of truth is extensions/hexcore-disassembler/src/automationPipelineRunner.ts. Validate jobs before treating examples in prose as executable contracts.

Release train: HexCore 3.8.4 in development, "Analysis Contract".

Contract additions an agent must respect: pipeline admin commands (runJob, validateJob, listCapabilities, queueJob, jobStatus) return contract-decorated responses (contractVersion: 1, canonical status (ok|partial|failed|skipped), typed diagnostics, artifacts) with all legacy fields preserved. The file-level hexcore-pipeline.status.json keeps the legacy error run status; only the command response uses failed. Error codes come from the ANALYSIS_ERROR_CODES registry in hexcore-common. When the target's .hexcore_session.db exists, run provenance records the persisted session ID/generation and the session engine manifest instead of a synthetic session; contextGeneration remains the runner's execution counter. Finding IDs are stable contract IDs (finding:sha256:<digest>:...) that survive re-runs with saved marks intact; cross-target references are rejected as wrong-target.

Wave 2.1 additions: detected PRNG mode/seed are propagated to later Debugger emulation unless explicitly overridden; API traces are bounded/grouped with exact counters; disassembly pages expose truncation and nextAddress; and dominant callfuscation uses instruction-aware rewriting plus reachable-only Remill lifting from the requested logical entry.

P2 hardening: ASCII line boundaries separate adjacent messages; disassembly context is contiguous or carries an explicit recovery reason; failed steps participate in slowest-duration accounting; Debugger state/run artifacts use the same bounded trace counters and compact execution summary.

FlareAuthenticator closure: hexcore.constraints.solveHeadless runs the packaged Z3 and returns concrete models; disassembly pages that fill the 10,000-instruction cap report truncated:true, stopReason:"count-limit", and nextAddress; Helix output exposes translation, lift-coverage, and semantic-type confidence separately. Never treat one 100% coverage field as whole-function or type correctness.

Engine Routing

Current native package versions

capstone 1.3.6, unicorn 1.3.2, remill 0.5.4, llvm-mc 1.0.2, better-sqlite3 2.0.3, souper 0.2.2, elixir 1.0.4, common 1.3.0.

Current integrated extensions relevant to automation: hexcore-disassembler 1.4.57, hexcore-debugger 2.1.21, hexcore-strings 1.3.3, hexcore-yara 2.1.3, hexcore-revenant 0.4.0, hexcore-helix 0.9.3, hexcore-peanalyzer 1.1.3, and hexcore-report-composer 1.0.11.

Job Rules

1. Put the canonical job at .hexcore_job.json, or use a descriptive *.hexcore_job.json name.
2. Keep outDir inside the workspace or the job-file directory. External output is rejected unless the user deliberately enables hexcore.pipeline.allowExternalOutDir.
3. Use hexcore.pipeline.validateJob or validateWorkspace before expensive runs.
4. Prefer a single job plus $step[N].output when one step consumes another step's artifact.
5. $step[prev] is valid except in step 0; forward references are invalid. Conditional jumps that may skip a referenced producer are reported by validation.
6. Top-level continueOnError is inherited by steps unless a step overrides it.
7. Set realistic timeouts. analyzeAll runs in a killable child process; inspect nativeExecution.lastPhase and the .hexcore-meta/*.heartbeat.json before increasing a deadline. A timeout must terminate the worker and reach terminal pipeline status.
8. Inspect both hexcore-pipeline.log and hexcore-pipeline.status.json. The status records attempts, output bytes, totals, slowest step, and queue snapshot.
9. A command disabled by hexcore.emulator is skipped, not a tool failure.
10. Do not invoke hexcore.pipeline.runJob from a pipeline step; recursive pipeline execution is blocked.
11. Preflight is mandatory and prevents command dispatch on validation errors. Do not bypass it to obtain a partial artifact.
12. Semantic child failures fail by default. Use allowPartial: true only when incomplete coverage is intentional, and preserve the terminal partial status.
13. Verify each artifact in the consolidated .hexcore-meta/provenance.json manifest before cross-run comparison. New jobs do not emit visible per-artifact provenance sidecars.

Watcher and queue behavior

• On startup, root-level canonical/named jobs are auto-discovered.
• The recursive watcher reacts to later create/change events under the workspace.
• A stale unchanged running status is archived and terminalized after hexcore.pipeline.staleRunningMs (15 minutes by default), then the revision may run again.
• Watch events are debounced and content-deduplicated; outputs are protected from re-trigger loops.
• Queue slots are configured by hexcore.pipeline.queue.poolSize (default 2, range 1..16). Stateful jobs serialize across the shared Extension Host; only audited stateless tools use parallel slots.
• Use sessionId for sticky routing when multiple queued jobs share a keepAlive emulation session.

Pipeline-Safe Commands

This list mirrors the v3.8.4 capability registry. Aliases are listed separately.

Static, format, and reporting

• hexcore.filetype.detect
• hexcore.hashcalc.calculate
• hexcore.entropy.analyze
• hexcore.strings.extract
• hexcore.strings.extractAdvanced - accepts optional minConfidence, maxDeobfuscated, highSignalOnly, decodeChains, and maxTransformChains; budgets and every transform stage remain auditable.
• hexcore.peanalyzer.analyze
• hexcore.pe.extractSection - bounded passive extraction of one named PE section.
• hexcore.crypto.rc4 - bounded passive RC4 transform; chain binary input with inputPath: "$step[N].output".
• hexcore.disasm.analyzePEHeadless
• hexcore.elfanalyzer.analyze
• hexcore.disasm.analyzeELFHeadless
• hexcore.base64.decodeHeadless
• hexcore.yara.scan
• hexcore.yara.updateRules
• hexcore.ioc.extract
• hexcore.hexview.dumpHeadless
• hexcore.hexview.searchHeadless
• hexcore.minidump.parse
• hexcore.minidump.threads
• hexcore.minidump.modules
• hexcore.minidump.memory
• hexcore.pipeline.composeReport - compact evidence index by default; use includeFullSources:true only for a deliberately self-contained report.

Disassembly, decompilation, and semantic analysis

• hexcore.disasm.analyzeAll
• hexcore.disasm.detectPacker - detection only; no unpacking or external UPX dependency.
• hexcore.disasm.buildFormula
• hexcore.constraints.solveHeadless
• hexcore.disasm.checkConstants
• hexcore.disasm.searchStringHeadless
• hexcore.disasm.exportASMHeadless
• hexcore.disasm.disassembleAtHeadless
• hexcore.disasm.rttiScanHeadless
• hexcore.disasm.searchBytesHeadless
• hexcore.disasm.extractStrings
• hexcore.disasm.liftToIR
• hexcore.helix.decompile
• hexcore.helix.decompileIR
• hexcore.revenant.decompile
• hexcore.revenant.decompileIL
• hexcore.hql.scanHeadless
• hexcore.souper.optimize
• hexcore.extractStructInfo
• hexcore.audit.refcountScan
• hexcore.rellic.decompile and hexcore.rellic.decompileIR - legacy compatibility only.

For HQL, binary targets and Remill-compatible LLVM IR are supported. Preserve every clean function record, signatureSetSha256, node count, adapter coverage, and unsupported-node counts. Interpret structuralCompleteness as rule satisfaction and evidenceLevel as signal, candidate, or proven; confidence is valid only when the signature names a hashed calibration corpus. Never map HQL presentation severity directly to vulnerability severity.

HQL 0.3 also reads typed HXDB facts. Preserve semanticFactCount, semanticFactsSha256, semantic match provenance, and proofStatus; do not flatten prototypes, xrefs, summary effects, conflicts, or barriers into strings. Runtime observations are corroboration bound to binary/input/trace hashes, not static proof.

For semantic edits, use hexcore.types.*, hexcore.references.*, hexcore.propagation.*, hexcore.typeManager.*, hexcore.records.recover, hexcore.pdb.*, and hexcore.signatures.apply. A stored edit is not complete until typed consumer propagation commits. Never infer one global struct member from an equal numeric offset without proven object identity.

Persistent analysis session

• hexcore.disasm.getSessionDbPath
• `hexcore.disasm.renameFunct

Content truncated.