Binfiddle

Binary utilities for developers and hackers

Version 0.30.0 | Cross-platform (Windows/Linux/macOS) | x86_64 / Arm64 Support | Linux process-memory features

Binfiddle is a developer-focused binary manipulation toolkit designed for flexibility, modularity, and clarity. It enables inspection, patching, differential analysis, statistical analysis, hashing, checksum verification, and custom exploration of binary data across a variety of formats.

Whether you're reverse-engineering firmware, debugging binary protocols, analyzing malware samples, patching live processes, or building custom workflows for binary files, Binfiddle provides essential tools without the bloat.

Features

Core Capabilities

Personal note: This is a reimplementation of a very old, with poor command line, version in C (and - believe it or not - plain Bash) I created for my own needs. I have been trying to renovate my personal tooling and making them publicly available. Nowadays we have LLMs that help a lot on crafting nice documentation and assisting on converting my tools. For this reason, I greatly appreciate any feedback, bug reports, issues, feature requests, fixes and improvements.

Feature	Description
Read	Extract and display byte ranges in multiple formats
Write	Overwrite bytes at specified positions
Edit	Insert, remove, or replace byte sequences
Search	Find patterns using exact match, regex, or wildcards
Analyze	Statistical analysis: entropy, histograms, Index of Coincidence
Diff	Compare binary files with multiple output formats
Patch	Apply binary patches (works with `diff --format patch` output)
Convert	Text encoding conversion and line ending normalization
Hash	Compute MD5, SHA-1, SHA-256, BLAKE3, CRC32, xxhash64 digests
Verify	Check files against md5sum/sha256sum-style checksum lists
Chain	Pipe multiple binfiddle commands together without shell escaping
Process Memory	Read/write memory from any same-user process via `/proc/<pid>/mem` (Linux)
Struct	Parse binary data using YAML templates for structure definitions
NN Workbench	Inspect, address, extract, edit, and compare neural-network artifacts (SafeTensors/GGUF/ONNX/PyTorch/NumPy captures) — static, offline, no model execution
Progress Bars	Opt-in throughput/ETA feedback for long-running commands

Key Differentiators

Feature	Binfiddle	Traditional Tools
Pipeline Integration	First-class stdin/stdout support	Often interactive-only
Unified Operations	Read/Write/Edit/Search/Analyze/Diff/Hash in one tool	Separate tools per operation
Configurable Chunking	1-64 bit granularity	Fixed 8-bit (byte) chunks
Multi-Format I/O	hex/dec/oct/bin/ascii/raw/base64	Usually hex-only
Large Files	Memory-mapped + streaming modes	Loads whole file into RAM
Live Process Inspection	Read/write process memory on Linux	Requires separate debuggers
Script Friendly	Deterministic, non-interactive, progress bars opt-in	Often requires user interaction
Built-in Analysis	Entropy, histograms, IC analysis	Requires external tools

Design Principles

Unix Philosophy: Composable, pipeline-friendly, text streams as interface
Safety by Default: No silent data loss, explicit modification flags required
Determinism: Identical inputs produce byte-identical outputs
Memory Safety: Built in Rust with no buffer overflows or data races
Non-Interactive First: Progress bars are opt-in; stdout stays clean for scripts

Installation

From Source (Recommended)

# Install Rust toolchain (if not already installed)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Clone and build
git clone https://github.com/araray/binfiddle.git
cd binfiddle
cargo build --release

# Binary is at target/release/binfiddle
sudo cp target/release/binfiddle /usr/local/bin/

From Releases

Download pre-built binaries from the Releases page.

Available targets:

x86_64-unknown-linux-gnu (Linux x64)
x86_64-unknown-linux-musl (Linux x64, static)
x86_64-pc-windows-gnu (Windows x64)
x86_64-apple-darwin (macOS Intel)
aarch64-apple-darwin (macOS Apple Silicon)

Linux process-memory features (--process-self, --pid, --force-writable) are only available on Linux builds.

Build for Other Platforms

# Use the build script
./scripts/build_releases.sh --native    # Current platform only
./scripts/build_releases.sh --help      # See all options

# Or manually with cargo
rustup target add x86_64-pc-windows-gnu
cargo build --release --target x86_64-pc-windows-gnu

Cross-check

# Verify aarch64 Linux cross-compilation
cargo check --target aarch64-unknown-linux-gnu

Quick Start

# Read first 16 bytes as hex
binfiddle -i file.bin read 0..16

# Read entire file as ASCII
binfiddle -i file.bin read .. --format ascii

# Write bytes at offset 0x100
binfiddle -i file.bin write 0x100 DEADBEEF -o modified.bin

# Search for a pattern
binfiddle -i file.bin search "7F 45 4C 46" --all

# Insert bytes at position
binfiddle -i file.bin edit insert 0x200 CAFEBABE -o modified.bin

# Analyze entropy (find encrypted/compressed sections)
binfiddle -i firmware.bin analyze entropy --block-size 4096

# Compare two binary files
binfiddle diff original.bin modified.bin --diff-format unified

# Hash a file
binfiddle -i firmware.bin hash sha256

# Verify a checksum list
binfiddle hash sha256 --check SHA256SUMS

# Pipeline usage
cat data.bin | binfiddle read 0..32 | grep "7f 45"

# xxd-style output with addresses and ASCII sidebar
binfiddle -i firmware.bin read 0..256 --show-ascii

# Raw binary output (pipe-friendly, no formatting)
binfiddle -i archive.bin read 0..4 --format raw | file -

# Show a progress bar on a long operation
binfiddle -i huge.bin hash sha256 --stream --read-block-size 64M --progress

# Inventory a neural-network model package (see docs/NN_USAGE.md)
binfiddle -i model-dir/ nn discover --verify-content --out-catalog m.nn.json
binfiddle nn ls --catalog m.nn.json --sort bytes --limit 20

Command Reference

Global Options

Option	Short	Description	Default
`--build-info`	—	Print machine-readable build provenance (commit, dirty flag, rustc, target, profile — embedded at build time) and exit	—
`--input <FILE>`	`-i`	Input file (use `-` for stdin)	stdin
`--output <FILE>`	`-o`	Output file (use `-` for stdout)	—
`--in-file`	—	Modify input file in-place	false
`--format <FMT>`	`-f`	Output format: hex, dec, oct, bin, ascii, raw	hex
`--input-format <FMT>`	—	Input value format	hex
`--chunk-size <BITS>`	`-c`	Bits per display chunk (1-64)	8
`--width <N>`	—	Chunks per output line (0=no wrap)	16
`--show-offset`	—	Show hex address prefix on each line	false
`--show-ascii`	—	Show ASCII sidebar (implies offset display)	false
`--silent`	—	Suppress change diff output and warnings	false
`--progress`	—	Show progress bars for long-running commands	false
`--color <MODE>`	—	Color output: always, auto, never	auto

Process-Memory Global Options (Linux only)

Option	Description
`--process-self`	Target the current process
`--pid <PID>`	Target another process
`--list-regions`	List mapped memory regions
`--address <ADDR>`	Base address (hex or decimal)
`--size <N>`	Number of bytes (hex or decimal)
`--allow-write`	Opt-in required for process-memory writes
`--force-writable`	Temporarily make read-only pages writable
`--zero-fill-inaccessible`	Fill unreadable pages with zeroes
`--skip-inaccessible`	Skip unreadable pages (read command only)

Commands

`read <RANGE>` — Read bytes from binary data

binfiddle -i file.bin read 0..64              # Bytes 0-63
binfiddle -i file.bin read 0x100..0x200       # Hex offsets
binfiddle -i file.bin read 10..               # Byte 10 to end
binfiddle -i file.bin read ..100              # First 100 bytes
binfiddle -i file.bin read ..                 # Entire file
binfiddle -i file.bin read 42                 # Single byte at index 42
binfiddle -i file.bin read 0..256 --show-offset        # xxd-style with address prefixes
binfiddle -i file.bin read 0..256 --show-ascii         # xxd-style with ASCII sidebar
binfiddle -i file.bin read 0..64 --format raw | file - # Raw binary output for piping

`write <POSITION> <VALUE>` — Write bytes to binary data

binfiddle -i file.bin write 0x100 DEADBEEF -o out.bin
binfiddle -i file.bin write 0 "127 69 76 70" --input-format dec -o out.bin
binfiddle -i file.bin --in-file write 16 FF   # In-place modification

`edit <OPERATION> <RANGE> [DATA]` — Structural modifications

Insert — Add bytes at position (data shifts right):

binfiddle -i file.bin edit insert 0x100 DEADBEEF -o out.bin

Remove — Delete byte range (data shifts left):

binfiddle -i file.bin edit remove 0x500..0x600 -o out.bin

Replace — Remove range and insert new data:

binfiddle -i file.bin edit replace 0..4 7F454C46 -o out.bin

`search <PATTERN>` — Find patterns in binary data

# Exact hex pattern
binfiddle -i file.bin search "DE AD BE EF" --all

# ASCII string
binfiddle -i file.bin search "PASSWORD" --input-format ascii --all

# Regex pattern
binfiddle -i file.bin search "[A-Z]{4}" --input-format regex --all

# Wildcard mask (? = any byte)
binfiddle -i file.bin search "DE ?? BE EF" --input-format mask --all

# Count matches only
binfiddle -i file.bin search "00 00" --all --count

# Show context around matches
binfiddle -i file.bin search "CAFE" --all --context 8

# Prevent overlapping matches
binfiddle -i file.bin search "AA AA" --all --no-overlap

# Stream-search a file larger than RAM
binfiddle -i huge.bin search "7F454C46" --all --block-size 64M

`hash <ALGORITHM>` — Compute digests

# Whole-file hashes
binfiddle -i file.bin hash md5
binfiddle -i file.bin hash sha1
binfiddle -i file.bin hash sha256
binfiddle -i file.bin hash blake3
binfiddle -i file.bin hash crc32
binfiddle -i file.bin hash xxhash64

# Base64 output
binfiddle -i file.bin hash sha256 --output-format base64

# Per-block CRC32 (find corrupted regions)
binfiddle -i disk.img hash crc32 --block-size 4096

# Stream-hash a file that does not fit in memory
binfiddle -i huge.bin hash sha256 --stream --read-block-size 64M --progress

# Verify a checksum list
binfiddle hash sha256 --check SHA256SUMS

`analyze <TYPE>` — Statistical analysis of binary data

# Entropy analysis (find encrypted/compressed sections)
binfiddle -i firmware.bin analyze entropy --block-size 4096

# Byte frequency histogram
binfiddle -i file.bin analyze histogram

# Index of Coincidence (cryptanalysis)
binfiddle -i file.bin analyze ic --block-size 0

# Output as CSV for graphing
binfiddle -i file.bin analyze entropy --output-format csv > entropy.csv

# Output as JSON
binfiddle -i file.bin analyze histogram --output-format json

# Stream-analyze a huge file
binfiddle -i huge.bin analyze entropy --block-size 64M --progress

Analysis Types:

Type	Description	Use Case
`entropy`	Shannon entropy (0-8 bits/byte)	Find encrypted/compressed sections
`histogram`	Byte frequency distribution	Identify file types, encoding
`ic`	Index of Coincidence	Cryptanalysis, detect encryption

Entropy Interpretation:

Range	Meaning
0.0 - 1.0	Highly repetitive (null bytes, single value)
1.0 - 4.0	Text, code, structured data
4.0 - 6.0	Mixed content
6.0 - 7.5	Compressed data
7.5 - 8.0	Encrypted or random data

`diff <FILE1> <FILE2>` — Compare two binary files

# Simple format (one line per difference)
binfiddle diff original.bin modified.bin

# Unified format (like text diff, with context)
binfiddle diff original.bin modified.bin --diff-format unified --context 3

# Side-by-side comparison
binfiddle diff original.bin modified.bin --diff-format side-by-side

# Generate patch file (for use with binfiddle patch)
binfiddle diff original.bin modified.bin --diff-format patch > changes.patch

# Ignore specific ranges (e.g., timestamps)
binfiddle diff v1.bin v2.bin --ignore-offsets "0x0..0x10,0x100..0x110"

# With color output
binfiddle diff file1.bin file2.bin --color always

# Show summary statistics
binfiddle diff file1.bin file2.bin --summary

Diff Output Formats:

Format	Description
`simple`	One line per difference: `Offset: 0xXX != 0xYY`
`unified`	Unified diff with context lines and hex dump
`side-by-side`	Two-column parallel comparison
`patch`	Machine-readable format for `binfiddle patch`

`convert` — Text encoding and line ending conversion

# Convert UTF-8 to UTF-16LE
binfiddle -i document.txt convert --to utf-16le -o document_utf16.txt

# Convert UTF-16LE to UTF-8
binfiddle -i windows_file.txt convert --from utf-16le --to utf-8 -o unix_file.txt

# Convert Windows line endings (CRLF) to Unix (LF)
binfiddle -i script.bat convert --newlines unix -o script.sh

# Add UTF-8 BOM
binfiddle -i file.txt convert --bom add -o file_with_bom.txt

# Remove BOM from a file
binfiddle -i file_with_bom.txt convert --bom remove -o file_no_bom.txt

# Full conversion: UTF-16LE → UTF-8, Unix newlines, no BOM
binfiddle -i windows_doc.txt convert \
    --from utf-16le --to utf-8 --newlines unix --bom remove \
    -o unix_doc.txt

Convert Options:

Option	Values	Default	Description
`--from`	utf-8, utf-16le, utf-16be, latin-1, windows-1252	utf-8	Source encoding
`--to`	utf-8, utf-16le, utf-16be, latin-1, windows-1252	utf-8	Target encoding
`--newlines`	unix, windows, mac, keep	keep	Line ending conversion
`--bom`	add, remove, keep	keep	BOM handling
`--on-error`	strict, replace, ignore	replace	Error handling mode

`patch <TARGET> <PATCH_FILE>` — Apply binary patches

# Apply a patch file to create a new output
binfiddle --output patched.bin patch original.bin changes.patch

# Preview changes without modifying (dry run)
binfiddle patch original.bin changes.patch --dry-run

# Modify file in-place with backup
binfiddle --in-file -i target.bin patch target.bin changes.patch --backup .bak

# Revert a patch (undo changes)
binfiddle --output reverted.bin patch patched.bin changes.patch --revert

Full Diff-Patch Workflow:

# 1. Create a patch from two files
binfiddle diff original.bin modified.bin --diff-format patch > changes.patch

# 2. Apply the patch to original to recreate modified
binfiddle --output reconstructed.bin patch original.bin changes.patch

# 3. Verify the result matches
diff modified.bin reconstructed.bin && echo "Perfect match!"

Patch Options:

Option	Description
`--backup <SUFFIX>`	Create backup before modifying (e.g., `.bak`)
`--dry-run`	Show what would be done without making changes
`--revert`	Apply patch in reverse (undo changes)

Patch File Format:

# binfiddle patch file
# source: original.bin
# target: modified.bin
# format: OFFSET:OLD_HEX:NEW_HEX
# differences: N
#
0x00000000:de:ff
0x00000002:be:ca

`chain --step <COMMAND>` — Execute multiple commands in sequence

Run several binfiddle commands in order, passing the byte output of each step as the input to the next. This avoids shell pipe escaping and makes multi-step transformations explicit.

Intermediate steps must produce byte output (e.g., write, edit, replace, convert). The final step may produce text output.

# Replace a header and then patch a byte
binfiddle -i firmware.bin -o patched.bin chain \
    --step "edit replace 0..4 44415431" \
    --step "write 8 00"

# Chain from stdin and read the result
printf '\x00\x11\x22\x33' | binfiddle chain \
    --step "edit replace 0..2 4142" \
    --step "read 0..4"

# Run silently so intermediate diagnostics do not pollute stderr
binfiddle --silent -i data.bin -o out.bin chain \
    --step "edit replace 0..8 1234567890abcdef" \
    --step "write 0 ff"

Chain Options:

Option	Description
`--step <COMMAND>`	One step to execute (repeatable, required). Quoting follows shell rules.

`nn` — Neural-network artifact workbench (early access)

A static workbench for neural-network artifact files — SafeTensors, GGUF, ONNX, PyTorch checkpoints (.pth/.pt/.ckpt, read as pure data), and NumPy captures (.npy/.npz) — and the directories that package them, including sharded releases with a model.safetensors.index.json. It inventories, addresses, extracts, edits, analyzes, and compares model artifacts without executing model code, without Python or ML frameworks, without network access, and without modifying inputs (all outputs are fresh files). Every report ends with a claims: line stating exactly what it proves — and what it does not.

Full guides: docs/NN_USAGE.md (every command and option), docs/NN_QUICK_REFERENCE.md (one-page card), and the real-model deep dives — see Documentation & references.

# Honest self-description of this build's capabilities
binfiddle nn capabilities

Discovery — `discover`, `ls`, `show`, `select`

nn discover inventories model artifacts descriptor-only (no payload reads): SafeTensors, GGUF v2/v3 (including split-shard groups with completeness and cross-shard uniqueness checks), sharded-SafeTensors packages (missing shards per the index are visible findings that fail --require-complete), ONNX at the descriptor tier (exact spans for raw_data initializers; external data and packed fields stay visible and honestly labeled), and PyTorch .pth/.pt/.ckpt checkpoints through a data-only pickle opcode reader (read-only tier), plus package directories where unknown files are kept visible as classified assets. Stdin (-i -) is spooled to a bounded private temp file with its own content-verified identity. nn ls / nn show / nn select browse and resolve tensors from a saved catalog; identities are content-addressed, so a saved selection never silently rematches different bytes (--rebind rebinds explicitly and reports additions/removals). nn shard-map plans which shards of a large release to download for a selection — statically, from the index alone.

binfiddle -i model-dir/ nn discover --verify-content --out-catalog m.nn.json
binfiddle -i model.gguf nn discover --require-complete     # exit 8 if incomplete
binfiddle nn ls --catalog m.nn.json --sort bytes --limit 20
binfiddle nn ls --catalog m.nn.json --name-regex 'gate|down' --view architecture --pack p.yaml
binfiddle nn show --catalog m.nn.json --tensor w --explain
binfiddle nn select --catalog m.nn.json --tensor w --out-selection w.sel.json

Addressing — `where`, `locate`, `impact`

Forward and reverse addressing with precision classifications (exact_contiguous / exact_bits), bit masks for packed encodings, shared decode dependencies (a Q4_0 element depends on its nibble and its block scale), and influence analysis that keeps decode dependencies distinct from numerical influence (a Q4_0 scale byte influences all 32 elements of its block; a code nibble exactly one). Behavior is never predicted.

binfiddle nn where --catalog m.nn.json --tensor w --index 1,1   # element → bytes/bits
binfiddle nn locate --catalog m.nn.json --offset 0x27c          # offset → owner
binfiddle nn impact --catalog m.nn.json --span 0x27c..0x280     # deps + influence
binfiddle nn impact --catalog m.nn.json --plan e.plan.json

Extraction — `slice`, `split`, `assemble`

Id-verified plans re-verify the catalog and full source digests before extracting — stale sources abort, never extract silently. Storage policies: reference (payloads stay in content-verified sources) or materialized (exact spans copied and hashed); quantization policies: preserve_encoding, cover_blocks, or decode (records the loss of encoding identity). nn split decomposes by layer into child selections plus plans or bundles, with coverage partitioning and member-vs-unique byte accounting.

binfiddle nn slice --catalog m.nn.json --selection w.sel.json --dry-run
binfiddle nn slice --catalog m.nn.json --selection w.sel.json --out-dir slices/w
binfiddle nn slice --catalog m.nn.json --selection q.sel.json --storage reference
binfiddle nn assemble --bundle slices/w --out-dir rebuilt/w
binfiddle nn split --catalog m.nn.json --pack p.yaml --storage materialized --out-dir split/

Inspection — `analyze`

Numerical inspection with honest coverage modes (metadata reads no payload, sample is seeded and deterministic, full scans within its budget): Welford statistics, min/max with coordinates and ties, non-finite category counts, overflow-safe L2 norm, histograms, reference-error metrics (MAE/RMSE/maxAE with explicit zero-denominator policies), and quantization block views through the reference decoder. --selection runs one uniform mode over every tensor of a selection in a single table with robust cross-tensor outlier flags.

binfiddle nn analyze --catalog m.nn.json --tensor w
binfiddle nn analyze --catalog m.nn.json --tensor w --mode sample --seed 17 --sample-size 4096
binfiddle nn analyze --catalog m.nn.json --tensor w --reference ref.f32
binfiddle nn analyze --catalog m.nn.json --tensor q4w --blocks 4
binfiddle nn analyze --catalog m.nn.json --selection gates.sel.json --mode sample

Storage codecs — `exl3 decode`, `exl3 block`

Qualified decode of EXL3 trellis storage, statically: the four fields of an EXL3 projection (.trellis, .suh, .svh, .mcg) resolve from a saved catalog by trellis name, the selector must carry the mcg codebook magic, and each weight decodes through the 16-bit window ending at its last code bit inside its 16×16 tile. decode reports one Hadamard-domain value Wq[n, k]; block summarizes a logical 128×128 block after the reconstruction W = H·diag(svh)·Wq·diag(suh)·H. The codec is a transcription of the reference CUDA decode, qualified bit-for-bit against an independent implementation on authentic samples — and it claims dequantized storage values, never model quality.

binfiddle nn exl3 decode --catalog k2.nn.json --trellis model.layers.12.mlp.experts.27.up_proj.trellis --index 0,0
# → value: -1.2127686
binfiddle nn exl3 block --catalog k2.nn.json --trellis model.layers.12.mlp.experts.27.up_proj.trellis --origin 0,0

Mutation — `edit set`, `edit apply`, `edit undo`, `edit prune`

Transactional fixed-size edits: the plan records the write unit, the observed preimage bytes, and the replacement; apply re-verifies catalog + source digest

preimage, writes a fresh output (originals never modified), verifies every byte outside the span unchanged, and revalidates the container. Sub-byte (Q4_0 nibble) writes preserve the neighbor by mask. Undo bundles reverse an edit against its exact edited revision. Multi-write sessions stage any number of operations (staging-time conflict checks) and apply them all in one copy pass — one model copy serves many edits. edit prune removes MLP channels structurally (gate/up rows, down columns) into a fresh SafeTensors file. Negative values need --value=-0.5 syntax.

binfiddle nn edit set --catalog m.nn.json --tensor w --index 0,0 --value 9 \
    --save-plan e.plan.json
binfiddle nn edit apply --catalog m.nn.json --plan e.plan.json \
    --out-model edited.safetensors --undo-bundle undo/e
binfiddle nn edit undo --bundle undo/e --target edited.safetensors \
    --out-model restored.safetensors     # byte-identical to the original

# Many edits, one copy pass
binfiddle nn edit set --catalog m.nn.json --tensor w --index 0,0 --value 101 --session sess/
binfiddle nn edit set --catalog m.nn.json --tensor w --index 3,3 --value 202 --session sess/
binfiddle nn edit apply --catalog m.nn.json --session sess/ \
    --out-model edited.safetensors --undo-bundle undo/
binfiddle nn edit prune --catalog m.nn.json --pack p.yaml --channels 0 \
    --out-model pruned.safetensors

Comparison — `diff`, `fingerprint`, `adapter`, `tokenizer`

Layered diff that never confuses content changes with repacks (same bytes, different offsets), descriptor changes, or package membership; a missing tensor is never a zero tensor. --precision audits decoded error metrics (per-tensor MAE/RMSE/maxAE/relative-L2 under each side's own codec) for fp32-vs-fp16 and quantized-variant families. Exact fingerprints are content-identity records — evidence, never lineage claims. With --compare, an experimental evidence graph adds exact-payload, structural, and sampled-block similarity edges (method/threshold/score recorded; unsampled bytes never certified). Adapter inspection inventories LoRA factor pairs at the descriptor level; tokenizer commands inspect assets and diff vocabularies without ever rendering templates.

binfiddle nn diff --left v1.nn.json --right v2.nn.json --decoded --policy exact_bits
binfiddle nn fingerprint --catalog v1.nn.json
binfiddle nn fingerprint --catalog v1.nn.json --compare v2.nn.json --threshold 0.75
binfiddle nn adapter inspect --catalog adapter.nn.json
binfiddle nn tokenizer inspect --package model-dir/
binfiddle nn tokenizer diff --left v1/tokenizer.json --right v2/tokenizer.json

Packages — `pack verify/lint/scaffold`, `partition`, `ledger`

Model packs are pure-data YAML manifests mapping tensor-name patterns to named components with shape expressions over configuration parameters — no executable content, ever. Recognition keeps contradictions visible; scaffolding derives heuristic-labeled drafts from an observed catalog. nn partition plans contiguous layer groups per stage balanced by encoded weight bytes — a static estimate that states exactly what it excludes (activations, workspace, runtime behavior); unlayered tensors are reported, never silently distributed. nn ledger then accounts that plan against supplied per-device capacity observations (with provenance: source, timestamp, ECC state), placing stage i on device i % N: replication multiplies encoded bytes, overheads are tracked separately, and the per-rank verdict stands even when the nominal aggregate would fit — exit 7 on any device over its selected capacity.

binfiddle nn pack verify --pack my-model/pack.yaml
binfiddle nn pack lint --pack my-model/pack.yaml
binfiddle nn pack scaffold --catalog m.nn.json > draft.pack.yaml
binfiddle nn partition --catalog m.nn.json --pack my-model/pack.yaml --stages 2
binfiddle nn ledger --stages-json plan.json --devices devices.json --replication 2

Validation & research — `validate`, `carve`, `research align`

Precise per-source structural verdicts (structurally_valid_for_reader, unsupported_feature, invalid, incomplete; exit 7 on defects); carving of embedded SafeTensors/GGUF containers out of raw files with stated confidence; and exact, reproducible research: row-permutation alignment decided by per-row digest multisets with ambiguity counted, never guessed.

binfiddle -i model-dir/ nn validate
binfiddle nn carve --target disk.img
binfiddle nn research align --left v1.nn.json --right v2.nn.json

Reading `nn` output

Every command takes --report-format json and emits the same canonical envelope (binfiddle.nn.result/v1) with status, coverage, diagnostics, and the semantic payload — integers as decimal strings, byte-identical output for identical runs. Exit codes are stable: 2 invalid request, 3 unsupported/ambiguous, 4 malformed, 5 source changed/missing/conflict, 6 I/O or budget, 7 validation failed, 8 incomplete, 130 cancelled.

Process memory — Linux experimental

Read memory from the current process or any same-user process via /proc/<pid>/mem, list mapped memory regions, and write back to the current process with an explicit opt-in.

# List memory regions of the current process
binfiddle --process-self --list-regions

# List regions of another process
binfiddle --pid 1234 --list-regions

# Read 16 bytes from the current process at a known address
binfiddle --process-self --address 0x7ffd12345678 --size 16 read 0..16

# Read from another process
binfiddle --pid 1234 --address 0x7f8a1b2c3000 --size 16 read 0..16

# Search process memory for a pattern
binfiddle --process-self --address 0x400000 --size 0x1000 search 474343

# Overwrite 4 bytes in the current process (requires --allow-write)
binfiddle --process-self --address 0x7ffd12345678 --size 4 \
    --allow-write write 0 DEADBEEF

# Overwrite bytes in another process's writable memory
binfiddle --pid 1234 --address 0x7f8a1b2c3000 --size 4 \
    --allow-write write 0 CAFEBABE

# Force-write a read-only region in the current process (dangerous)
binfiddle --process-self --address 0x7ffd12345678 --size 4 \
    --allow-write --force-writable write 0 DEADBEEF

# Force-write a read-only region in another process (Linux x86_64/aarch64, dangerous)
binfiddle --pid 1234 --address 0x7f8a1b2c3000 --size 4 \
    --allow-write --force-writable write 0 CAFEBABE

# Sparse read: zero-fill inaccessible pages
binfiddle --process-self --address 0x400000 --size 0x100000 \
    --zero-fill-inaccessible read 0..0x100000

Process Memory Options:

Option	Description
`--process-self`	Target the current process.
`--pid <PID>`	Target another process.
`--list-regions`	Print memory regions from `/proc/<pid>/maps`.
`--address`	Base address to read from or write to (hex or decimal).
`--size`	Number of bytes to read or write (hex or decimal).
`--allow-write`	Opt-in required for any process-memory write.
`--force-writable`	Temporarily make read-only pages writable before writing.
`--zero-fill-inaccessible`	Fill unreadable pages with zeroes.
`--skip-inaccessible`	Skip unreadable pages (read command only).

Note: --force-writable uses mprotect for the current process and ptrace syscall injection for cross-process writes. It is inherently risky and should be used with care.

`struct <TEMPLATE>` — Parse binary data using structure templates

Parse and interpret binary data according to YAML structure templates, useful for analyzing file headers, network protocols, and data structures.

# Parse an ELF file header
binfiddle struct elf_header.yaml < /bin/ls

# List fields in a template without parsing data
binfiddle struct my_format.yaml --list-fields

# Get a specific field value
binfiddle struct header.yaml --get version < firmware.bin

# Output as JSON
binfiddle struct format.yaml --output-format json < data.bin

# Output as YAML
binfiddle struct format.yaml --output-format yaml < data.bin

Template Format (YAML):

name: MyHeader
description: Binary header structure
endian: little  # or 'big'
fields:
  - name: magic
    offset: 0x00
    size: 4
    type: hex_string
    assert: "7f454c46"  # Verify magic bytes
    description: "Magic number"
  - name: version
    offset: 0x04
    size: 2
    type: u16
    enum:
      "1": "v1.0"
      "2": "v2.0"

Supported Field Types:

Type	Size	Description
`u8`, `u16`, `u32`, `u64`	1/2/4/8 bytes	Unsigned integers
`i8`, `i16`, `i32`, `i64`	1/2/4/8 bytes	Signed integers
`hex_string`	Variable	Raw bytes as hex
`string`	Variable	ASCII/UTF-8 string
`bytes`	Variable	Raw byte array
`computed`	—	Virtual field from an expression

Dynamic Templates:

Templates support field references ($fieldname), magic variables ($@prev_end, $@file_size), conditional fields (when:), computed fields, bitfields, counted arrays, and bit-level fields (bit_offset/bit_size).

fields:
  - name: filename_length
    offset: 0x1A
    size: 2
    type: u16
  - name: filename
    offset: 0x1E
    size: $filename_length
    type: string

Struct Options:

Option	Description
`--list-fields`	List template fields without parsing data
`--get <FIELD>`	Get specific field value(s)
`--output-format <FMT>`	Output format: human, json, yaml

Range Syntax

Syntax	Meaning
`10`	Single byte at index 10
`10..20`	Bytes 10-19 (10 bytes)
`10..`	Byte 10 to end of file
`..20`	Bytes 0-19
`..`	Entire file
`0x100`	Hex index (256)
`0x100..0x200`	Hex range

Output Formats

Format	Example Output
`hex`	`de ad be ef`
`dec`	`222 173 190 239`
`oct`	`336 255 276 357`
`bin`	`11011110 10101101 10111110 11101111`
`ascii`	`....` (non-printable shown as `.`)
`raw`	Raw binary bytes (no formatting)

Large Files & Streaming

Binfiddle handles large files through two mechanisms:

Memory-mapped file input (memmap2) — read-only operations on file-backed data do not load the whole file into RAM.
Streaming/block modes — commands can read the input in fixed-size blocks, keeping only a small window resident.

Commands that benefit from streaming:

Command	Flag	Example
`hash`	`--stream --read-block-size`	`binfiddle -i huge.bin hash sha256 --stream --read-block-size 64M`
`search`	`--block-size`	`binfiddle -i huge.bin search DEADBEEF --all --block-size 64M`
`analyze`	`--block-size`	`binfiddle -i huge.bin analyze entropy --block-size 64M`

Size-changing edits (insert, remove, replace) still require an in-memory copy because the file length changes.

Examples

Firmware Analysis

# Check ELF magic bytes
binfiddle -i firmware.bin read 0..4
# Output: 7f 45 4c 46

# Extract section as ASCII
binfiddle -i firmware.bin read 0x1000..0x1100 --format ascii

# Patch version string
binfiddle -i firmware.bin edit replace 0x200..0x210 "v2.0.0" \
    --input-format ascii -o patched.bin

# Find encrypted sections via entropy
binfiddle -i firmware.bin analyze entropy --block-size 4096

# Hash the firmware
binfiddle -i firmware.bin hash sha256

Binary Diffing

# Compare two firmware versions
binfiddle diff v1.bin v2.bin --diff-format unified --context 5

# Generate a patch file
binfiddle diff original.bin modified.bin --diff-format patch > changes.patch

# Quick comparison with summary
binfiddle diff v1.bin v2.bin --summary

# Compare ignoring timestamps at offset 0x10
binfiddle diff v1.bin v2.bin --ignore-offsets "0x10..0x18"

Security Analysis

# Check for high-entropy (encrypted/packed) sections
binfiddle -i suspicious.exe analyze entropy --block-size 4096 --output-format csv > entropy.csv

# Analyze byte distribution for anomalies
binfiddle -i malware.bin analyze histogram --output-format json

# Search for shellcode patterns
binfiddle -i dump.bin search "31 c0 50 68" --all --context 16

# Compute multiple hashes for a sample
binfiddle -i malware.bin hash md5
binfiddle -i malware.bin hash sha256
binfiddle -i malware.bin hash xxhash64

Data Recovery

# Search for JPEG headers
binfiddle -i disk.img search "FF D8 FF" --all --offsets-only

# Extract data at found offset
binfiddle -i disk.img read 0x15000..0x20000 -o recovered.jpg

Pipeline Integration

# Read from stdin
cat data.bin | binfiddle read 0..16

# Chain with other tools
binfiddle -i binary read 0..100 | xxd -r -p > raw.bin

# Use in scripts
MAGIC=$(binfiddle -i file.bin read 0..4)
if [ "$MAGIC" = "7f 45 4c 46" ]; then
    echo "ELF file detected"
fi

# Verify a release checksum list
binfiddle hash sha256 --check SHA256SUMS

Scripted Patching

#!/bin/bash
# Patch multiple offsets
OFFSETS=(0x100 0x200 0x300)
for offset in "${OFFSETS[@]}"; do
    binfiddle -i target.bin write "$offset" 90 -o temp.bin
    mv temp.bin target.bin
done

Batch Checksums

# Create a SHA-256 checksum file for every file in a directory
for f in *.bin; do
    binfiddle -i "$f" hash sha256
done > SHA256SUMS

# Verify them later
binfiddle hash sha256 --check SHA256SUMS

Architecture

Module Structure

src/
├── main.rs             # CLI entry point and argument parsing
├── lib.rs              # Library exports
├── error.rs            # Error types (BinfiddleError)
├── process_memory.rs   # Linux process memory helpers
├── commands/
│   ├── mod.rs          # Command trait and exports
│   ├── read.rs         # Read command
│   ├── write.rs        # Write command
│   ├── edit.rs         # Edit command (insert/remove/replace)
│   ├── search.rs       # Search command (exact/regex/mask/parallel)
│   ├── hash.rs         # Hash and checksum verification
│   ├── analyze.rs      # Analyze command (entropy/histogram/IC)
│   ├── diff.rs         # Diff command (simple/unified/side-by-side/patch)
│   ├── convert.rs      # Convert command (encoding/line endings/BOM)
│   ├── patch.rs        # Patch command (apply/revert binary patches)
│   ├── chain.rs        # Command chaining
│   └── struct_cmd.rs   # Struct command (template-based parsing)
├── nn/                 # NN workbench library (nn commands are a thin CLI
│                       # over this reusable layer; see docs/NN_USAGE.md)
└── utils/
    ├── mod.rs          # Utility exports
    ├── parsing.rs      # Range and format parsing
    ├── display.rs      # Output formatting
    └── progress.rs     # Progress bar helpers

Core Data Model

pub struct BinaryData {
    chunk_size: usize,   // Display chunk size in bits (1-64)
    width: usize,        // Chunks per output line
    source: BinarySource,
    // Backing storage is either an owned Vec<u8>, a read-only Mmap,
    // or a writable MmapMut for in-place file modifications.
}

pub enum BinarySource {
    File(PathBuf),              // Read from file (memory-mapped)
    WritableFile(PathBuf),      // File opened for in-place mutation
    Stdin,                      // Read from stdin
    RawData(Vec<u8>),           // In-memory data
    ProcessSelf { .. },         // Current process memory
    Process { .. },             // Another process's memory
    MemoryAddress(usize),       // Raw memory address (reserved)
}

Error Handling

All operations return Result<T, BinfiddleError> with specific error types:

Io — File not found, permission denied, etc.
Parse — Invalid hex, decimal, template YAML, or format strings
InvalidRange — Out of bounds or invalid range specification
InvalidChunkSize — Chunk size is 0 or exceeds data
InvalidInput — Unknown format or invalid input
UnsupportedOperation — Feature not yet implemented
ProcessMemoryError — /proc/<pid>/mem access or permission failure
ChecksumVerificationFailed — hash --check mismatch

Contributing

Contributions are welcome!

Building and Testing

# Run tests
cargo test --release

# Run with verbose output
cargo test --release -- --nocapture

# Build release
cargo build --release

# Build for all platforms
./scripts/build_releases.sh --native

# Cross-check aarch64 Linux
cargo check --target aarch64-unknown-linux-gnu

Code Style

Follow Rust standard formatting (cargo fmt)
Run clippy with zero warnings (cargo clippy --all-targets -- -D warnings)
Add tests for new functionality
Document public APIs with doc comments

Roadmap

Phase	Theme	Status
1	Core read/write/edit	✅ Complete
2	Search, analyze, diff	✅ Complete
3	Convert, patch, struct	✅ Complete
4	Template system evolution	✅ Complete
5	Bit-level precision	✅ Complete
6	Command chaining & pipelines	✅ Complete
7	Live process memory	✅ Complete
8	Large files, hashing, streaming, progress	✅ Complete
9	NN artifact workbench (`nn`)	✅ Complete (early access; runtime replay intentionally out of scope)
10	NN hardening: PyTorch tier, edit sessions, MoE expert views, shard planning, precision audit	✅ Complete (v0.29.0)
11	NN pipeline program: NumPy captures, build provenance, EXL3 profile + qualified codec, evidence sidecars, derived views, per-GPU ledger	✅ Complete (v0.30.0)
12	Advanced analysis & intelligence	🔲 Planned

Documentation & references

Classic toolkit

docs/USAGE.md — the full user guide (every command, streaming, troubleshooting, cookbook)
docs/context.md — binfiddle in the command-line ecosystem (vs. xxd, radare2, model tooling)
This README's Command Reference — syntax, options, and examples for every classic command

NN workbench (`nn`)

docs/NN_USAGE.md — the complete manual: concepts (catalogs, selections, plans, bundles, packs, envelopes), every command with full option tables, selector grammar, exit codes, JSON envelope reference, pack authoring, howtos, boundaries
docs/NN_QUICK_REFERENCE.md — one-page cheat sheet: pipeline map, command table, selector grammar, exit codes

Real-model deep dives

Complete walkthroughs of public model releases, every output a real capture — the fastest way to see the workbench in action on real weights:

Dissecting Kokoro-82M — the pickle boundary (a PyTorch checkpoint measured as a ZIP), the ONNX descriptor tier, statistics that find real structure, transactional editing, and an fp32/fp16/q8 precision comparison
Dissecting Gemma-4 E4B — a 16 GB SafeTensors release: the E4B per-layer embedding slab, addressing at offset 14.6 billion, a model pack for a dual-head-dimension architecture, and a two-byte transactional edit verified against the whole file
Dissecting Qwen3.8-Flash-Next — a 131-shard, 360 GB release dissected from a curated 9-shard subset: sharded-package findings, hybrid linear/full anatomy, fused head views carved from real tensors, and stacked MoE experts
Dissecting EXL3 storage on GLM-5.3-Flash — quantized projections as four-field storage groups (trellis/mcg/ suh/svh), a variant-versioned profile pack, a mismatched-variant contradiction demo on authentic payloads, and the qualified trellis codec decoding them to Hadamard-domain values

License

This project is licensed under the BSD-3-Clause License. See LICENSE for details.

Acknowledgments

Built with Rust
CLI parsing by clap
Pattern matching by memchr and regex
Parallel processing by rayon
Memory mapping by memmap2
Progress bars by indicatif
Hashing by sha2, sha1, blake3, md5, crc32fast, and xxhash-rust

Pro Tip: Combine with radare2 for full analysis workflows:

# Extract .text section using radare2 section info
RANGE=$(rabin2 -S binary | awk '/\.text/{print $2".."$3}')
binfiddle -i binary read "$RANGE" -o text.bin

# Analyze entropy of extracted section
binfiddle -i text.bin analyze entropy --block-size 256

# Hash the extracted section
binfiddle -i text.bin hash sha256

Name		Name	Last commit message	Last commit date
assets		assets
benches		benches
docs		docs
packs		packs
scripts		scripts
src		src
structs_library		structs_library
tests		tests
.gitignore		.gitignore
.zcodeignore		.zcodeignore
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
LICENSE		LICENSE
README.md		README.md
build.rs		build.rs
q.nn.json		q.nn.json

Latest commit

History

Folders and files

Repository files navigation

Binfiddle

🌐 Overview

Features

Core Capabilities

Key Differentiators

Design Principles

Installation

From Source (Recommended)

From Releases

Build for Other Platforms

Cross-check

Quick Start

Command Reference

Global Options

Process-Memory Global Options (Linux only)

Commands

read <RANGE> — Read bytes from binary data

write <POSITION> <VALUE> — Write bytes to binary data

edit <OPERATION> <RANGE> [DATA] — Structural modifications

search <PATTERN> — Find patterns in binary data

hash <ALGORITHM> — Compute digests

analyze <TYPE> — Statistical analysis of binary data

diff <FILE1> <FILE2> — Compare two binary files

convert — Text encoding and line ending conversion

patch <TARGET> <PATCH_FILE> — Apply binary patches

chain --step <COMMAND> — Execute multiple commands in sequence

nn — Neural-network artifact workbench (early access)

Discovery — discover, ls, show, select

Addressing — where, locate, impact

Extraction — slice, split, assemble

Inspection — analyze

Storage codecs — exl3 decode, exl3 block

Mutation — edit set, edit apply, edit undo, edit prune

Comparison — diff, fingerprint, adapter, tokenizer

Packages — pack verify/lint/scaffold, partition, ledger

Validation & research — validate, carve, research align

Reading nn output