Four Languages, Four Public Projects: Before and After Artifact Checks
We tested one selected artifact from each of four open-source projects. A separate owned browser fixture adds a Chromium run under enforcing CSP in BoxFleet E2B. Each result states its artifact, environment and limits.
rea-agents@6.7.0; Python was parsed with its standard AST module; native executables were checked for one selected repository-path string. Counts from different analyzers are not directly comparable. The separate browser fixture run below executed in BoxFleet E2B.
1. Pinned Open-Source Inputs
- Marked at 2f907438: TypeScript Markdown parser, bundled from
src/marked.tsinto one Node.js ESM file. - Click at 2247b35e: Python CLI framework, with only
src/click/core.pyreplaced inside a copy of the package. - fzf at 55c122f1: Go fuzzy finder, built locally as a Windows executable.
- ripgrep at 3fce3b5b: Rust search CLI, built locally in release mode as a Windows executable.
2. Measured Before and After
The table names the exact static measure for each format. "After" is the transformed artifact under test. One selected artifact does not represent full-project protection.
| Project and measured unit | Before | After | Behavior checked |
|---|---|---|---|
| Marked REA AST nodes in one ESM bundle | 13,657 nodes 11 return-shape observations 18 exports | 480 nodes (-96.5%) 0 return-shape observations 18 exports | 4 Markdown outputs matched; export names matched |
| Click Python AST nodes in core.py | 14,857 nodes 4 selected strings visible | 133 nodes (-99.1%) 0 selected strings visible | 4 CLI cases matched, including an invalid option |
| fzf Exact repository-path byte matches in one PE executable | 1,898 matches 8,916,992 bytes | 0 matches 8,916,992 bytes | --version and a filter result matched on repeat |
| ripgrep Exact repository-path byte matches in one PE executable | 6 matches 4,913,152 bytes | 0 matches 4,913,152 bytes | --version and a fixed-string count matched |
3. How the Checks Were Run
Marked's bundled JavaScript was transformed and both artifacts were passed to rea-agents@6.7.0 analyze-javascript-application --format json. Click's Python module was transformed in a copied package and both versions were parsed with ast.walk. For fzf and ripgrep, we scanned both binaries for an exact repository-path byte string and compared two CLI controls. The run used Node.js 24.19.0, Python 3.12.10, Go 1.27.0, Rust 1.98.1, and esbuild 0.28.2.
An initial fzf control timed out after 30 seconds; running the baseline and transformed commands directly succeeded, and the repeated benchmark recorded matching output. These are a small set of smoke tests. AST and exact-string counts describe what those static checks saw; they do not demonstrate that an agent failed to recover behavior, nor do two CLI commands establish full binary parity.
4. Chromium under enforcing CSP in BoxFleet E2B
On October 21, 2026, a separate run exercised an owned ESM fixture in Chromium 156.0.8078.4 on Linux x64, inside a fresh BoxFleet E2B box. The fixture covers state changes, fractional amounts, Unicode strings and rejected inputs. Both versions matched the same independently specified outcomes. The verified JSON run report contains the scenarios, hashes, versions, observed CSP violations and run identifiers.
| Owned browser fixture | Before | After |
|---|---|---|
| Source artifact size | 819 bytes | 3,273 bytes |
| Scenarios matching expected outcomes | 12 / 12 | 12 / 12 |
| Export names | 5 | 5 |
| Enforcing script policy | script-src 'self' | script-src 'self' |
| eval, Function and WebAssembly compilation controls | 3 / 3 blocked, enforcement observed | 3 / 3 blocked, enforcement observed |
| Unhandled browser exceptions | 0 | 0 |
| Artifact SHA-256 | 738651799d9698223d797c756220860f861af4db0e1a654a815a3bf689b4ace1 | 3886ccfcffa9c1b49f76e17b97d147dfdf46fb61be3e55045c2d17e39a00a6aa |
The harness also rejected a deliberately wrong result and an incomplete scenario list. It used Node.js 22.23.3 and Playwright 1.64.1. The browser received the enforcing HTTP policy with CSP bypass disabled; both variants produced the expected blocking events. The run started at 2026-10-21T02:27:23.491Z and finished at 2026-10-21T02:27:36.682Z.
Scope: this verifies one owned fixture in one Chromium run. It does not qualify frameworks, PWAs, service workers, extensions, mobile runtimes or arbitrary web applications, and it does not measure reverse-engineering resistance. Chromium's process sandbox was disabled inside the E2B box; no additional Chromium sandbox or OpenShell enforcement is claimed. The report identifies the tested source snapshot through artifact and harness hashes.
See the earlier JavaScript bundle benchmark with separate before and after rows ↗