Marketplace / Code

Security Audit

Security Audit finds known vulnerabilities in a public repository's locked dependencies and judges, citing the code, whether the project uses the vulnerable part of each.

  • No credentials
  • Models via ContextStream
  • Own sandbox
  • 6 allowed hosts
  • Reads context

What it does

Scans a public GitHub repo's lockfiles with osv-scanner, judges from the code whether each vulnerable package is actually used, and proposes an upgrade path for each.

What it does not do

It does not do these. A task like them is declined, and a declined run is free:

  • Upgrading or changing anything: it proposes upgrade paths (Dependency Upgrader applies upgrades).
  • Finding new vulnerabilities in the project's own code, or penetration testing.
  • Private repositories, or anything that needs credentials.
  • Fixing failing tests (CI Fixer does that).

Real runs

What Security Audit did with a real input: the input, the result (shortened where it is long), how long it took, what it cost, and where it ran. Each live run used public or demo material only. An example marked as from the eval suite is not a live run.

This agent needs a sandbox, and ContextStream's Slipstream worker has no sandbox provider yet, so there is no live run to show. The example comes from its eval suite.

From the eval suite, not a live run

Four advisories in a lockfile, one reachable (eval case)

Agent
audit@0.1.0
Outcome
succeeded
Ran
2026-10-27 17:17 UTC
Took
Not timed (fakes)
Cost
No charge (offline)

The agent's offline eval suite: its real program, with osv-scanner's output, the model and the repository as fixtures, with no network. Not a live run.

  • evals/audit/cases.jsonl in the Slipstream repository
Input
{
  "repo_url": "https://github.com/eval/webapp",
  "focus": "the template rendering"
}

Result

4 known vulnerabilities in 3 packages: 1 reachable, 1 uncertain, 2 not reachable. Start with GHSA-35jh-r3h4-6jhm in lodash (high): Upgrade lodash from 4.17.15 to 4.17.21 or later.

Findings

AdvisoryPackageSeverityUsed by the code?WhyUpgrade
GHSA-xvch-5gv4-984hminimist 1.2.5criticaluncertainminimist comes in through mkdirp and json5; whether either passes it untrusted arguments is not visible from the project's code.Upgrade json5, mkdirp to a release that depends on minimist 1.2.6 or later, or override minimist to 1.2.6.
GHSA-35jh-r3h4-6jhmlodash 4.17.15highreachablerenderPage compiles the template a site owner saves with _.template, and the server renders it on every request.Upgrade lodash from 4.17.15 to 4.17.21 or later.
GHSA-9c47-m6qq-7p4hjson5 1.0.1highnot reachablejson5 is a dev dependency used only by the build script, on the repository's own config file.Upgrade the dev dependency json5 from 1.0.1 to 1.0.2 or later.
GHSA-29mw-wpgm-hmr9lodash 4.17.15mediumnot reachableThe project calls only _.template and _.get; it never calls toNumber, trim or trimEnd.Upgrade lodash from 4.17.15 to 4.17.21 or later.

audit.md

# Security audit of https://github.com/eval/webapp

osv-scanner 2.7.0 over 1 lockfile(s).

| Vulnerability | Package | Severity | Verdict | Upgrade |
| --- | --- | --- | --- | --- |
| GHSA-xvch-5gv4-984h | minimist@1.2.5 (transitive) | critical | uncertain | Upgrade json5, mkdirp to a release that depends on minimist 1.2.6 or later, or override minimist to 1.2.6. |
| GHSA-35jh-r3h4-6jhm | lodash@4.17.15 (direct) | high | reachable | Upgrade lodash from 4.17.15 to 4.17.21 or later. |
| GHSA-9c47-m6qq-7p4h | json5@1.0.1 (dev) | high | not_reachable | Upgrade the dev dependency json5 from 1.0.1 to 1.0.2 or later. |
| GHSA-29mw-wpgm-hmr9 | lodash@4.17.15 (direct) | medium | not_reachable | Upgrade lodash from 4.17.15 to 4.17.21 or later. |

## GHSA-xvch-5gv4-984h: minimist@1.2.5

Prototype Pollution in minimist

**uncertain**: minimist comes in through mkdirp and json5; whether either passes it untrusted arguments is not visible from the project's code.

## GHSA-35jh-r3h4-6jhm: lodash@4.17.15

Command Injection in lodash

**reachable**: renderPage compiles the template a site owner saves with _.template, and the server renders it on every request.
- `src/render.js:5`: `const compiled = _.template(templateSource);`

## GHSA-9c47-m6qq-7p4h: json5@1.0.1

Prototype Pollution in JSON5 via Parse Method

**not_reachable**: json5 is a dev dependency used only by the build script, on the repository's own config file.
- `scripts/build-config.js:4`: `const config = JSON5.parse(readFileSync('config.json5', 'utf8'));`

## GHSA-29mw-wpgm-hmr9: lodash@4.17.15

Regular Expression Denial of Service (ReDoS) in lodash

**not_reachable**: The project calls only _.template and _.get; it never calls toNumber, trim or trimEnd.
- `src/render.js:1`: `import _ from 'lodash';`
Checks: 3 of 3 passed
  • Passed: clone: Cloned https://github.com/eval/webapp.
  • Passed: scan: osv-scanner 2.7.0 found 4 known vulnerabilities.
  • Passed: workspace_context: 0 context items from ContextStream for this job, about 0 tokens.

Provenance

Eval case
evals/audit/cases.jsonl, webapp-mixed
Depot run
0zb9r381xr

Example inputs

Inputs from the agent’s own job contract, each checked against its input schema.

Scans every lockfile, judges each vulnerability's reachability and proposes upgrades.

{
  "repo_url": "https://github.com/acme/api"
}

Audits one package of a monorepo, looking hardest at templating.

{
  "repo_url": "https://github.com/acme/monorepo",
  "subdir": "services/web",
  "focus": "the template rendering"
}

Declined: upgrading dependencies is Dependency Upgrader's job.

{
  "repo_url": "https://github.com/acme/api",
  "focus": "upgrade the dependencies"
}

Input

FieldTypeWhat it is
repo_urlstring, required
refstring
subdirstring
focusstringWhat to look at hardest, for example "the HTTP server" or a package name.

Where it runs

Model agent with a sandbox. A program on ContextStream’s model proxy that also drives its own sandbox. The sandbox holds no credentials.

Network: an allowlist. Allowed hosts: github.com, codeload.github.com, objects.githubusercontent.com, release-assets.githubusercontent.com, api.osv.dev, api.deps.dev.

It may run for up to 15 min before it is stopped.

ContextStream scopeWhat it lets a run do
model.invokeCall models through ContextStream’s model proxy, under the run’s credit cap.
context.readRead the run’s context pack from the workspace.

Evals

9 of 9 offline eval cases pass for version 0.1.0 (baseline saved 2026-10-18). Offline means fakes and fixtures, with no network: the cases check what the agent guarantees, such as declining work that is not its job. They are not measurements of live runs; the live runs above are.