Hivemind Sign in
Tools

Flaky tests

Tool Code @dip497 0 installs

Runs the tests several times and shows each test's pass rate; the ones that pass only sometimes are flaky, and an agent can fix one.

hive plugins install @dip497/flaky-tests

Run it in a terminal, or find it in Hivemind under Settings ▸ Plugins.

OverviewInstallPermissionsVersions

Flaky tests

Runs the repository's tests several times, one after another, and shows every test's pass rate: a row of green and red squares, one per run, and how often it passed. A test that passed in some runs and failed in others is flaky, and comes first; "Ask an agent to fix" starts one in the tile's frame with the test and its history (you read the prompt before it is sent). A meter fills as the runs finish; stop after the current one at any time.

It reads the per-test lines of bun test, node --test (TAP and the spec reporter), any TAP runner, pytest -v, jest --verbose, vitest --reporter=verbose and cargo test (without -q, which prints dots in place of names).

hive plugins install @dip497/flaky-tests
hive plugins enable @dip497/flaky-tests
hive plugins set @dip497/flaky-tests command "pytest -v tests/"
hive plugins set @dip497/flaky-tests runs 10
hive ctl open-tool @dip497/flaky-tests/flaky

An agent drives it, while its tile is open:

hive ctl plugin call @dip497/flaky-tests run     # run them; answers within 25 s or keeps going
hive ctl plugin call @dip497/flaky-tests flaky   # the last runs' flaky tests

The command runs without a shell (no pipes, no &&, no environment assignments), and each run has the app's limit of 60 seconds and 10 MB of output. It asks for two permissions: start agents (workspace:spawn) and tell an agent something (workspace:prompt).

How it was made

Built by a coding agent (Claude) from the tool-plugin contract (the brief hive plugins new tool writes, and the tool plugin guide). It uses host.run of the test command you set, useTileState to keep the last results with the tile, and hm.commands.spawnAgent with the permissions above. Reading the reporters is in lib.js, tested on their real output.

A tool arrives switched off. Turn it on in Settings ▸ Plugins, or with hive plugins enable @dip497/flaky-tests, if you trust who wrote it.

Files

From dip497/hivemind-plugins at fb488a4. Hivemind checks each file's checksum as it downloads, and stops if one changed.

  • hivemind-tool.jsone61b18017d8d
  • icon.svgc835b955a50b
  • index.jsbce6c7c76ba4
  • lib.js9ab2971954be
  • README.md240d778131fd
hivemind-tool.json 
{
  "id": "@dip497/flaky-tests",
  "name": "Flaky tests",
  "version": "1.0.1",
  "apiVersion": 1,
  "minAppVersion": "2026.10.3",
  "category": "code",
  "description": "Runs the tests several times and shows each test's pass rate; the ones that pass only sometimes are flaky, and an agent can fix one.",
  "entry": "index.js",
  "permissions": [
    "workspace:spawn",
    "workspace:prompt"
  ],
  "commands": [
    {
      "name": "run",
      "description": "Run the test command the set number of times and answer with the flaky and always-failing tests. Waits up to 25 seconds; longer runs go on in the tile, then ask with flaky."
    },
    {
      "name": "flaky",
      "description": "The last runs' flaky tests (passed sometimes, failed sometimes), with how often each passed and failed, and the tests that always failed."
    }
  ],
  "settings": [
    {
      "id": "command",
      "label": "Test command",
      "type": "text",
      "default": "bun test",
      "maxLength": 300,
      "description": "Run in the repository without a shell: a program and its arguments, \"quoted\" words kept whole."
    },
    {
      "id": "runs",
      "label": "Runs",
      "type": "number",
      "min": 2,
      "max": 20,
      "default": 5,
      "description": "How many times to run it, one after another."
    }
  ],
  "tools": [
    {
      "key": "flaky",
      "label": "Flaky tests",
      "description": "Each test's pass rate over several runs.",
      "tileKind": "flaky-tests",
      "size": {
        "width": 900,
        "height": 640
      },
      "needsRepo": true,
      "singleton": true,
      "glyph": "≈",
      "icon": "icon.svg"
    }
  ]
}

Hivemind shows you this list and asks before it installs.

  • Runs inside Hivemind, with the app's access to your files and network, once you switch it on
  • Start agents in your workspace
  • Tell an agent something (you send or cancel each one)
VersionCommitListed
1.0.1 fb488a4 October 28, 2026

Needs Hivemind 2026.10.3 or newer.

HiveHub keeps the version listed last. The ones before it are in the repository's history.

Something wrong with it? Report it.