Qualflare
CLIImprovementv0.1.9

qf 0.1.9: no more false-green uploads, plus debug and privacy flags

qf 0.1.9 stops reporting failed or crashed tests as passing, adds --debug and --no-capture-output, and records platform and milestone on launches.

qf 0.1.9 makes the results Qualflare shows you match what actually happened in CI. A report with failures, crashes or unrecognized statuses can no longer appear as a passing launch.

What's new

  • No more false greens. Across all 16 report formats qf reads:
    • A test with an unknown status now counts as an error instead of a pass.
    • Suite totals are recalculated from the individual tests, so a report with real failures can't show as green.
    • Crashes that used to drop tests are reported as failures: a Jest suite crash, RSpec errors outside examples, a failing Cucumber Background, Go tests with no final event.
  • Framework-specific fixes. Interrupted Playwright runs are errors, while expected failures from test.fail() count as passed. Cypress it.skip is reported as skipped. A Newman request whose assertions were all skipped is skipped too.
  • --debug logs every request and response to stderr, with your token removed, so you can see what qf sent without leaking credentials.
  • --no-capture-output keeps captured stdout and stderr from pytest, PHPUnit and JUnit XML reports off Qualflare. Status, timing and failure messages are still uploaded.
  • --platform and --milestone. Tag a launch as android, ios, desktop, web or api, and link it to a milestone by its number.
  • Keep your token off the command line. qf login <id> now reads the token from QF_TOKEN, a hidden prompt or piped input, so it doesn't end up in shell history.

Also in this release

  • Clearer exit codes for scripts: 3 for authentication, 4 for quota or permission, 5 for not found, 7 for temporary errors.
  • Error messages include a request ID for support.
  • A file passed twice is counted once.
  • Release downloads are signed and ship with SBOMs.

How to get it

Upgrade with brew upgrade qualflare/tap/qf, or download the release from GitHub.

Learn more