ourfault

More about ourfault

What else ourfault does, how its judgements are checked, and what it keeps of the telemetry you send.

What else it does

  • Urgency first

    Each failure is judged urgent, soon or later, so the ledger reads from the top down.

  • One defect, one issue

    The same defect in a new shape is linked to its open issue instead of filed again.

  • Noise held, not hidden

    A failure of a kind your team keeps closing unread is held, and the ledger says why and lets an owner or admin file it anyway.

  • Monday's assessment

    Each Monday the owners get the week before in five sentences, by email and on the ledger.

  • Findings from metrics

    Every hour your metrics are read for changes worth a look before they turn into failures, such as memory that keeps rising.

  • Commands in issues

    Comment @ourfault-dev explain, status, wrong or mute on an issue it filed, and ourfault reacts, then replies where there is something to say.

Honest by construction

  • Reasons on both sides

    Every verdict comes with the evidence that counted for your fault and the evidence that counted against it.

  • Measured in the open

    Each judgement is measured against a labelled set, and the latest numbers are on this page.

  • Bounded

    Judgements count against a monthly allowance and issues against hourly limits, so nothing runs without a ceiling.

  • Nothing to watch

    There is no chat and no dashboard for its own sake: the work arrives in your repository, and the workspace keeps the record.

The numbers

Each judgement is measured against a labelled set before and after any change to it. Last measured .

What is judgedLabelled setResult
Whose fault, given every kind of evidence50 failures47 right
Whose fault, given your deployments alone50 failures38 right
Whose fault, given the failing request's trace12 failures11 right
The same defect as an open issue16 failures15 right, no false links
How urgent17 failures16 right
A change in a service's metrics5 services with one5 found
No change in a service's metrics315 services5 false alarms (1.6%)

The sets were written alongside the rules, so the numbers show the judgements behave as intended, not how they will do on your traffic.

What is kept

Kept

  • A hash for each distinct failure, with its exception type and top frame
  • The files, functions and lines of your code its stack trace names
  • Two sample traces per failure, the first and the latest: their spans' timing, status and names
  • Each metric's name and unit, and its numbers added up per service and minute
  • Counts of what you send and what becomes of it
  • Service names, repositories and times
  • Our verdict and severity with the answers behind them, and the issue filed for it

Not stored

  • Log and message bodies
  • Exception messages and stack traces, once judged
  • Attribute values, and spans' attributes, events and links
  • Source files from your repository
  • Pull request changes
  • Ingest tokens, of which we keep a hash

A new failure's message and stack trace are held while it is judged and sent to our AI providers for judgement, then deleted. The source files read to find a root cause or propose a fix are held in memory only, and sent to our AI providers too; a pull request's changes are read in memory and dropped. The data retention policy lists every kind of data we keep and for how long. The security statement says how it is protected, and the GitHub App permissions page says what the App may do in your repositories. In strict mode a failure is kept without its exception type, frames or file names, and spans without their names. A filed issue holds the stack trace and the root cause, in your repository.