More about ourfault
What else ourfault does, how its judgements are checked, and what it keeps of the telemetry you send.
What else it does
-
Urgency first
Each failure is judged urgent, soon or later, so the ledger reads from the top down.
-
One defect, one issue
The same defect in a new shape is linked to its open issue instead of filed again.
-
Noise held, not hidden
A failure of a kind your team keeps closing unread is held, and the ledger says why and lets an owner or admin file it anyway.
-
Monday's assessment
Each Monday the owners get the week before in five sentences, by email and on the ledger.
-
Findings from metrics
Every hour your metrics are read for changes worth a look before they turn into failures, such as memory that keeps rising.
-
Commands in issues
Comment
@ourfault-dev explain,status,wrongormuteon an issue it filed, and ourfault reacts, then replies where there is something to say.
Honest by construction
-
Reasons on both sides
Every verdict comes with the evidence that counted for your fault and the evidence that counted against it.
-
Measured in the open
Each judgement is measured against a labelled set, and the latest numbers are on this page.
-
Bounded
Judgements count against a monthly allowance and issues against hourly limits, so nothing runs without a ceiling.
-
Nothing to watch
There is no chat and no dashboard for its own sake: the work arrives in your repository, and the workspace keeps the record.
The numbers
Each judgement is measured against a labelled set before and after any change to it. Last measured .
| What is judged | Labelled set | Result |
|---|---|---|
| Whose fault, given every kind of evidence | 50 failures | 47 right |
| Whose fault, given your deployments alone | 50 failures | 38 right |
| Whose fault, given the failing request's trace | 12 failures | 11 right |
| The same defect as an open issue | 16 failures | 15 right, no false links |
| How urgent | 17 failures | 16 right |
| A change in a service's metrics | 5 services with one | 5 found |
| No change in a service's metrics | 315 services | 5 false alarms (1.6%) |
The sets were written alongside the rules, so the numbers show the judgements behave as intended, not how they will do on your traffic.
What is kept
Kept
- A hash for each distinct failure, with its exception type and top frame
- The files, functions and lines of your code its stack trace names
- Two sample traces per failure, the first and the latest: their spans' timing, status and names
- Each metric's name and unit, and its numbers added up per service and minute
- Counts of what you send and what becomes of it
- Service names, repositories and times
- Our verdict and severity with the answers behind them, and the issue filed for it
Not stored
- Log and message bodies
- Exception messages and stack traces, once judged
- Attribute values, and spans' attributes, events and links
- Source files from your repository
- Pull request changes
- Ingest tokens, of which we keep a hash
A new failure's message and stack trace are held while it is judged and sent to our AI providers for judgement, then deleted. The source files read to find a root cause or propose a fix are held in memory only, and sent to our AI providers too; a pull request's changes are read in memory and dropped. The data retention policy lists every kind of data we keep and for how long. The security statement says how it is protected, and the GitHub App permissions page says what the App may do in your repositories. In strict mode a failure is kept without its exception type, frames or file names, and spans without their names. A filed issue holds the stack trace and the root cause, in your repository.