Cache invalidation¶

plintz caches Stored Signal Samples and Calculated Item results in the block cache. It reads from the cache until an Invalidation request discards the data. An Invalidation can target a Datasource or one Data ID, either for all time or over a [start, end) range.

When to invalidate¶

plintz does not watch its Datasources for changes. If a pipeline corrects or backfills existing data, the cache keeps the old Samples until the pipeline requests Invalidation. Make that request the pipeline's final step.

• Do not invalidate Uncertain Data. plintz never caches it and reads it from the Datasource on every request, so invalidating that region has no effect.
• You do not need a separate Invalidation request for Calculated Items. Invalidating a Stored Signal reaches every Calculated Item that depends on it, through any chain.
• Changes in plintz do not need Invalidation. Editing a Formula or its bindings, or an Administrator changing a Stored Signal's property, gives affected Calculated Items new blocks.

Calling the Invalidation API¶

POST /api/invalidation, through the Workbench's address, authenticated with the service token as Authorization: Bearer <token>. A user's session doesn't authenticate it.

The body is a batch, a JSON list with one entry for each piece of data to invalidate:

For example, to invalidate the hour that a pipeline corrected for the Data ID sine of the Datasource synthetic:

curl -X POST \
  -H "Authorization: Bearer $INVALIDATION_TOKEN" \
  -H 'Content-Type: application/json' \
  -d '[{"datasource": "synthetic", "data_id": "sine", "start": "2025-01-01T00:00:00Z", "end": "2025-01-01T01:00:00Z"}]' \
  http://localhost:8080/api/invalidation

Other entries:

{"datasource": "synthetic", "data_id": "sine", "start": "2025-01-01T00:00:00Z"}
{"datasource": "synthetic", "data_id": "sine"}
{"datasource": "synthetic"}

These invalidate sine from 1 January 2025 onward, all of sine, and every Stored Signal in the Datasource. Use the narrowest range available. It drops fewer blocks and reduces reads from the Datasource.

The answers:

• 204: the blocks are dropped. A Data ID with no cached blocks isn't an error. The call is recorded in the Audit Trail as one entry, with the actor service token and the batch in its state after, an empty batch too, in the same transaction that drops the blocks: if recording fails, the request fails and no block is dropped.
• 401: the token is missing or invalid.
• 422: an entry has start at or after end, a timestamp without a time zone, or a datasource that isn't configured. plintz rejects the whole batch without dropping any blocks.

Clearing a Datasource's cache from the Workbench¶

An Administrator can invalidate the whole of a Datasource without a service token: Clear cache on the Datasources tab of the Administration page does what an entry with only a datasource does, after a confirmation. It is for a Datasource whose data changed in ways nobody reported. The page says when the blocks are dropped. Use the API with a range for a known correction, which drops far less.

What an invalidation drops¶

plintz drops every cached block that overlaps an entry's range. Blocks are one hour long, aligned to the UTC epoch, so a 5-minute correction drops one block, or two if it spans an hour boundary. A range that ends exactly where a block begins leaves that block cached. A block's Samples and its Rollups are dropped together.

The next read of a dropped block fetches it from the Datasource again. A fetch that was already in flight when Invalidation dropped the block does not cache its data. The next read recomputes any Calculated Item that used the block. These fetches and computations happen only when a client requests the data.

The block cache's disk budget¶

The block cache's files are in cache_dir, within the block-cache volume, and its index is in Postgres. The files stay within cache_disk_budget_gib, the budget you set in plintz.yaml:

• If a read would push the cache over budget, plintz evicts the least recently read blocks, including their Samples and Rollups, until the cache fits. A cache read or fill marks a block as the most recently read.
• At startup, plintz evicts as well if you lowered the budget since the blocks were cached. To change the budget, edit plintz.yaml and recreate the backend.
• Blocks of Calculated Items count against the budget, and those that Invalidation or an edit made stale stay until they are evicted.
• Set the budget below the free space of the volume, so that the disk doesn't fill before plintz evicts.
• The Configuration tab of the Administration page shows how much of the budget the block files use.

A missing or unreadable block file causes a cache miss. plintz logs a warning when it cannot read a file. See Logs. If you lose the block-cache volume, plintz fetches missing blocks from the Datasource again.