Scheduled Views: Pay for the Expensive Query Once

Turn your heaviest recurring log queries into small, always-current tables, and keep the answers long after the raw logs are gone.

Table of Contents

Every observability team has the same dashboard panel. Error rate by service, 5-minute buckets, last 30 days. It's the first thing anyone opens during an incident, and it's the slowest thing on the page, because every load rescans the same mountain of raw log lines to compute the same numbers that were already computed a minute ago.

Then, eventually, it stops working at all. Raw logs are expensive to store, so retention is short. The moment the window you care about falls off the back of retention, the panel goes blank. The data that would have told you "this started drifting six weeks ago" is gone.

A scheduled view fixes both: write the query once, let it run continuously in the background, and query the small result instead of the enormous input.

What it looks like

A scheduled view is a FuseQL query with a name and a schedule:

Register that as nginx_errors_5m, and it evaluates on an interval (60s by default), each run covering only the sliver of time since the last one. Results land in the view's own store, and you query them back by name like any other source:

That second query reads pre-aggregated rows: thousands, not billions. Same chart. A fraction of the work.

Each run reads one window of raw logs and appends a handful of pre-aggregated rows.

What you actually get

Dashboards that stay fast as you grow. A 30-day chart over raw logs is a 30-day scan, and it gets slower every month you ingest more. Over a view that's already bucketed at 5 minutes, the cost barely moves. The expensive work still happens. It just happens once, incrementally, a minute at a time, instead of on every page load by every engineer.

Long-term history without long-term storage. This is the real unlock. Raw logs are costly to keep, so you keep them for days. Aggregates are tiny, so you can keep them for as long as you like. A view carries its own retention, independent of the logs it was built from. Drop raw logs on an aggressive schedule and still answer "how has p99 latency trended this year?" Capacity planning, SLO reporting, and year-over-year comparisons stop being a storage-budget conversation.

Load you can plan for. Ad-hoc heavy queries arrive in bursts. Someone opens a dashboard, an alert fires, three engineers dig into the same incident from three angles. Scheduled views convert that spiky, user-triggered work into a steady background trickle the platform can schedule on its own terms. Your worst-case query load stops being determined by how many people are panicking at once.

How it works, briefly

Each view runs on its own schedule, and every run covers one slice of time:

  1. Work out the window to process, everything since the last run, up to a few minutes short of now.

  2. Run the view's query over just that window.

  3. Append the results to the view's own store.

  4. Remember where it stopped, so the next run picks up exactly there.

No window is processed twice, and none is skipped. One detail is worth naming, because it's where this kind of system usually goes wrong:

Catch-up. A view created with a start time in the past, or one resumed after a pause, is behind. When a view falls far enough behind it shifts into catch-up mode, chewing through much larger windows per run until it's current, while still leaving room for everything else to keep running. That's what makes backfill and pause/resume work rather than leaving permanent holes in your history.

Views can be paused, resumed where they left off, stopped, or deleted, and you can create one with a start time in the past to backfill history you already have.

When to reach for one

Use a scheduled view when a query is known in advance, asked repeatedly, and expensive: dashboard panels, alert conditions, weekly reports, executive rollups, or anything you'd otherwise re-run against a month of raw logs.

Skip it when the query is exploratory. Views are pre-computed along a specific set of dimensions, so you can't slice by a field the view never grouped by. Chasing down a novel failure means going back to the raw logs, and that's exactly what they're for.

The rule of thumb: if you know the question ahead of time, materialize the answer. If you're still figuring out the question, query the logs.

Scheduled views are part of Kloudfuse Logs. If you want to see one running against your own log volume, talk to our team.

Observe. Analyze. Automate.

logo for kloudfuse

Observe. Analyze. Automate.

logo for kloudfuse

Observe. Analyze. Automate.

logo for kloudfuse