Migration mechanics, cardinality, Terraform, and operations notes for teams moving off Datadog, Splunk, and similar contracts onto self-hosted stacks in their own AWS accounts.
September 7, 2026
Grafana Alloy's gateway mode looks straightforward on paper: run a small fleet of collectors in front of your Mimir, Loki, and Tempo backends, and stop punching thousands of individual agent connections directly at your storage tier. …
August 31, 2026
Grafana Alloy looks like a straightforward drop-in for your telemetry pipeline. Run it as a DaemonSet or a central gateway, point your apps at it, done. That story holds until you're pushing 2 million spans per …
August 24, 2026
Most teams discover they have a cardinality problem when Mimir starts rejecting series or their AWS bill for object storage doubles in a quarter. By that point they're already in triage mode: dropping labels, adding recording …
August 17, 2026
Metric cardinality is the silent budget killer in every self-hosted observability migration we have done. You move off Datadog, celebrate the cost savings, and then six months later your Mimir ingestor memory is climbing past 64 …
August 10, 2026
OpenTelemetry's new cardinality limits spec gives the Collector a formal way to drop high-cardinality label combinations before they reach your TSDB. That sounds like good news. It is — but only if you understand where in …
August 3, 2026
The OpenTelemetry Go compile-time instrumentation library hit v1 in late July 2026. For teams running self-hosted Grafana stacks on AWS, this changes the calculus on Go service instrumentation in ways that aren't obvious from the release …
July 27, 2026
The OpenTelemetry Go compile-time instrumentation library hit v1 this month. If you run a self-hosted observability stack on AWS — Grafana, Tempo, Mimir, Loki — this changes how you think about instrumenting Go services without touching …
July 20, 2026
The OpenTelemetry project shipped v1 of Go compile-time instrumentation this month. If you run Go services on AWS and you've been living with the manual span-annotation tax — or worse, skipping instrumentation entirely because the setup …
July 13, 2026
The OpenTelemetry project published a post this month with a blunt title: "Don't Wrap OpenTelemetry." We've been saying the same thing to clients for two years, usually after inheriting a codebase where a well-intentioned platform team …
July 6, 2026
The OpenTelemetry project published a blunt warning this month: stop wrapping the OTel SDK in your own abstraction layer. If you are mid-migration from Datadog or Splunk and you built a thin internal telemetry library on …
June 29, 2026
The OpenTelemetry Arrow Phase 2 release quietly shipped a capability that most teams running self-hosted collector fleets on AWS haven't accounted for in their sizing models: columnar encoding and pipeline-level compression that cuts wire bytes by …
June 22, 2026
The OpenTelemetry project shipped OTel-Arrow Phase 2 this month, and the headline — 'efficient telemetry pipelines' — undersells what actually changed. This is not just a transport-layer optimization anymore. Phase 2 restructures how the Collector batches, …
June 15, 2026
OpenTelemetry Arrow Phase 2 shipped last month and the project blog claims 70–90% wire-size reduction for telemetry data. That number is real — but it applies narrowly, and the path from 'efficient transport' to 'lower AWS …
June 8, 2026
Most observability migration guides assume you control the codebase. You don't always. Here's how we instrument legacy Java monoliths — some of them 15 years old, running on JDK 8, with no test coverage and a …
June 1, 2026
The OpenTelemetry GenAI semantic conventions reached stable status in early 2026, and if you are running LLM-backed features in production, you now have a vendor-neutral way to capture token counts, model latency, prompt content, and cost …
May 25, 2026
OpenTelemetry reached CNCF graduation in May 2026. That's a meaningful signal — the project is stable, the APIs are locked, and the ecosystem is mature enough that betting your observability pipeline on it is no longer …
May 18, 2026
Grafana's Adaptive Logs feature, which shipped to Grafana Cloud recently, uses a sampling model to automatically drop high-volume, low-signal log lines. The pitch is compelling: cut your log ingestion bill without writing a single drop rule …
May 11, 2026
Skyscanner's OpenTelemetry write-up landed last week and it's one of the more honest operational accounts of running OTel at scale that I've seen from a large engineering org. They're managing 24 production Kubernetes clusters with a …
May 4, 2026
Skyscanner published a detailed post this week on how they manage OpenTelemetry Collectors across 24 production Kubernetes clusters. The architecture decisions they made — and the ones they avoided — are a useful stress test for …
April 24, 2026
Buyers who type this query into Google want three things: a defensible architecture, retention they can explain to Finance, and a cost model that includes S3 and query path, not only ingest. We start from a …
April 24, 2026
This is an opinionated cost teardown, not a vendor hit piece. We name the mechanisms that inflate hosted observability bills: custom metrics, indexed logs, APM cardinality, and seat-adjacent pricing. Then we show what moves when you …
April 24, 2026
Tail sampling is where OpenTelemetry configs go to die: policy mistakes show up as missing traces in prod, not as a failing unit test. This is a working collector shape we use when buyers search for …
April 24, 2026
Numbers are anonymized; failure modes are not. The useful part of a vendor exit is rarely the architecture diagram alone. It is what broke in week six when cardinality, ownership, and on-call habits collided with the …
April 24, 2026
Between one and five terabytes of logs per day, every storage engine becomes a finance and operations problem, not a benchmark chart. Here is how we compare Grafana Loki, Elasticsearch, and ClickHouse after running all three …
April 24, 2026
How we size Grafana Mimir clusters for 10M active series on AWS EKS — replica counts, cost tradeoffs, and the failure modes to plan for.
July 15, 2025
Observability is not free, but the cash story is usually MTTR, fully loaded engineer hours in bridges, and AWS lines nobody charts until Finance asks. We separate real savings from license theater for teams leaving SaaS …
July 15, 2025
Monitoring is the coverage you wired in advance. Observability is whether you can answer the next question without shipping a new metric for every hypothesis. Here is how we use both when we move teams to …
January 14, 2025
Logs feel definitive; traces feel modern. Metrics still anchor SLOs, autoscaling, and cardinality discipline on Prometheus and Mimir. This is the case we make when a team wants to log everything 'just in case.'