ELK alternative for Kubernetes: what to take out of Elasticsearch and what to keep

Start at the end: for searching text across terabytes of logs, Elasticsearch does something Moonin does not. If you came to be told you can turn it off, this is not that page. What you can do is stop using it for things that do not need a search engine.

Free to download is not free to operate

Elasticsearch, Logstash and Kibana install without a licence fee, and the basic tier capabilities are free. That is a real merit: there is no monthly invoice growing with the cluster.

The cost shows up in three separate places and none of them appears in a contract. The first is infrastructure: an Elasticsearch cluster ingesting Kubernetes logs needs data nodes with fast disks, dedicated master nodes so the cluster does not split in two, and enough memory that the JVM is not spending its day collecting garbage.

The second is storage, and it is the most underestimated. When you index a document, Elasticsearch stores the document plus the structures that make it searchable. How much that totals depends on the mapping: a field you never search still costs disk if it was left indexed, and Kubernetes logs have many such fields — labels, annotations, identifiers nobody queries.

The third is recurring human work, which is the one you do not see until it is missing.

The cost that appears on no invoice

Running Elasticsearch in production comes with a list of tasks that return every month. Index lifecycle has to be managed so older data moves to slower storage or gets deleted, and that policy has to be decided and maintained.

Shards need watching: how many, what size, and rebalancing when they end up uneven. A shard that is too large makes queries slow; too many small shards consume cluster memory in metadata.

Version upgrades have to happen, which on a cluster holding production data is not a button.

And someone has to answer when the cluster goes yellow at two in the morning, which requires a team member who genuinely understands how it works inside. That knowledge is scarce and expensive, and when that person leaves, it leaves with them.

The managed option, and where its surprise lives

Elastic Cloud solves the operational work in exchange for an invoice. Its pricing is resource-based: you pay for the infrastructure you choose — compute, storage, transfer — plus a subscription tier (Standard, Gold, Platinum or Enterprise) that determines which capabilities you can reach.

That second axis is where surprises tend to appear, according to third-party analyses: the capability you need turns out to sit in a tier above the one you contracted, and the jump is not proportional to usage.

We do not publish concrete Elastic Cloud figures because we could not verify them with confidence and they vary by region, cloud and configuration. Their calculator is the right place for that maths. We prefer to state the gap rather than invent a number that will expire.

The licence moved three times, and that is a cost too

Elasticsearch was Apache 2.0 until 2021, when it changed to SSPL and the Elastic License. That change led AWS to fork the project and create OpenSearch. In September 2024 Elastic added AGPLv3 as an option alongside the other two.

Today you can choose which licence to use it under, and that is better than before. But for anyone planning several years out, a component whose licensing changed three times in five is a risk worth having on the register, especially for commercial use or when packaging it inside a product.

Worth saying without drama: for most teams that only operate it internally, none of the three licences is an obstacle. The risk is one of planning, not of compliance.

Where ELK clearly wins, and Moonin does not compete

Searching an arbitrary text string across terabytes of logs, with aggregations over the result, is exactly what Elasticsearch does well and what it was designed for. No focused tool replaces it.

If your incident investigation consists of finding the exact error message among millions of lines, or correlating fields you did not know you would need, that is a search-engine capability and there is no shortcut.

Kibana as a place to build arbitrary dashboards over your own data has no equivalent here either. Moonin is not a dashboard engine.

So this article does not propose turning ELK off. It proposes something smaller and more real.

What you can take out of Elasticsearch

Many people store data in Elasticsearch that does not need text search: what was deployed, when, with which image and commit, how many replicas it had, what changed between one revision and the next. It is there because there was nowhere else to put it, not because a search engine is the right tool.

That data is structured and small in volume. Moonin keeps it natively: the resource inventory, the revision history per service, DORA metrics computed from that history. Taking it out of the indices reduces volume without losing capability.

The second thing you can remove is the reason many teams index more than they query. When the only way to know what changed is to search the logs, everything gets kept just in case. With a separate deployment history that pressure drops, and retention can then be trimmed on judgement rather than on budget.

And service-to-service traces arrive via eBPF in the node kernel, without instrumenting code or passing through Logstash. The agent needs node privileges to read kernel traffic: the trade-off for not touching your applications, and it is documented.

How to do it without breaking anything

Install the Moonin chart on one cluster and leave it running in parallel for two or three weeks without touching your log pipeline. Access to the Kubernetes API is read-only and does not interfere with Filebeat, Fluent Bit or whichever agent you use.

Then review your indices and separate two groups: the fields someone actually queries, and the ones indexed by default. That second group is a candidate to stop indexing, which lowers storage without losing anything that was being used.

And review how many of your frequent Kibana queries are really the question "what changed just before this". If it is many, those no longer need the search engine.

What you should not do is touch retention before the deployment history is working. That order is what avoids losing visibility exactly when something breaks.

See whether it applies to you

One question settles it: of your most frequent Kibana queries, how many are really "what changed just before this". If several are, those did not need a search engine.

About usPricingSecurity and permissionsPartner programmevs Datadogvs Prometheus & Grafanavs New Relicvs Dynatracevs ELK and ElasticsearchCrashLoopBackOffOOMKilledImagePullBackOffPod stuck in PendingDocumentationDemoSign upContact usArguz, the consultancy

Frequently asked questions

Does Moonin replace Elasticsearch?

No, and it does not try to. For searching arbitrary text across terabytes of logs with aggregations, Elasticsearch does something Moonin does not. What Moonin replaces is the use of Elasticsearch as a record of what was deployed and when, which is structured, low-volume data that does not need a search engine.

If the software is free, where does the cost come from?

From three places that appear in no contract. Infrastructure: data nodes with fast disks, dedicated masters and memory for the JVM. Storage: indexing stores the document plus the structures that make it searchable, and a field nobody queries still costs disk if it was left indexed. And human work: index lifecycle, shard sizing and rebalancing, version upgrades, and answering when the cluster goes yellow overnight.

Which licence is Elasticsearch under today?

You can choose among three: the Elastic License, SSPL, and AGPLv3, which Elastic added as an option in September 2024. Before 2021 it was Apache 2.0, and that change is what led AWS to create OpenSearch. For internal use none of the three is usually an obstacle; the risk is one of planning, since this is a component whose licensing moved three times in five years.

How much does Elastic Cloud cost?

We do not publish a figure because we could not verify one with confidence: pricing is resource-based and varies by region, cloud and configuration, plus a subscription tier that determines capability access. That second axis is where invoice surprises tend to appear. Their own calculator is the right place for that maths.

Can I run Moonin alongside my current log stack?

Yes, and that is recommended. Moonin does not touch your log pipeline: it does not replace Filebeat or Fluent Bit, does not pass through Logstash and does not write to your indices. It installs as a separate Helm chart and its Kubernetes API access is read-only.