Itamar Syn-Hershko
ZSTD codecs, segment replication, star-tree indexes, derived source and derived fields: five OpenSearch-only features that cut storage and latency with a one-line change.
We at []BigData Boutique](https://bigdataboutique.com) review a lot of OpenSearch clusters. Most of them run with settings that made sense on Elasticsearch 7, because that's where the runbooks, the blog posts and the muscle memory all come from. Meanwhile, OpenSearch has quietly grown a set of features that don't exist in Elasticsearch at all, and almost nobody turns them on.
That's a shame, because none of these are architecture projects. Every tweak below is a one-line index setting or a small mapping change, and each one buys you something real: less storage, faster aggregations, or more indexing throughput on the same hardware. Engineers with a decade of Elasticsearch experience miss these all the time, precisely because they are OpenSearch-only.
A quick note on versions. Everything here is verified against the docs as of OpenSearch 3.x, and we note the minimum version for each tweak, since plenty of production clusters (especially on Amazon OpenSearch Service) still run 2.x.
1. Switch the index codec to ZSTD
OpenSearch 2.9 added two Zstandard-based codecs for stored fields: zstd and zstd_no_dict. Everyone knows about best_compression. Almost nobody uses these, and for most workloads they win outright.
The interesting part is that you don't pay the usual CPU tax. In the project's own benchmarks, best_compression shaves 34% off storage but costs you 2% write throughput. zstd gets you a 35% better compression ratio and 7% better write throughput than the default codec. zstd_no_dict trades a little ratio (30%) for even more speed (14% better throughput). Free disk space that comes with faster indexing is a rare deal in this business.
PUT /logs-000042
{
"settings": {
"index.codec": "zstd_no_dict",
"index.codec.compression_level": 3
}
}
The compression_level knob accepts 1 to 6; higher means smaller segments and slower compression. The codec is a static setting, so put it in your index template and it applies from the next rollover onward. No reindexing needed for new data.
Two caveats. The ZSTD codecs can't be used on k-NN or Security Analytics indexes. And as always with compression claims, run your own numbers, since the gains depend on how compressible your documents are. For log and metrics workloads we've seen results in line with the published benchmarks. If you're chasing storage costs more broadly, this pairs well with the rest of our OpenSearch cost optimization checklist.
2. Turn on segment replication
This one is our favorite conversation starter in cluster reviews. By default, OpenSearch replicates the way Elasticsearch always has: every document is sent to each replica and indexed there again, from scratch. If you have one replica, you're paying the full analysis-and-indexing CPU bill twice.
Segment replication changes that, and is extremely efficient for clusters with high ingestion rates. The primary indexes the document once, then ships the resulting segment files to the replicas, which just copy them. In the initial benchmarks, that meant roughly 40% higher indexing throughput on identical hardware, and up to about 60% at the median on larger workloads. For ingest-heavy use cases (logs, metrics, traces) this is the closest thing to free capacity you'll find.
PUT /logs-000042
{
"settings": {
"index.replication.type": "SEGMENT"
}
}
Or make it the cluster default in opensearch.yml:
cluster.indices.replication.strategy: 'SEGMENT'
The trade-off is replica freshness. Replicas now lag the primary by however long segment copying takes, and refresh=wait_for isn't supported. GET, multi-GET and term vector requests are routed to primaries to stay consistent, so a very read-heavy cluster concentrates more load there. The benefit also shrinks as replica count grows; past a handful of replicas the segment copying itself becomes the bottleneck. For the typical one-or-two-replica logging cluster, none of this matters and the throughput win is real. If your cluster uses remote-backed storage, replicas can pull segments straight from the remote store, which takes the primary out of the copy path entirely.
3. Star-tree indexes: pre-computed aggregations
If your OpenSearch cluster mostly exists to power dashboards (and let's be honest, a lot of them do), you're running the same terms and date_histogram aggregations thousands of times a day, and each run scans the same documents again.
A star-tree index pre-computes those aggregations at segment flush time and stores them in a tree keyed by your dimension combinations. Queries that fit the tree read the pre-aggregated values instead of scanning documents. The project reports up to a 100x reduction in query work, and it's now generally available in the 3.x line (with multi-terms aggregation support added in 3.3). Amazon OpenSearch Service picked it up in late 2025.
PUT /http-logs
{
"settings": {
"index.composite_index": true,
"index.append_only.enabled": true
},
"mappings": {
"composite": {
"request_aggs": {
"type": "star_tree",
"config": {
"ordered_dimensions": [
{ "name": "status" },
{ "name": "port" }
],
"metrics": [
{ "name": "size", "stats": ["sum"] },
{ "name": "latency", "stats": ["avg"] }
]
}
}
}
}
}
You pick the dimensions you group by and the metrics you compute, and matching aggregations (sum, min, max, avg, value count, plus date histogram, terms, range and multi-terms buckets) get answered from the tree without any query changes.
The constraints tell you where it fits. Star-tree indexes require append-only data, so no updates or deletes. Array values aren't supported, and you shouldn't use high-cardinality fields like _id as dimensions. In other words: perfect for observability data, wrong for a product catalog. If you want a refresher on how aggregations work under the hood first, we have a whole guide on that.
4. Derived source: stop storing _source twice
Here's an uncomfortable fact about how these engines store data: most of your field values exist on disk at least twice. Once in the _source JSON blob, and again in doc values or stored fields. You've been paying for that duplication in every cluster you've ever run.
OpenSearch 3.2 introduced derived source. The engine stops storing the _source blob and instead reconstructs documents on demand from doc values and stored fields. Search, updates and reindexing keep working; the JSON you get back is rebuilt rather than read.
PUT /logs-000042
{
"settings": {
"index": {
"derived_source": {
"enabled": true
}
}
}
}
The benchmark numbers are hard to ignore: 41-58% storage reduction across the tested workloads, up to 18% better indexing throughput, and merge times down 20-48%, since smaller segments merge faster. Combined with the ZSTD codec from tweak #1, this is how you cut a storage bill roughly in half without touching your data.
It's not for every index, though. Document reconstruction isn't free, so fetch-heavy workloads can see noticeable retrieval latency regressions. Reconstruction also normalizes some things: dates come back in the first format of your mapping, keyword arrays come back sorted and deduplicated, and geopoints can lose some precision. If your application treats _source as a byte-for-byte copy of what it ingested, test carefully. The setting must also be applied at index creation; you can't flip it on an existing index. For append-only logs where nobody fetches raw documents at high QPS, it's close to a pure win.
5. Derived fields: new fields without reindexing
Not to be confused with derived source (the naming could be better), derived fields landed in OpenSearch 2.15 and solve a different, very familiar problem: you need to query a field you never indexed, and the data is already in the cluster.
A derived field is defined by a script that emits values computed from existing fields, either in the mapping or right inside a search request:
POST /logs-*/_search
{
"derived": {
"status_class": {
"type": "keyword",
"script": {
"source": "emit(doc['status'].value >= 500 ? '5xx' : 'other')"
}
}
},
"query": {
"term": { "status_class": "5xx" }
}
}
The field behaves like a real one: you can run term, match and range queries against it, highlight it, and (since 2.17) aggregate on it. The classic use case is retroactively extracting structure from raw log messages. The incident is happening now, and reindexing three months of logs is not a serious option. It's also a nice way to prototype a mapping change against production data before committing to it.
The obvious caveat: values are computed at query time, so this is slower than a proper indexed field, and scoring and sorting aren't supported. Filter down the search space with indexed fields first (the prefilter_field option helps here), and when a derived field earns a permanent place in your queries, promote it to a real field at the next rollover. Think of it as schema-on-read for the 5% of queries that need it, not a replacement for mappings.
Honorable mentions
Two more that didn't make the list. Concurrent segment search (searching a shard's segments in parallel instead of one at a time) is the tweak you no longer need to make: it's on by default since 3.0, including for k-NN. If you're still on 2.x, check search.concurrent_segment_search.mode; large-shard workloads often see a healthy latency drop. And search backpressure ships in monitor_only mode by default. It watches for runaway queries on nodes under duress but doesn't actually cancel anything until you switch it to enforced. A rejected query you can retry beats a node you have to reboot; we've written before about what expensive queries can do to a cluster.
The common thread
None of this is exotic. It's all documented, benchmarked and shipped. It just isn't the default yet, and it isn't in the Elasticsearch playbooks and blogs most teams learned from. That gap between "shipped" and "actually turned on" is where most of the easy wins in our cluster reviews come from. An automated (and free) cluster review offering is also available through NeverBlink AI.
