BigData Boutique's Hebrew Search Analyzer, a dictionary-based morphological analyzer, now installs on managed Amazon OpenSearch Service domains as a custom package. Why Hebrew needs it, how installation differs from a self-managed cluster, and why it matters for hybrid search.

Our Hebrew Search Analyzer Now Runs on Amazon OpenSearch Service

Our Hebrew Search Analyzer now installs directly on managed Amazon OpenSearch Service domains as a custom plugin package. Until now, if you were running Hebrew search on Amazon's managed service you had two bad options: ship without proper morphological analysis, or move to a self-managed cluster just so you could get shell access and install a plugin. We've had that conversation with quite a few teams over the years, and it was never a fun one. It's no longer necessary.

Here's why the gap existed in the first place, and what changes now.

Why Hebrew breaks standard search analyzers

OpenSearch ships built-in language analyzers for more than 30 languages, from Arabic and Armenian through Thai and Ukrainian. Hebrew isn't one of them, and neither Elasticsearch nor Lucene have one either. That's not an oversight. It's a consequence of how the language works.

Hebrew words are built by slotting a three- or four-letter root into a pattern (mishkal) and then attaching prefixes and suffixes for gender, number, tense, and possession - all without spaces. A single string like וכשלרהיטים is a stack of prepositions and conjunctions (and-when-to-the-furniture) glued onto a noun. On top of that, written Hebrew usually omits vowel marks (niqqud) entirely, so the same consonant sequence can be several unrelated words depending on context. The stemming algorithms that work fine for English, French, or German - strip a suffix, maybe apply a rule table - simply fall apart here. You need a dictionary and a grammar, not a suffix list.

That's what a morphological analyzer is: software that maps an inflected word form back to its dictionary lemma using the language's root-and-pattern grammar and a reference dictionary, rather than by chopping off suffixes.

Get this wrong and search quality degrades in a very specific way. Users type the natural inflected form of a word (which is nearly always what they type), the index holds a different inflected form of the same word, and a query that should obviously match returns nothing. No amount of query tuning fixes that - the tokens have to be normalized to the same lemma at both index time and query time.

Why this matters even more in the hybrid search era

Hybrid search runs a lexical retriever and a dense vector retriever in parallel and fuses the two ranked lists. The idea is that BM25 covers exact matches and identifiers while embeddings cover paraphrase and semantic recall. That idea quietly breaks when the lexical retriever can't normalize inflected forms.

Without a morphological analyzer, BM25 over Hebrew text is basically literal string matching: a document containing מלפפונים won't match a query for מלפפון, even though any Hebrew speaker reads them as the same word. In a hybrid pipeline that starves the lexical leg of the fusion. RRF or a weighted blend has nothing useful to combine with the dense retriever's ranking, because the keyword signal is noise on anything but exact string equality. What you end up with is vector search with an unreliable tiebreaker bolted on, not real hybrid retrieval.

So morphological correctness matters more today than it did in the BM25-only days, not less. When lexical search stood alone, a weak Hebrew analyzer was a recall problem you could partly paper over with query expansion or fuzzy matching. In a hybrid pipeline, a broken lexical leg actively works against the fusion, because RRF and weighted blends assume both retrievers are contributing signal on their own. Run the same field through our hebrew analyzer at index time and hebrew_query or hebrew_query_light at query time, and BM25 starts doing what it's supposed to: inflected forms, construct-state variants, and prefixed forms of the same lemma all match. That's what gives you a lexical signal that's actually complementary to the dense retriever, which is the whole point of hybrid search. For Hebrew product catalogs, legal archives, or support content, this is usually the difference between hybrid search that measurably beats the BM25-only and dense-only baselines, and one that doesn't.

About the analyzer

The Hebrew Search Analyzer is our commercial product: a proprietary, actively maintained Hebrew dictionary paired with the analyzer engine, and direct support from us - the people who built it. It's the production-hardened descendant of over a decade of Hebrew information retrieval work, going back to the open-source HebMorph project I started in 2010 to bring proper Hebrew retrieval to Lucene, Solr, and eventually Elasticsearch.

It is licensed software, so plan for that from the start. A valid license token is required on every platform it runs on, including Amazon OpenSearch Service, and none of the installation steps below do anything useful without one.

Hebrew Search Analyzer
Dictionary Proprietary, actively maintained by us
License Commercial, required on every platform
Platforms Lucene, Lucene.NET, Solr, Elasticsearch, OpenSearch, SQL Server FTS
Amazon OpenSearch Service Supported as a custom package
Support Directly from our team

The plugin exposes a few analyzers: hebrew for indexing and full morphological querying, hebrew_query and hebrew_query_light for lighter-weight query-side expansion, and hebrew_exact for when a field needs literal token matching without lemma expansion - handy for names, acronyms, and product codes where morphological normalization would hurt more than help.

Installing it on a managed Amazon OpenSearch Service domain

A managed domain gives you no shell, no opensearch.yml, and no way to call custom REST endpoints, so the installation path looks quite different from a self-managed cluster even though the analyzer behaves identically once it's in.

Self-managed OpenSearch Amazon OpenSearch Service
Install bin/opensearch-plugin install <url> Upload to S3 → create-package → associate-package
License PUT _hebrew/license Document in the hebmorph_license index
Dictionary overrides PUT _hebrew/additions or custom.txt Same document, additions field
_hebrew/* diagnostic APIs Available Not available
Applying a new build Restart the node Blue/green deployment

Before you can attach a custom package at all, the domain needs node-to-node encryption, encryption at rest, and enforced HTTPS with the Policy-Min-TLS-1-2-PFS-2023-10 TLS policy. Custom plugins also require OpenSearch 2.15 or later and are only available in some AWS regions, so check Amazon's custom plugins guide before you start.

The plugin is built against one exact OpenSearch version. Amazon only offers MAJOR.MINOR.0 releases, so the build has to match precisely - tell us which version your domain runs and we'll send you the right zip. From there it's three AWS CLI calls:

# 1. upload the plugin to S3, in the same region as the domain
  aws s3 cp analysis-hebrew-3.1.0-commercial.zip s3://$BUCKET/analysis-hebrew.zip
  
  # 2. register it as a package (validation takes 10-25 minutes)
  aws opensearch create-package \
    --package-name analysis-hebrew \
    --package-type ZIP-PLUGIN \
    --engine-version OpenSearch_3.1 \
    --package-source "S3BucketName=$BUCKET,S3Key=analysis-hebrew.zip" \
    --region $REGION
  
  # 3. attach it to the domain via blue/green deployment (20-40 minutes)
  aws opensearch associate-package --domain-name $DOMAIN --package-id $PKG_ID --region $REGION

Since there's no _hebrew/license endpoint on a managed domain, licensing goes through an index instead. The license token is a document in hebmorph_license, which the plugin polls every 30 seconds. A nice side effect: the license and any dictionary overrides survive blue/green deployments and node replacements on their own, because they live in the cluster's data rather than on a node's filesystem.

curl -X PUT "$ENDPOINT/hebmorph_license/_doc/default?refresh=true" \
    -H 'Content-Type: application/json' \
    -d '{"license": "<the license token we sent you>"}'

To confirm it took, analyze an inflected word and check that you get a lemma back, not just the original token:

curl -X POST "$ENDPOINT/_analyze" -H 'Content-Type: application/json' \
    -d '{"analyzer": "hebrew", "text": "מלפפונים"}'
  # tokens: מלפפונים$, מלפפון  <- the second token is the lemma

Search interface mock-up showing a Hebrew search box with the term "שולחן" and result rows including inflected forms like "שולחנות" and "השולחן"

One more difference: the diagnostic _hebrew/* endpoints (check-word, status) don't exist on the managed service. Amazon only accepts plugins that extend analysis and search behavior, not ones that register new endpoints. So _analyze becomes your main tool for checking how a word gets tokenized, and CloudWatch logs (with ES_APPLICATION_LOGS enabled) take the place of the status endpoint when troubleshooting.

Key takeaways

  • Hebrew's root-and-pattern morphology and vowel-free spelling defeat suffix-stripping stemmers, which is why neither OpenSearch nor Elasticsearch ship a Hebrew analyzer despite supporting 30+ other languages.
  • In a hybrid search pipeline, a working Hebrew analyzer is what keeps the BM25 leg meaningful. Without it, the lexical side degrades to exact-string matching and hybrid search stops being hybrid.
  • Our Hebrew Search Analyzer is commercial software: a proprietary dictionary, support from the team that built it, and a license token required on every platform, including Amazon OpenSearch Service.
  • On Amazon OpenSearch Service, installation goes through create-package/associate-package instead of opensearch-plugin install, and the license lives in a hebmorph_license index document rather than behind a _hebrew/license API call.
  • Verify it's working by running _analyze on an inflected word and confirming you get a lemma token back, not just the original string.

If you're running Hebrew search on Amazon OpenSearch Service, or building hybrid retrieval over Hebrew content and want the lexical side to actually pull its weight, reach out. We built this analyzer and we'll get it running on your domain.