Zevi Reinitz
How EverC worked with BigData Boutique to cut data platform costs on AWS by migrating Spark workloads to EMR on EKS — without disrupting analyst workflows or operational flexibility.
When a data platform's cost curve starts outpacing the value it delivers, the right answer is rarely "use less Spark." It's usually a more honest question: are we paying for capabilities we actually use, on infrastructure shaped to our actual workloads? That's the question EverC — now part of G2 Risk Solutions — set out to answer when its Spark-based data platform on AWS started costing more than the business could justify.
Working with BigData Boutique, EverC redesigned its data processing platform around Amazon EMR on EKS. The result: a more cost-efficient, AWS-native architecture that preserved familiar analyst workflows while restoring predictability and control over compute economics.
The Setup: Spark at the Center, Costs Climbing
EverC is a pioneer in AI-driven risk intelligence, providing the technology that helps banks, payment providers, and marketplaces manage merchant risk across the lifecycle. That mission depends on processing large volumes of data — both batch ETL and interactive analysis — and Apache Spark sits at the center of how the company's engineering and analytics teams get that work done.
As workloads grew, so did infrastructure spend. The pricing premium of EverC's existing Spark platform became increasingly hard to justify, particularly for jobs that didn't need its higher-level abstractions. This wasn't a crisis — the system worked — it was a strategic call. EverC wanted to keep what was working for its teams and change what no longer made economic sense.
The constraints made the problem interesting:
- Continue supporting both interactive analyst workflows and ETL workloads
- Keep a familiar, notebook-driven experience for data teams
- Stay aligned with AWS-native infrastructure and security controls
- Improve cost predictability and reduce platform overhead
In other words: cut cost without forcing a workflow migration, and stay inside the AWS-native lane EverC's security and compliance posture already lived in.
Picking the Right EMR Flavor
BigData Boutique partnered with EverC to evaluate alternatives based on how Spark was actually used in production — not on a generic recommendation. Three AWS-native options were on the table:
- EMR Serverless, for simplified operations and bursty workloads
- EMR on EC2, for traditional YARN-based execution
- EMR on EKS, for deeper integration with Kubernetes-based infrastructure
EverC already had meaningful investment in Kubernetes and wanted to reuse compute capacity rather than carve out a dedicated Spark estate. That made EMR on EKS the natural fit.
Choosing EMR on EKS let EverC:
- Reuse existing Kubernetes clusters instead of provisioning dedicated Spark infrastructure
- Take advantage of fine-grained scheduling and scaling
- Improve utilization of spot instances and reduce idle compute
- Eliminate the platform premium associated with managed Spark services
The deeper benefit is structural: EverC now has direct control over how and when compute is consumed, without forcing a radical change in how its teams work.
Keeping the Notebook Experience Intact
A platform migration that breaks the analyst workflow is a platform migration that gets rolled back. The team made preserving the notebook experience a hard requirement. Bringing in EMR Studio kept the notebook-driven interface analysts were used to, so the platform shift happened underneath them rather than to them.
The Outcome
The redesigned platform delivered improvements across several dimensions:
- Reduced platform costs by eliminating unnecessary service premiums
- Improved cost predictability, with clearer visibility into how workloads consume compute
- Operational flexibility, allowing different workload types to coexist on shared infrastructure
- Continuity for data teams, who maintained their established workflows
Beyond the migration itself, EverC now has a foundation that supports ongoing optimization — scaling resources up or down as needs change, without being locked into a single execution model.
Why This Pattern Works
A few takeaways for teams considering a similar move:
- Kubernetes you already run is a Spark platform you already pay for. If your organization has invested in EKS, the marginal cost of running Spark on it is small compared to the platform premium of a managed Spark service. EMR on EKS turns existing capacity into Spark capacity.
- Workload shape should drive the deployment model. EMR Serverless, EMR on EC2, and EMR on EKS are not interchangeable; the right choice falls out of the actual mix of batch jobs, interactive sessions, and existing infrastructure investments — not out of the marketing comparison.
- Cost wins fall apart if analysts revolt. Anchor the migration around preserving the workflow surface (notebooks, scheduling, access patterns) and change the compute layer underneath. EMR Studio is a useful piece of that puzzle.
Read the Full Case Study
The full story — including more on the evaluation, why EMR on EKS won out, and what EverC has lined up next — is available in our customer story:
Read the full EverC case study →
If your data platform costs are growing faster than the value they deliver, and you'd like a second opinion on whether you're paying for capabilities you don't actually need, get in touch. We work with teams running large-scale Spark, EMR, and Kubernetes-based data platforms on AWS every day.
