When their data center held them back too much, we jumped in to architect Buzzilla's next generation, cloud-native platform. Read about Buzzilla's journey to Kubernetes on Google Cloud, and how we helped them scale their Elasticsearch usage to use multi-lingual full-text search, monitoring and alerting.

Buzzilla is a market leading social media monitoring and analysis company, headquartered in Tel Aviv, Israel. Founded in 2008, they help many organizations understand and leverage the conversations that are being had about them on the web, at Petabyte scale.

While working with Buzzilla, BigData Boutique seamlessly moved their infrastructure to the cloud, increased their monitoring and scaling capabilities, brought to them a wealth of Elasticsearch expertise and grew their Hebrew text search and entity extraction functionality. This resulted in a more flexible, scalable, and stable system which, in turn, allowed the business to pursue new opportunities, relying on modern technology and cutting edge practices.

Old On Premise Infrastructure

The existing on-prem solution was resource intensive, with Buzzilla employees frequently needing to travel to the datacenter to arrange and configure servers. These servers were outdated and, with the rising operational burden, were preventing Buzzilla from scaling out to meet demand or develop new functionalities.

No Down Time

Buzzilla’s need for a migration to a modernized cloud-based tech-stack was matched by their need to have no disruption or downtime during said migration. The impact of downtime would have been substantial for Buzzilla, as their web crawling technology updates their clients in real time and service is fast becoming a key differentiator in how consumers select their suppliers. Buzzilla’s developers often refer to the equally demanding internal and external customers: service downtime has a knock-on effect to everyone in the business, from customer success to product management.

Observability across a distributed system

Buzzilla runs and aggregates data from numerous disparate sources, using a distributed system assembled of many services. Logging and monitoring these services is of paramount importance. Buzzilla needed a single platform to monitor and maintain all of these different services. System observability was the key goal. Buzzilla wanted to be able to rapidly act on outages and be proactive and act on early warning signs to even prevent them from happening in the first place.

Search and text analytics over Hebrew texts

Buzzilla’s demanding work is made more difficult by the innate complexities of the Hebrew language and being able to accurately, automatically and intelligently crawl myriad sites for customer-relevant data, with entity extraction at the heart of what they do as a company, is of paramount importance. This, says Lior, is what made selecting BigData Boutique such an obvious decision - “[BigData Boutique] are known experts with Elasticsearch and Hebrew morphology… which are the core technical challenges for Buzzilla”.

From the outset, BigData Boutique brought expertise in areas unknown to Buzzilla, providing an effective and critical consultancy offering. During the selection process, BigData Boutique were always able to answer questions quickly, demonstrating not only their knowledge of Elasticsearch and cloud migration projects, but also their eligibility as an in-house partner.

This, Lior emphasises, made BigData Boutique’s offering superior to other tech firms. There was no doubt in the minds of Buzzilla’s lead developer that BigData Boutique was the right choice.

The Seamless Move to the Cloud

BigData Boutique were able to deliver a seamless migration to Buzzilla’s new cloud-based infrastructure, hosted on Google Cloud Platform (GCP) and powered by Kubernetes, with zero downtime or disruption experienced. This, Lior was quick to state, was huge for Buzzilla:

“Through a side-by-side deployment with both the data centre and GCP and centralising the logging from the data centre and GCP, Big Data Boutique were able to rapidly prove that the GCP infrastructure was outperforming the on-prem infrastructure, throughout the migration… it was really impressive”.

“I can tell you that, in hindsight, having worked at previous companies with on-premise hosting, the migration process is no easy challenge”, states Lior. BigData Boutique were able to provide this service seamlessly, with zero downtime during the migration.

Modernizing Buzzilla’s Codebase

BigData Boutique addressed some issues with Buzzilla’s code prior to the migration - “we weren’t great at writing code in an organised and forward looking way” notes one Buzzilla employee. Containerization is a vital part of a successful migration of applications to the cloud and often involves decoupling monolithic applications, so they are more fault tolerant during and post-migration.

This modernization of the code exposed many hidden issues within the codebase, some of them existing for years . Such issues were sometimes severe enough to prevent containerization of a specific service, thus making it not cloud-ready. BigData Boutique were able to resolve and rewrite many of these issues, with the new, more maintainable, more visible codebase.

Sharing the knowledge

With paired programming sessions and informed instruction, BigData Boutique were able to get Buzzilla’s developers up to speed on the cutting edge stack, allowing them to be totally self-sufficient when the engagement finished.

BigData Boutique successfully imparted knowledge and expertise, empowering Buzzilla’s developers beyond the scope of their engagement. This piece of work gave Buzzilla a modernised, cloud native application architecture, which is scalable and future-proof.

Enabling search and entity extraction for Hebrew texts

Eyfo is an off-the-shelf set of tools developed by BigData Boutique to solve some of the biggest linguistic and semantic challenges faced by organisations dealing with search and entity extraction. “We integrated Eyfo to enhance our text search and analytics capabilities, and were able to benefit from BigData Boutique’s expertise in entity extraction through this product” Deborah states.

Entity extraction, relating specifically to the Hebrew language, is exceptionally difficult, says Lior:

“When talking about entity extraction, there’s a lot of challenges with [Hebrew] itself: double meanings, morphology, creating specific challenges not found in other languages. When talking about selecting Eyfo as a solution and the associated evaluation process, a developer said “there are not a lot of products on the market that can do it, and there is one that does it very well”.

Observability, monitoring and alerting

We have been able to integrate full end-to-end monitoring, visualizations and health checks in a holistic view to minimize any wider service impacts from third-party downtime. Simple but multi-touchpoint monitoring is of paramount importance for Buzzilla - “we’re now able to collect data and centralize logging from a number of datasources running in the Buzzilla ecosystem”.

A production ready, cost-effective Elasticsearch Cluster

Since Buzzilla’s system is data rich and Elasticsearch is used for many of their search and analytics features over that data, we had to make sure it’s deployed correctly and is as optimized as it can be. We applied our years-long experience for making the most out of the hardware used.

The Elasticsearch cluster topology and its configurations were improved, resulting in performance optimizations, increased resilience, scalability and fewer vulnerabilities. Finally, the Buzzilla codebase was improved to interact more fluidly with the Elasticsearch APIs, incentivizing the engineers to more readily engage with the Elasticsearch cluster and maximise the value gained from this key piece of infrastructure.

99.999% Uptime

A brand new tech stack with state of the art technologies brought huge benefits for Buzzilla as an organization, as well as to its individual employees. For Lior, the cloud-based infrastructure has opened a raft of opportunities and dependability light years ahead of what was previously possible:

“Ever since we moved to the cloud with BigData Boutique, we have had 99.999% (five 9s) uptime, we have a much better monitoring and alerting system and observability of our infrastructure and systems - which it’s really something that was impossible before Big Data Boutique got involved”.

Simplified Operational Challenge

As a developer, Lior also lauds the elasticity and scalability of the solutions that were deployed. When comparing the previously onerous task of leasing infrastructure just to run a proof of concept with the new simplicity of cloud computing - “now we’re just scaling up a deployment on Kubernetes, which is basically the click of a button”. The introduction of Terraform to manage the cloud infrastructure, and CI/CD pipelines to rapidly release changes, was revolutionary to Buzzilla. It enabled improved engineering practices and a quicker turnaround time for features.

Better Customer Experience

At the heart of what Buzzilla does is their customer base and the service that they provide. “Guaranteeing our customers virtually no downtime due to our new technologies - that’s massive”. Lower latency means faster response times, meaning Buzzilla’s customers reap the benefits of their new cloud-native technology ecosystem.

A New Outlook

For Deborah, as a product manager, outside of the overhaul of the tech stack and no longer having to labour away on on-prem infrastructure, one of the biggest changes is the culture and outlook within the development team.

Before BigData Boutique came in, the development team had to deal with old and unruly code that hadn’t necessarily been developed with scalability and future expansion in mind. ”BigData Boutique brought order to the messy code we previously had.”. Deborah states “for our tech team, as a customer of Big Data Boutique, the difference is really significant… these changes facilitate us doing our work in a different and better way”.

Making significant organizational changes above and beyond the implementation of tools like Shob.io and Elasticsearch is what made Buzzilla’s engagement with BigData Boutique such a resounding success, guaranteeing sustainable and self-perpetuating improvements for the future.

Lior wraps up by saying:

“...things are now much more efficient, workloads are divided better and we don’t have downtime - that was [thanks to] Itamar and BigData Boutique”.