A desire to revolutionize their internal knowledge platform and search functionality led Keller Williams, the leading global real estate company, to seek our help in implementing a new Elasticsearch solution. From data discovery through to exhaustive benchmarking and testing, we integrated ourselves with KW's tech team to deliver a fully future-proof search solution.

Keller Williams (KW) is a leading global real estate company with a focus on technology-driven growth. With more than 189,000 agents operating out of 1,000+ offices across the world, KW pride itself on equipping its staff and customers with all the proper knowledge to make informed property decisions, delivered through advanced technology solutions.

Not only does KW supply traditional real estate solutions, but they also have a dedicated technology arm, KW Technology, which services a suite of AI-driven tools designed to assist real estate agents with all aspects of the property sales cycle.

One of the backbones of KW's continued success is its global knowledge platform, called Connect. Connect provides sales enablement and training materials for all of KW's many agents, it gives agents the ability to collaborate and team up for complex real estate deals, it serves as a corporate directory, and it has a search function to help agents navigate the vast array of materials and documents that they might need to carry out their day-to-day roles.

“The search functionality is the portal to everything that agents might do in the Keller Williams universe,” says Gio, Director of Engineering and overall technical lead for Connect.

Gio and his team had huge ambitions for the Connect platform, but it was built on legacy search technology and they knew it needed modernization. Elasticsearch seemed like the natural choice for KW, as it provided the open architecture and scalability they needed. KW needed a highly available, multi-lingual, and AI-ready search platform to keep supporting their agents, and they needed someone to help them deliver it.

Given the specialized nature of search technology, KW knew that selecting the right consultancy partner for the task was crucial to its success.

They conducted extensive market research, looked at sector leaders in Gartner, and scanned online reviews for qualified partners. Through a mixture of research and recommendations, the architecture team at KW came across BigData Boutique.

“We drew up a long list, then a shortlist, and interviewed all the prospective partners, asking them increasingly detailed questions and presenting more challenging scenarios along the way”, says one of KW's technical architects. “Ultimately, BigData Boutique made us feel the most comfortable and gave us the most confidence that they would deliver this project exactly as expected”.

KW's technology team didn't have the requisite level of experience and knowledge to deliver the solution, maintain it, and continue to develop it as new features were released after it went to general availability.

“From reviewing our own architecture and internal proof of concepts, we knew we'd need a strong partner like BigData Boutique to help us adapt, configure, and launch the new platform”.

Given that the existing search platform wasn't based on Elasticsearch, it was vital for BigData Boutique to carry out a thorough discovery to ascertain how best to build the new solution and spot any challenges early in the journey.

The first thing to tackle was a deep dive into the existing legacy platform. “What did the current platform not have? What were its challenges? These were the key topics we needed to explore before going any further” states one of KW's technical team. This gap analysis was vital in ensuring the new platform met every requirement.

The next aspect for discovery was a vast data exploration and classification phase. The new search functionality would be handling huge amounts of unstructured data in a variety of formats and multiple languages. This phase was key in ensuring that the new Elasticsearch tooling was futureproofed and capable of handling the quantities and variety of data.

The final aspect of discovery was use-case defining. This pulled together the outputs of the two prior phases to create tangible metrics and requirements for the Elasticsearch configuration.

“We went through all of the data from the discovery phase, all of the parameters and requirements, which was multiple four-hour sessions, and we got to a point where we knew the exact Elasticsearch configuration needed now, and what we would need in the future”.

BigData Boutique was able to build a fully functioning proof of concept based on the thorough discovery phase. KW's technical team then deployed this brand new Elasticsearch cluster on their QA environment to better understand how it would perform.

Whilst the Elasticsearch cluster had been configured and built exactly to the specifications and requirements identified in the discovery phase, it was critical to carry out performance testing and load testing to identify performance bottlenecks and any irregularities.

“It was a very professional process,” recalls Gio, “BigData Boutique instructed us on run testing, benchmarking, and then hosted another session with us to look at the outputs in detail to identify performance enhancements and tweaks that could be made”.

The testing, reporting, and tuning ran for several cycles. Once the KW technical team and the BigData Boutique consultants were satisfied that the Elasticsearch configuration was perfect, it was implemented into production.

“After we put the tuned configuration into production, and loaded some sample data, we carried out the same performance and benchmarking as in QA… BigData Boutique had a very thorough approach”.

After the Elasticsearch configuration had been moved into production and had been tested to numerous acceptance criteria, it was time to get it up and running. First, all of the legacy data was copied, downloaded, and then the legacy repository was set to read-only.

All new traffic was then pointed at the new production Elasticsearch cluster, which was even more complex as it was deployed entirely on Google Cloud Platform. The KW agents who used the system experienced no downtime.

“I would say that it was an exceptionally clean cutover to production,” says Gio.

One of the key reasons for KW seeking outside help with this project was their lack of Elasticsearch skills. Throughout the consultancy period, it was very important for Gio's team to upskill and learn from BigData Boutique along the way, to give them the necessary skills for the future.

“They are very knowledgeable, highly technical, easy to work with, and you can tell they know their craft inside and out” notes a member of KW's technical team. “You can tell just by talking to them, how deep their knowledge in search technology is.”

There were concerns about how much knowledge KW's technical team would be able to learn by osmosis as the project was delivered entirely remotely, but that did not pose a problem for the BigData Boutique team. “Their knowledge, and willingness to share that knowledge, led to a very good experience overall as their clients” states Gio.

As KW continues to upskill in Elasticsearch, there is an ongoing relationship for serious configuration changes.

“When we are making major alterations to the production environment, we will always consult BigData Boutique and bounce ideas off them,” says a member of the KW infrastructure team. KW has maintained a bank of available consultancy hours, giving them full flexibility to call on BigData Boutique's expertise whenever it is needed.

Whilst the new Elasticsearch-based platform is still in beta at KW, the user feedback has been outstanding. “Performance, accuracy, and relevance have all greatly increased for our agents, which is a great outcome”.