Chapter 1: Overview of Berlin Buzzwords 2023

Berlin Buzzwords is a prominent conference in Germany focused on the complex processes of managing, analyzing, and navigating large sets of digital data. With a strong emphasis on open-source software, it attracts developers, data scientists, and tech enthusiasts eager to share and learn from one another.

This year, I attended five sessions that spanned a variety of topics, including the establishment of MLOps infrastructure in e-commerce, the debate between semantic and keyword search in the context of Generative Pre-training Transformers (GPT), the nuances of hybrid search, the significance of observability in fraud detection, and strategies for effective technical communication.

Here are my main takeaways from each session.

The Talks

Talk 1: Building MLOps Infrastructure at Japan's Largest C2C E-Commerce Site

Speakers: Ryan Ginstrom, Teo Narboneta Zosa

This engaging session detailed the process of constructing an MLOps infrastructure to enhance Machine Learning (ML) capabilities in search at Mercari, Japan’s leading C2C e-commerce platform. It covered the implementation of a "learning to rank" (LTR) model and the transformation of their in-house systems into a modern MLOps architecture.

Mercari's strategy of integrating Machine Learning into their existing search framework exemplifies the idea of "starting small and evolving over time." Initially, they adopted a minimally invasive method to incorporate ML via LTR, which, while not revolutionary, effectively demonstrated ML's value to stakeholders.

Below is a visual representation of Mercari's initial search pipeline, which was optimized around traditional keyword search techniques, particularly ElasticSearch:

As their infrastructure advanced, they faced various hurdles. Initially, their ML model was tied to the search server, restricting their experimentation pace. To address this, they transitioned the model to a microservice with the addition of a feature store. However, the demand for A/B testing from business users posed further challenges.

Alongside their ML developments, Mercari encountered significant data engineering obstacles. Their data pipeline, constructed with dbt, became overly convoluted due to repetitive SQL queries, highlighting the urgent need for implementing best practices in data engineering. This realization led to a more efficient pipeline, emphasizing the importance of sound engineering principles from the onset.

Interestingly, Mercari opted to incorporate a mature MLOps platform—Seldon—later in their journey. Seldon provides robust ML model monitoring tools and was only introduced when their in-house solution became too complex. Given the market's maturity, I believe they could have benefited from exploring such platforms earlier to potentially streamline their processes and mitigate some challenges.

Below is a visual representation of Mercari's final search pipeline, which now features the Seldon MLOps platform:

Talk 2: Semantic vs. Keyword Search as Context for GPT

Speaker: Tudor Golubenco

In this session, Tudor Golubenco, CTO of Xata, challenged traditional views on using Language Models (LLMs) for Question-Answering (QnA) within documentation data. While LLMs are typically combined with vector search, Xata's innovative strategy integrates LLMs with a keyword search mechanism for enhanced tunability.

Xata aims to enable users to perform QnA on documentation by leveraging OpenAI's ChatGPT to extract context from user inquiries, followed by applying a keyword search—likely a BM25 variant—to retrieve relevant documentation.

An illustration of their unique approach is provided below:

The tunability of keyword search was a key factor in Xata's decision-making process. This method offers multiple variables for adjustment, such as boosters and column weights, enhancing relevance. Below is a comparative illustration of the tunability between the two methods:

This approach appears promising, provided there is a team skilled in fine-tuning keyword searches. However, maintaining a comprehensive dictionary of synonyms—common in BM25 searches—can be laborious. Additionally, the challenge of internationalization could complicate matters unless the team includes multilingual experts. Overall, it is exciting to see the blending of LLM and keyword search to improve Q&A on documentation data.

The first video titled "Berlin Buzzwords 2023: Learning to Hybrid Search" dives deeper into these concepts and showcases innovative methodologies.

Talk 3: Learning to Hybrid Search

Speakers: Roman Grebennikov, Vsevolod Goloviznin

Roman Grebennikov and Vsevolod Goloviznin, the minds behind the Metarank project, presented a fresh perspective on hybrid search. Traditionally, hybrid search has been linked to the combination of BM25 and vector searches; however, their presentation proposed that hybrid search can encompass an ensemble of models, allowing for a robust ranking system that capitalizes on the strengths of various methodologies.

At the heart of their approach is the Metarank project, designed to re-rank search results from any search engine:

Their experiments utilized the enriched Amazon ESCI dataset, which includes a wide range of product-related metadata (available at github.com/shuttie/esci-s), providing a realistic testing ground for e-commerce applications. They employed Metarank to analyze different search strategies against this dataset.

The essence of their methodology lies in combining various techniques. The hybrid model integrates traditional text matching, Learning-to-Rank, and advanced neural search methods, achieving performance surpassing that of any single approach.

Although the cross-encoder incurs higher latency, their hybrid approach yielded encouraging outcomes, particularly when the document list to be re-ranked remained small.

The second video, titled "Berlin Buzzwords 2023: Towards a Decentralized and Collaborative Search Engine," further explores these themes.

Talk 4: Catch the Fraud — with Observability and Analytics

Speaker: Philipp Krenn

Philipp Krenn’s captivating presentation unveiled the unexpected challenges Elastic faced when they initiated an annual competition to reward community contributions, such as pull requests, blog posts, and talks. The introduction of prizes like MacBooks resulted in a surge of fraudulent entries, prompting the need for innovative detection methods.

What made this narrative compelling was how Elastic utilized its own tools, particularly observability utilities and Kibana, to identify fraudulent activities. Through meticulous analysis, curiosity, and diverse data visualization techniques, they successfully differentiated between authentic contributions and attempts to game the system.

The session highlighted the value of integrating search, observability, and analytics data to uncover correlations and build a comprehensive understanding of complex scenarios. The effective use of these capabilities showcased their potential to address unforeseen challenges, demonstrating the power of Elastic's observability tools and Kibana when fully leveraged.

Talk 5: Avoiding Anti-patterns in Technical Communication

Speaker: Sophie Watson

Sophie Watson, a Tech Marketing Engineer at NVidia, delivered practical insights on how to communicate technical ideas effectively while steering clear of common pitfalls. She outlined several anti-patterns:

• Keeping it Brief: Excessive brevity can lead to a lack of context, making it difficult for others to grasp the issue. An illustration exemplifying this problem is provided below:

• Over-Communicating: While context is vital, overwhelming your audience with too much information can be counterproductive. An example is shown below:

Sophie emphasized the importance of striking a balance, with a clear request at the end, such as whether to schedule a call to troubleshoot together or simply vent and move on.

• Chasing Likes: The pursuit of popularity can create a disconnect between your original intentions and your target audience. Viral posts often appeal to the masses, but it's essential to focus on your specific audience and objectives.

• Using Useless Analogies: Analogies can simplify complex concepts, but they must resonate with your audience. Sophie provided an example that illustrates this point:

In essence, ensure your audience shares the same frame of reference.

• Focusing on the Tricky Bits: As technical experts, there’s a tendency to dwell on intricate issues. However, prioritizing user-facing changes—which impact user experience—is often more valuable to your audience.

Sophie concluded that the secret to avoiding these pitfalls lies in always considering your audience and remaining receptive to their feedback. Her talk was an invaluable resource for me as I maintain a technical blog, providing fresh perspectives on refining my communication skills and engaging more effectively with my audience.

Conclusion

In summary, Berlin Buzzwords 2023 was an enlightening conference that delved into various aspects of search technology. If you were unable to attend, I highly recommend watching the session recordings available on YouTube. These presentations serve as a rich resource for anyone seeking innovative insights and approaches in the search domain.