Bahaaldine A.’s Post

How Elasticsearch Search Fast? Ever wonder how this open source Elasticsearch achieves near real time search experience. Short answer is by inverted index. Let's break it down with a hands-on example! Consider three simple documents indexed in Elasticsearch: { "_id": 1  "question": "What is LLM ?" } { "_id": 2  "question": "How to Fine tune LLM Models ?" } { "_id": 3  "question": "Top Ten LLM Models" } When these documents get indexed inside Elasticsearch, the data will be stored in inverted index format. Let us have a look on inverted indexed structure after indexing. Term      | Document Id    --------------|------------------ Fine      2         How      2         LLM      1, 2, 3      Models  2, 3        Ten      3         Top      3         What      1         is       1         to       2         tune      2        Now when a user searches for the term 'LLM' Elasticsearch, utilizing the inverted index fetches Document 1,2, 3 as results. Without this inverted index search, we need to go through each document and filter all the documents that match with the search results. This will hugely impact performance when we work with millions of documents. #Elasticsearch #ELK #SearchEngine #Tech

To Elastic Search users - how would you approach this: "We recommend taking the semantics of a common query or query pattern in Elasticsearch and translating it into SQL"? How do you do that for Kibana? Please let us know either in the comments or by reaching out to us directly. We want to talk to you about Elastic! A couple of other interesting thoughts from that Rockset's blog: - "Migration is often viewed as a 4 letter word in IT" - "Elasticsearch became so popular that folks wanted to see what else it could do or just assumed it could cover a slew of use cases, including real-time analytics use cases." - "The lack of proper joins, immutable indexes that need constant vigil, a tightly coupled compute and storage architecture, and highly specific domain knowledge needed to develop and operate it has left many engineers seeking alternatives." Kudos Patrick Druley for that great blog... https://lnkd.in/gPM_V-CB

For anyone interested in learning about everything Elastic is doing on Vector Search and our continuous efforts to make #Elasticsearch the most high performance and efficient vector database out there, this is a must read..... TLDR: Up to 12x faster out of the box....

💡 Some Fundamental Concepts of Elastic Search 💡 ➡ Document: In Elasticsearch, data is stored in JSON format as documents. A document is a piece of information you want to store and make searchable. For example, a document could represent a product, a user, a blog post, or any other data entity. ➡ Index: An index is like a container that holds a collection of documents sharing similar characteristics or properties. Each document belongs to a specific index. Indices are used to logically group related data for efficient searching and retrieval. ➡ Mapping: Before you can index documents, you define a mapping for the index. A mapping specifies the data types and structure of the fields within the documents. It helps Elasticsearch understand how to analyze and store the data for efficient searching. ➡ Indexing: Once the mapping is defined, you can start indexing documents into the specified index. Indexing involves sending JSON documents to Elasticsearch for storage. Elasticsearch automatically parses the JSON, indexes the content, and stores it in an optimized data structure called an inverted index. ➡ Inverted Index: The inverted index is a core component of Elasticsearch's search functionality. It stores terms (words) found in the documents along with references to the documents that contain those terms. This allows for very fast and efficient text-based searches. ➡ Search: After documents are indexed, you can perform searches using Elasticsearch's query DSL (Domain Specific Language) to retrieve relevant documents. Elasticsearch leverages the inverted index to quickly identify and return the matching documents. #elasticsearch #elkstack #elk #monitoring #logging #logs #metrics

This is a great writeup on the new Retriever capability in Elasticsearch, why it's powerful, and how to use it - coming very soon in 8.14! #elasticsearch #vectorsearch

Elevate your Elasticsearch game! Discover 10 essential tips to enhance indexing performance and streamline data storage, search, and analysis. Dive in now and unlock the potential of your Elasticsearch queries! #Elasticsearch #DataAnalytics #PerformanceTips

We're excited to announce the GA release of Atlas Vector Search and Search Nodes, adding even more value to the MongoDB Atlas. There are two key use cases for Atlas Vector Search to build next-gen applications: - Semantic search: searching and finding relevant results from unstructured data, based on semantic similarity - Retrieval augmented generation (RAG): augment the incredible reasoning capabilities of LLMs with feeds of your own, real-time data to create GenAI apps uniquely tailored to the demands of your business. Let's get building! #mongodb #genai #LLM #atlas

Wondering how to detect which index template Elasticsearch will use before creating the index itself? Find out in this blog post: https://gag.gl/NjRrjQ #ElasticSearchLabs #Index #Elasticsearch

🗣 Check out Elasticsearch 8.14 and leverage the power of Elasticsearch Query Language (ES|QL) to quickly search, aggregate and visualize massive data sets in Elasticsearch.

Wondering how to detect which index template Elasticsearch will use before creating the index itself? Find out in this blog post: https://gag.gl/NjRrjQ #ElasticSearchLabs #Index #Elasticsearch