Elasticsearch

Notes about Elasticsearch

About Elastic Index

Where all your data is logically stored, all ELK's operations depend on this concept, you have configurations like ILM, Index Template, Field Mapping, and many more. So it is essential that you understand Elastic Index

The Index Itself - Inverted Index

It is using an inverted index data structure, But why Elastic index is an inverted index? -> because the Elastic index is based on the Lucence index (which is an inverted index), for simplification the inverted index is similar to the book index (at the end of your book) which tells you which pages contain the information you search for, it looks like this

Book index

But on the technical side, it actually looks like this

Source: https://www.elastic.co/blog/found-elasticsearch-from-the-bottom-up

It tokenizes (NLP concept) the words and scores them, so when you search for a specific keyword it knows which documents have the highest score and show them to you.

-> Using this kind of data structure makes searching much faster (I don't actually benchmark it but most people already did), Well here is just the high-level concept, the speed lies in the implementation of the Lucence index (It is much more complex, maybe sometime, later on, I'll try to go deep on that)

The Making of Indexes or Shards

Introducing Shard - which basically is the Lucence index. The Elasticsearch index consists of Shards, Shard is where your real data is stored.

My use case

Troubleshooting