Zeroska
  • 🐱Zeroska - The Bold
  • Computer/Cyber Security
    • ⭐Fresh off the boat - for new Blue Teamer/ SOC Analyst
    • 💀Necromancer (DFIR)
      • 🕵️DFIR and DFIR case
        • 🗃️Case Files
          • Misconfiguration 0x01
      • 📔My own DFIR notes
        • The Mark of The Web (MOTW)
        • LNK Shortcut
        • Prefetch, AmCache, ShimCache in Windows
        • Malicious Document (VBA, Office, PDF, ...)
        • SRUM (System Resource Usage Monitor)
        • Volatility notes
        • Understand Logon Session in Windows
        • "Very" Hidden sheets in Excel
        • Hidden Processes
      • 📔Notes
        • Useful Resources
        • Useful Resources DFIR
        • Saved Content (Backup for the OG content get deleted)
          • How to be best SOC analyst ever
    • 🧙‍♂️Defense Witchcraft
      • Data Analysis as a Defender
      • Active Directory
      • Windows Event Logs
        • Authentication (Windows Account Life Cycle Events)
      • MS Exchange
      • Windows
    • 💙My Current Blue Team Operation
      • VLAN Monitoring Template
      • Daily Monitoring Report
      • General Playbook
      • JupyterHub Configuration Guide
      • How to train your Blue Team
    • 🏭ICS/OT
      • Setup ICS/OT Simulation Learning Lab
      • Smart Grids
      • Learning Modbus TCP
      • Learning DNP3
      • Resources/Blogs/Links
    • 🥷Threat Intelligence
      • Small Cyber Threat Intelligence Program
      • Intelligence Analysis [VN]
      • OSINT Information Monitoring
        • Telegram Info-Stealer Monitoring
      • C2 Tracker
      • Fake Website Tracker
      • Dark Web Investigation Attempt
      • Threat Intelligence Quick Win
      • Resources/Blogs/Links
  • 😒Computer and Technology
    • My Home Lab Setup
    • 🐴ELK Stack
      • Useful Resources
      • ELK Stack - Container - Docker Compose - AWS ECS
      • ECS - Elastic Common Schema
      • Elasticsearch
      • Logstash
      • Troubleshooting and Problems
    • 🐧Linux
      • Setting up good SSH Authentication
      • Configure Auditd and how to leverage it
      • ELF Format
      • Useful Links
    • 📦Containers
      • Docker container logs is quite big
      • Worth Reading
    • 🪠Splunk Learning Experience
      • Splunk Test Lab
      • Data Collection Tier (How to get the data)
      • SC4S Custom Filter For Windows Event Log in Syslog Format (NXLog)
      • Useful Resources
  • Threat Hunting
    • Hunting for Implant
    • Using STRIDE and DREAD
    • 🐳Predators and Preys (Computing)
      • Tracks or Hunting Artifacts
    • 📦Network Packet Analysis
      • The Basic
      • Tshark | Zeek Copy & Paste
      • Network Analysis Resources
    • Grep | Powershell Search | Regex
    • Hunting Resources
  • 🎵In my remains
    • Choices
    • The Art of Facing Unknown Problems
    • Build the best DFIR team
    • Reverse Engineering - Đồ án hướng ngành A "Hụt" của tôi
Powered by GitBook
On this page
  • About Elastic Index
  • The Index Itself - Inverted Index
  • The Making of Indexes or Shards
  • My use case
  • Troubleshooting

Was this helpful?

  1. Computer and Technology
  2. ELK Stack

Elasticsearch

Notes about Elasticsearch

PreviousECS - Elastic Common SchemaNextLogstash

Last updated 1 year ago

Was this helpful?

About Elastic Index

Where all your data is logically stored, all ELK's operations depend on this concept, you have configurations like ILM, Index Template, Field Mapping, and many more. So it is essential that you understand Elastic Index

The Index Itself - Inverted Index

It is using an data structure, But why Elastic index is an inverted index? -> because the Elastic index is based on the Lucence index (which is an inverted index), for simplification the inverted index is similar to the book index (at the end of your book) which tells you which pages contain the information you search for, it looks like this

But on the technical side, it actually looks like this

It tokenizes (NLP concept) the words and scores them, so when you search for a specific keyword it knows which documents have the highest score and show them to you.

-> Using this kind of data structure makes searching much faster (I don't actually benchmark it but most people already did), Well here is just the high-level concept, the speed lies in the implementation of the Lucence index (It is much more complex, maybe sometime, later on, I'll try to go deep on that)

The Making of Indexes or Shards

Introducing Shard - which basically is the Lucence index. The Elasticsearch index consists of Shards, Shard is where your real data is stored.

My use case

Troubleshooting

😒
🐴
inverted index
Book index
inverted index concept (table, scoring and freq)
Source:
https://www.elastic.co/blog/found-elasticsearch-from-the-bottom-up