Elasticsearch

What is ElasticSearch?

Elasticsearch is a distributed search and analytics engine. It is an open-source, RESTful search and analytics engine built on top of Apache Lucene. Lucene is a high-performance, full-featured text search engine library written in Java.

In recent times it's been used or adapted by IT companies to finalize with the database formation that doesn't require any prior knowledge to get in-depth into it to use it.

How it's Helpful nowadays?

Elasticsearch is designed to handle large volumes of data and provide near-real-time search and analytics capabilities. It is commonly used for a variety of use cases, including:

Full-Text Search: Elasticsearch is excellent at full-text search, making it a popular choice for applications that require fast and efficient searching of large amounts of textual data.
Log and Event Data Analysis: Many organizations use Elasticsearch to analyze and search through log and event data generated by various systems, applications, and services.
Structured and Unstructured Data Analysis: Elasticsearch can be used to index and analyze both structured and unstructured data. It supports a wide range of data types, including text, numeric, geospatial, and more.
Business Intelligence (BI): Elasticsearch can be integrated with BI tools to provide powerful analytics and visualization capabilities.
Monitoring and Metrics: It is commonly used for monitoring and analyzing system metrics and performance data.
Security Information and Event Management (SIEM): Elasticsearch is often a key component in SIEM solutions, helping organizations manage and analyze security-related data.
Content Discovery: It is used to power content discovery in applications like e-commerce platforms, websites, and content management systems.

Elasticsearch operates in a distributed manner, allowing it to scale horizontally across multiple nodes in a cluster. This enables it to handle large amounts of data and provide high availability and fault tolerance.

Alongside Elasticsearch, the Elastic Stack, also known as ELK Stack, is commonly used. The Elastic Stack includes Elasticsearch, Logstash (for log data ingestion and processing), and Kibana (for data visualization and exploration).

Overall, Elasticsearch is a versatile and powerful tool for searching, analyzing, and visualizing data in real-time, making it a valuable asset in various domains and industries.

Downloading Process

Follow this doc to download the setup of Elasticsearch

Elasticsearch Installation doc

Guide for setting up Elasticsearch

Setting up Elasticsearch involves several steps, including downloading and installing Elasticsearch, configuring it, and starting the Elasticsearch service.

Prerequisites:

Java Required but no worry only download the elasticsearch it's auto download java on it.

Installation:

Download Elasticsearch:
- Visit the official Elasticsearch download page: Elasticsearch Installation.
- Choose the appropriate version for your operating system.

Two paths of Elastic Configuration

 /usr/share/elasticsearch/

 /etc/elasticsearch/

Configuration:

Edit elasticsearch.yml from /etc/elasticsearch/elasticsearh.yml:
- Navigate to the config directory within the Elasticsearch installation directory.
- Open the elasticsearch.yml file in a text editor.
- Configure settings such as cluster name, node name, network host, etc.

Example (elasticsearch.yml):

    cluster.name: my_cluster #cluster name that you want it to be
    node.name: my_node   #node name that you mention
    network.host: 127.0.0.1 or your server ip address 192.168.1.1
    discover_seed.host: [] or #put the ip addresses of servers used to be host
    cluster.initial.master.nodes: ["#node-name#"] #put the node name here

Adjust the settings based on your requirements. and make sure for memory limit you can set jvm-options.yml file to be edit.

Starting Elasticsearch:

Run Elasticsearch:
- Open a terminal or command prompt.
- Navigate to the Elasticsearch bin directory.
- Run the following command to start Elasticsearch:
```
  ./bin/elasticsearch       # For Linux/Mac
  systemctl daemon-reload
  systemctl enable elasticsearch.service
  systemctl start elasticsearch.service
```
```
  .\bin\elasticsearch.bat   # For Windows
```
- Elasticsearch should start, and you'll see log messages indicating its status.

Testing:

Verify Installation:
- Open a web browser and go to http://localhost:9200/. You should see a JSON response with information about your Elasticsearch node.

    curl -XGET http://localhost:9200/_cat/health?v
    curl -XGET http://localhost:9200/_cat/indices

Example Response:

    {
      "name" : "my_node",
      "cluster_name" : "my_cluster",
      "cluster_uuid" : "some-uuid",
      "version" : {
        "number" : "8.11.3",
        "build_flavor" : "default",
        "build_type" : "tar",
        "build_hash" : "some-hash",
        "build_date" : "some-date",
        "build_snapshot" : false,
        "lucene_version" : "some-version",
        "minimum_wire_compatibility_version" : "some-version",
        "minimum_index_compatibility_version" : "some-version"
      },
      "tagline" : "You Know, for Search"
    }

Congratulations! You've successfully set up Elasticsearch. This is a basic configuration for a single-node Elasticsearch instance. In a production environment, you would typically set up a cluster with multiple nodes for scalability and fault tolerance.

Remember to consult the official Elasticsearch documentation for detailed information, advanced configurations, and best practices. Additionally, consider configuring security settings, especially in a production environment, to ensure the proper protection of your Elasticsearch cluster.

This is how a basic setup of elasticsearch to be done.

Next will upload Kibana setup and operations on elasticsearch.

ElasticSearch: A distributed data engine

Table of contents