Configuring a Self-Managed Elastic Cluster on Linux – Single and multiple nodes

-

Elasticsearch, renowned for its search engine capabilities, boasts powerful security features for data protection. This comprehensive guide walks you through the process of setting up a Multi-Node ElasticSearch cluster with custom configurations on Linux. Starting from scratch, stack limits, memory map configurations, and Elasticsearch deployment. The guide concludes with verification using CURL.

Elasticsearch, renowned for its power and scalability as a search engine, offers robust security features to safeguard your data. We are set to establish a Multi Node ES cluster with tailored configurations on linux machines. This blog post serves as a step-by-step guide, walking you through the process of configuring an Elastic cluster with everything from scratch using the settings outlined below.


On Linux systems, persistent limits can be set for a particular user by editing the /etc/security/limits.conf file. To set the maximum number of open files for the elasticsearch user to 65,535, add the following line to the limits.conf file:

Open the limits.conf file as root:

sudo vim /etc/security/limits.conf

Add the following line near the bottom:

elasticsearch  -  nofile  65535

Read more here : https://www.elastic.co/guide/en/elasticsearch/reference/current/file-descriptors.html and https://www.elastic.co/guide/en/elasticsearch/reference/current/setting-system-settings.html

Systemd configurationedit

When using the RPM or Debian packages on systems that use systemd, system limits must be specified via systemd.

The systemd service file (/usr/lib/systemd/system/elasticsearch.service) contains the limits that are applied by default.

To override them, add a file called /etc/systemd/system/elasticsearch.service.d/override.conf (alternatively, you may run sudo systemctl edit elasticsearch which opens the file automatically inside your default editor). Set any changes in this file, such as:

[Service]
LimitMEMLOCK=infinity

Once finished, run the following command to reload units:

sudo systemctl daemon-reload

Open the sysctl.conf file as root:

sudo vim /etc/sysctl.conf
Add the following line at the bottom:
vm.max_map_count=262144

Elasticsearch also requires the ability to create many memory-mapped areas. The maximum map count check checks that the kernel allows a process to have at least 262,144 memory-mapped areas and is enforced on Linux only. To pass the maximum map count check, you must configure vm.max_map_count via sysctl to be at least 262144.

Read more here : https://www.elastic.co/guide/en/elasticsearch/reference/current/_maximum_map_count_check.html#:~:text=The%20maximum%20map%20count%20check,to%20be%20at%20least%20262144%20.

Load the new sysctl values:

sudo sysctl -p

Install the binaries for Elasticsearch 7 :

https://dev.smirnov.app/2024/08/installing-and-configuring-elasticsearch.html


Now We will Configure each node’s elasticsearch.yml file here as per specifications.
Log in to each node and become the elastic user:

  • Open the elasticsearch.yml file:
  • vim /etc/elasticsearch/elasticsearch.yml

Elasticsearch Configuration File Overview

The supplied YAML file encompasses crucial configurations tailored for a production Elasticsearch cluster. Let’s delve into key sections:

Cluster Configuration

cluster.name: Your_Cluster_Name

Sets a descriptive name for your cluster. All nodes must share the same cluster name to join the same cluster.

Node Configuration

node.name: node-01

Specifies a descriptive name for your node. The default node name is the machine’s hostname upon Elasticsearch startup.

Path Configuration

path.data: /elastic_data_directory/ElasticData

path.logs: /elastic_log_directory/ElasticLogs

Defines data and log directory paths. Your Elastic data and logs will be stored in these paths.

Network Configuration

network.host: 0.0.0.0 #  https://www.elastic.co/guide/en/elasticsearch/reference/8.14/modules-network.html

http.port: 9200

Configures network settings, making Elasticsearch accessible on the specified IP and port. The setting network.host: 0.0.0.0 in the Elasticsearch configuration allows Elasticsearch to bind to all available network interfaces on the server. This means that Elasticsearch will listen for incoming connections on all IP addresses assigned to the server.

Discovery Configuration – Multi Node Cluster

discovery.seed_hosts: [“X.X.X.5”, “X.X.X.6”, “X.X.X.7”]

cluster.initial_master_nodes: [“node-01”, “node-02”, “node-03”]

Establishes the discovery process with seed hosts and initial master nodes. For a single-node cluster, use:

discovery.seed_hosts: [“X.X.X.5”]

cluster.initial_master_nodes: [“node-01”]

cluster.initial_master_nodes

After the cluster forms successfully for the first time, remove the cluster.initial_master_nodes setting from each node’s configuration. Do not use this setting when restarting a cluster or adding a new node to an existing cluster.

https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#initial_master_nodes


Security Configuration

xpack.security.enabled: false # true

xpack.security.transport.ssl.enabled: true

xpack.security.transport.ssl.verification_mode: certificate

xpack.security.transport.ssl.client_authentication: required

xpack.security.transport.ssl.keystore.path: elastic-certificates.p12

xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

We put xpack.security.enabled: false to acces elastic without password.


Generate the certificate authority

/usr/share/elasticsearch/bin/elasticsearch-certutil ca
/usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12

The command will create elastic-stack-ca.p12 and elastic-certificates.p12 :

ls  /usr/share/elasticsearch/elastic-certificates.p12


Copy elastic-certificates.p12 to /etc/elasticsearch/:

cp /usr/share/elasticsearch/elastic-certificates.p12 /etc/elasticsearch/


Give the right permissions to elasticuser on each node:

sudo chown elasticsearch: /etc/elasticsearch/elastic-certificates.p12


On every node in your cluster, copy the elastic-certificates.p12 file to the $ES_PATH_CONF directory.

scp root@xxx.xx.xx.xxx:/etc/elasticsearch/elastic-certificates.p12 /etc/elasticsearch/elastic-certificates.p12



Enables X-Pack Security and configures SSL for transport.

Configuring TLS between nodes is the basic security setup to prevent unauthorised nodes from accessing your cluster. 

Refer this documentation to create these certificates and read more about Security configuration: Set up basic security for the Elastic Stack | Elasticsearch Guide [8.11] and security-minimal-setup.html 

SSL HTTP Configuration

xpack.security.http.ssl.enabled: true

xpack.security.http.ssl.verification_mode: certificate

xpack.security.http.ssl.keystore.path: certificate.pfx

When you enable TLS on the HTTP layer it provides an additional layer of security to ensure that all communications to and from your cluster are encrypted.

For more information on creating certificates, refer to the official documentation.

Set up basic security for the Elastic Stack plus secured HTTPS traffic | Elasticsearch Guide [8.11]

Now we will configure the heap for each node per instructions.

Log in to each master node:

Open the jvm.options file:

vim /$ES_HOME/elasticsearch/config/jvm.options

Change the following lines:

-Xms1g

-Xmx1g

Log in to each data node:

Open the jvm.options file:

vim /$ES_HOME/elasticsearch/config/jvm.options

Change the following lines:

-Xms30g

-Xmx30g

Note : By default, Elasticsearch automatically sets the JVM heap size based on a node’s roles and total memory. Using the default sizing is recommended for most production environments.

To override the default heap size, set the minimum and maximum heap size settings, Xms and Xmx. The minimum and maximum values must be the same.

The heap size should be based on the available RAM:

Set Xms and Xmx to no more than 50% of your total memory. Elasticsearch requires memory for purposes other than the JVM heap. For example, Elasticsearch uses off-heap buffers for efficient network communication and relies on the operating system’s filesystem cache for efficient access to files. The JVM itself also requires some memory. It’s normal for Elasticsearch to use more memory than the limit configured with the Xmx setting. 


Source: 

- https://www.elastic.co/guide/en/elasticsearch/reference/7.17/settings.html

- https://www.elastic.co/guide/en/elasticsearch/reference/current/add-elasticsearch-nodes.html



Comments

Post a Comment

Popular posts from this blog

Installing the Certbot Let’s Encrypt Client for NGINX on Amazon Linux 2

-
Image
Installing the Certbot Let’s Encrypt Client The certbot package is not available through the package manager by default. You will need to enable the  EPEL  repository to install Certbot. sudo yum install epel-release sudo yum install certbot-nginx Set Up NGINX certbot  can automatically configure  NGINX  for SSL/TLS. It looks for and modifies the server block in your  NGINX  configuration that contains a server_name directive with the domain name you’re requesting a certificate for. In our example, the domain is  www.mydomain.com . Assuming you’re starting with a fresh NGINX install, use a text editor to create a file in the  /etc/nginx/conf.d  directory named  domain_name.conf  (so in our example, www.mydomain.com.conf). Specify your domain name (and variants, if any) with the  server_name  directive: server { listen 80 default_server; listen [::]:80 default_server; root /var/www/html; server_name myd...
Read more

psql: error: connection to server at "localhost" (127.0.0.1), port 5433 failed: ERROR: failed to authenticate with backend using SCRAM DETAIL: valid password not found

-
Image
ERROR psql: error: connection to server at "localhost" (127.0.0.1), port 5433 failed: ERROR: failed to authenticate with backend using SCRAM DETAIL: valid password not found Since Pgpool-II is a PostgreSQL proxy that works between clients and PostgreSQL servers, the authentication comprises two steps: Authentication between client and Pgpool-II Authentication between Pgpool-II and PostgreSQL servers Starting with Pgpool-II 4.0, Pgpool-II supports scram-sha-256 authentication. scram-sha-256 authentication method is strongly recommended because it is the most secure password-based authentication method. Solution 1 If PostgreSQL servers require  MD5  or  SCRAM  authentication for some user’s authentication but the password for that user is not present in  pool_passwd , then enabling  allow_clear_text_frontend_auth  will allow the  Pgpool-II  to use clear-text-password authentication with user to get the password in plain text form from the user ...
Read more

Deploy Nuxt.js app using Apache 2

-
  Deploy your dynamic Nuxt.js app Nuxt.js is an amazing server side rendering application framework for Vue.js. In this article we will see how to deploy Nuxt.js application to AWS instance in universal app mode. Where in Nuxt.js you can choose between Universal, Static Generated or Single Page application. Before we start deploying our dynamic app, if you want just to deploy a static website, you can run this command: npm run generate and it will generate the files and put them in  dist  folder which you can upload to any server you want. But this will not work for universal apps. 1. Build your app To build a ssr app you need to run the following command: npm run build The generated files will not be in the  dist  folder as for the static app, but you can find all generated files in  .nuxt  folder which will be the entry point. 2. Upload to the server There are two ways to do this upload your whole application with the source code or just upload the n...
Read more