SaltStack Support

SaltStack Multi-Master HA Setup for SSE

Some users of SaltStack Enterprise will want to setup a multi-master environment, in which separate hosts are established as Salt Masters, SaltStack Enterprise servers, Redis servers, and PostgreSQL servers. The following guide will describe how to deploy a SaltStack Enterprise Multi-Node HA System.

Part 1: Setup and Configure Basic Environment

The following nodes will need to be created: two Salt masters, two SaltStack Enterprise Servers (RaaS heads), a Redis Server, and a PostgreSQL Server.

  1. SSE-Master01
  2. SSE-Master02
  3. SSE-PGSQL01
  4. SSE-RAAS01
  5. SSE-RAAS02
  6. SSE-Redis01

Step 1: Install and Configure the Salt Master on SSE-Master01 and SSE-Master02

a. On both SSE-Master01 and SSE-Master02, import the SaltStack GPG Key:

rpm --import https://repo.saltstack.com/yum/redhat/7/x86_64/archive/2018.3.4/SALTSTACK-GPG-KEY.pub

b. On both SSE-Master01 and SSE-Master02, create the SaltStack repo file:

Create: /etc/yum.repos.d/salt-2018.3.4.repo
- - - - - - - - - - - - - - - - - - - - -
[saltstack-repo]
name=SaltStack repo for RHEL/CentOS $releasever
baseurl=https://repo.saltstack.com/yum/redhat/$releasever/$basearch/archive/2018.3.4
enabled=1
gpgcheck=1
gpgkey=https://repo.saltstack.com/yum/redhat/$releasever/$basearch/archive/2018.3.4/SALTSTACK-GPG-KEY.pub
Note: Pinning the installation to a specific release of Salt for your Salt Masters is strongly recommended. Please keep your minions at the same level as the master. As you upgrade your environment, please do not let your minions lapse more than two major versions behind the masters.

c. On both SSE-Master01 and SSE-Master02, clean the yum expire-cache:

yum clean expire-cache

d. On both SSE-Master01 and SSE-Master02, install the Salt Master:

yum install salt-master

e. Enable and Start the Salt Master on SSE-Master01:

systemctl enable salt-master
systemctl start salt-master

f. Retrieve the Salt Keypair Generated by Starting the Salt Master on SSE-Master01

cd /etc/salt/pki/master/
and scp the master.pem and master.pub files to the same location on SSE-Master02

g. Enable and Start the Salt Master on SSE-Master02:

systemctl enable salt-master
systemctl start salt-master

Step 2: Install and Configure the Salt Minion on SSE-Master01 and SSE-Master02

a. On both SSE-Master01 and SSE-Master02, install the Salt Minion:

yum install salt-minion

b. Set the Minion ID on SSE-Master01:

echo SSE-Master01 > /etc/salt/minion_id

c. Set the Minion ID on SSE-Master02:

echo SSE-Master02 > /etc/salt/minion_id

d. Set the Salt Master Configuration on both SSE-Master01 and SSE-Master02:

cat > /etc/salt/minion.d/masters.conf <<EOF
master:
  - SSE-Master01
  - SSE-Master02
EOF

e. Enable and Start the Salt Minion on SSE-Master01 and SSE-Master02:

systemctl enable salt-minion
systemctl start salt-minion

f. Accept the Salt Keys for Minions SSE-Master01 and SSE-Master02 from both Masters:

salt-key -ya SSE-Master01
salt-key -ya SSE-Master02
Note: There are various ways to keep Salt Keys synchronized including shared storage, rsync, Salt reactors, etc... a detailed discussion of this topic is beyond the scope of this guide.

g. Verify that both minions are responding to both masters via test.ping:

salt 'SSE-Master*' test.ping

h. As an additional exercise, observe the message bus of both Salt masters while performing a test.ping.  Execute the following command from a separate terminal window for each master to observe Salt events in real time:

salt-run state.event pretty=True

Execute the following command from your original terminal window for each master. Note that each Salt Minion responds to the Salt Master that published the event:

salt 'SSE-Master*' test.ping

Step 3: Install and Configure the Salt Minion on SSE-PGSQL01, SSE-RAAS01, SSE-RAAS02, and SSE-Redis01

a. On SSE-PGSQL01, SSE-RAAS01, SSE-RAAS02, and SSE-Redis01, import the SaltStack GPG Key:

rpm --import https://repo.saltstack.com/yum/redhat/7/x86_64/archive/2018.3.4/SALTSTACK-GPG-KEY.pub

b. On SSE-PGSQL01, SSE-RAAS01, SSE-RAAS02, and SSE-Redis01, create the SaltStack repo file:

Create: /etc/yum.repos.d/salt-2018.3.4.repo
- - - - - - - - - - - - - - - - - - - - -
[saltstack-repo]
name=SaltStack repo for RHEL/CentOS $releasever
baseurl=https://repo.saltstack.com/yum/redhat/$releasever/$basearch/archive/2018.3.4
enabled=1
gpgcheck=1
gpgkey=https://repo.saltstack.com/yum/redhat/$releasever/$basearch/archive/2018.3.4/SALTSTACK-GPG-KEY.pub

c. On SSE-PGSQL01, SSE-RAAS01, SSE-RAAS02, and SSE-Redis01, clean the yum expire-cache:

yum clean expire-cache

d. On SSE-PGSQL01, SSE-RAAS01, SSE-RAAS02, and SSE-Redis01, install the Salt Minion:

yum install salt-minion

e. Set the Minion ID on SSE-PGSQL01:

echo SSE-PGSQL01 > /etc/salt/minion_id

f. Set the Minion ID on SSE-RAAS01:

echo SSE-RAAS01 > /etc/salt/minion_id

g. Set the Minion ID on SSE-RAAS02:

echo SSE-RAAS02 > /etc/salt/minion_id

h. Set the Minion ID on SSE-Redis01:

echo SSE-Redis01 > /etc/salt/minion_id

i. Set the Salt Master Configuration on SSE-PGSQL01, SSE-RAAS01, SSE-RAAS02, and SSE-Redis01:

cat > /etc/salt/minion.d/masters.conf <<EOF
master:
  - SSE-Master01
  - SSE-Master02
EOF

j. Enable and Start the Salt Minion on SSE-PGSQL01, SSE-RAAS01, SSE-RAAS02, and SSE-Redis01:

systemctl enable salt-minion
systemctl start salt-minion

k. Accept the Salt Keys for Minions SSE-Master01 and SSE-Master02 from both Masters:

salt-key -ya SSE-PGSQL01
salt-key -ya SSE-RAAS01
salt-key -ya SSE-RAAS02
salt-key -ya SSE-Redis01

l. Validate that All Minions are Responding to both Masters:

salt \* test.ping
Note: If all minions are responding to both masters, you are ready to proceed with the SaltStack Enterprise Installation States.

Step 4: Install SaltStack Enterprise

Using the instructions from https://enterprise.saltstack.com, download the SSE Installer and follow the Multi Node guide from the "Use the Installer" section in the documentation.

Related articles