Set Up a Cluster of New Aternity Vertica Database Servers

You can deploy Aternity Vertica Database Servers as a cluster in larger deployments. The number of Vertica Database Servers you require depends on the size of your deployment, and the level of high availability that you need.

When you deploy several Vertica Database Servers in a cluster, Aternity is able to retrieve data more quickly, because Vertica distributes the query workload across the nodes of the cluster. A cluster also provides the advantage of redundancy and high availability (learn more).

The Aternity Vertica Database Server stores the performance data in the Vertica format, which is most efficient for displaying in Aternity dashboards.

To set up a Vertica Database Server cluster, prepare the correct number of computers for your sized deployment, pick one of the computers to run the setup process, configure the parameters file there, and then run two scripts on that computer:

Follow these steps to set up a Vertica Database Server cluster
  • Run the SSH script once. The SSH script establishes secure communications (SSH) between this computer and the other computers in the cluster using an RSA key, giving it access to those computers as the user which you define in the parameters file. As a result, when you run the Vertica Setup Script, you do not need to enter this user's password multiple times.

    If your IT policy requires that you secure an RSA key with a passphrase, do NOT use this SSH script; instead, run the Vertica Setup Script, and manually enter the password as many times as required.

  • Run the Vertica Setup Script just once to automatically set up Vertica on this computer and the other computers in the cluster.

Before you begin

  • Download the latest Aternity on-premise's main setup package from the Aternity Support Site (learn more).

  • Complete setting up the Aternity Oracle Database Server.

  • Remove any MOTD (a "message of the day") from all Vertica Database Servers in a cluster. It can harm the server setup or startup.

  • Prepare all the computers in the cluster to meet the following requirements:

    Attribute Requirement

    Hardware

    Hardware specifications depend on the size of your Aternity on-premise deployment. Choose the sizing and hardware specifications for your deployment size.

    Operating system for Aternity Vertica Database Server

    • Linux CentOS 7.4 or 7.5 or 7.6 or 7.7 or 7.8.

      To verify your version of CentOS, enter cat /etc/centos-release

    • Red Hat Enterprise Linux (RHEL) 7.4 or 7.5 or 7.6 or 7.7 or 7.8.

      To verify the RHEL version, enter cat /etc/redhat-release

    Note

    If you set up a trial deployment using VMWare virtual machines, the operating system in the OVA file comes with CentOS 7.8.

    Verify that the disk partition dedicated to the Vertica data folder (Target directory) uses the EXT4 file system. Enter df -Th to verify.

    If you deploy on virtual machines, you MUST define all resources as dedicated or reserved (Resource Allocation > Reservation). In addition, you can use a Logical Volume Manager (LVM) in Aternity 12 on any hard drive with no restrictions. See Vertica limitations for LVM.

    Vertica version for Aternity Vertica Database Server

    Aternity 12.0 requires that your Vertica Database Server runs HPE Vertica 9.2.1.-11. You can download this with the rest of the Aternity on-premise package.

    Virtual memory settings for Aternity Vertica Database Server

    Verify that your system has a swap file of at least 2GB which is active, by entering free -m.

    To create a 2GB swap file and enable it, enter the following commands, one line at a time.

    dd if=/dev/zero of=/swapfile count=2048 bs=1MiB
    chmod 600 /swapfile
    mkswap /swapfile
    swapon /swapfile
    
    To set the swap file as permanent, edit the system partition settings in /etc/fstab in a plain text editor such as vim:
    vim /etc/fstab

    Add the following line at the end of the file:

    /swapfile   swap    swap    defaults    0 0

    Then save and exit the text editor. The system enables the new swap file when you restart the server.

    Permissions

    Only users who are defined in the sudoers file can run the setup. Start as a non-root user and follow the setup procedure. At some point, you will be prompted to ask your IT representative to define permissions in the sudoers file. It is possible to run the setup as root user on the computer if this user is defined in the sudoers file as well.

    If you run the setup as a user that requires a password for sudo, setup will prompt you for the password. If the user does not require a password for sudo, the setup does not interrupt the process with those prompts.

    Resource limits for Vertica DBA user

    The Vertica DBA user must have bash as its shell, and have read, write and execute permissions on its home folder. (For several nodes in a cluster) It must also be able to SSH to other nodes in the cluster.

    The Vertica DBA user must have an open file limit of 65536 or more. The setup script automatically assigns this setting to the DBA user.

    You can enter a DBA username, password and group in the parameters file, or keep the name which is predefined (dbadmin), if the username does not exist, the script creates it with these properties.

    Network

    Ensure this server has a static IP v4 address, and that you open the required ports.

    Time zone

    Synchronize all Aternity components to have the same date, time AND time zone.

    Target directory

    Create a directory in the EXT4 partition in which the scripts can set up the Vertica software and database. This is the directory you will use in the vertica_parameters.sh file for VERTICA_DATA_DIR parameter.

    Important

    Create an identical directory (identical full pathname) on each computer in the cluster.

  • If you have a cluster of Vertica databases, Vertica recommends not to deploy firewalls on the individual servers within the cluster (learn more). All nodes should be behind a firewall, but if you must use a firewall between nodes, ensure the required ports are open on each node. For a complete list of ports within a cluster, see the ports list (learn more).

  • Configure anti-virus software like McAfee VSEL to exclude all .cat files in the Vertica target directory containing its catalogs. Add an exclusion to the anti-virus application for <Vertica_target_dir>/.*\.cat

Procedure

  1. Step 1 Access one of the cluster computers.
  2. Step 2 Download the latest Aternity on-premise's main setup package from the Aternity Support Site (learn more).

    Sign in to Aternity Support Site, then go to Product Downloads > Aternity Server > Version and select the required version to download. Note that only logged in users with certain permissions can see the download option.

    The setup package is for all Aternity servers.

    Download Aternity on-premise from the website
    There are also entries in the Additional Packages which are required for database setup and others:
    • Tableau Package

    • Oracle Packages

    • Install Scripts

    • DB Schema Scripts

    Refer to Additional Packages for the necessary Oracle images and other setup packages.
  3. Step 3 Locate the Vertica Database Server setup package, vertica-installer-<version>.tar, which you downloaded as part of the Aternity on-premise server setup package.

    Copy it to a temporary location on one of the Vertica nodes (if you have more than one). Perform all the setup operations only on this computer (node 1).

  4. Step 4 Unpack the .tar file by entering tar -xvf vertica-installer-<version>.tar.
    Unpack the Vertica setup folder
    Field Description
    -x

    Use -x to unzip the contents of the package.

    -v

    Use -v to output all messages (verbose).

    -f

    Use -f to specify the filename.

    The downloaded package contains a number of files including:

    File Description

    vertica_parameters.sh

    This file contains the parameters used by the setup script. You must edit this file so that the script will know how and where to set up the Vertica Database Server.

    setup_ssh.sh (for cluster setup only)

    The SSH script establishes secure communications (SSH) between this computer and the other computers in the cluster using an RSA key, giving it access to those computers as the user which you define in the parameters file. As a result, when you run the Vertica Setup Script, you do not need to enter this user's password multiple times.

    Do not edit the script file.

    If your IT policy requires that you secure an RSA key with a passphrase, do NOT use this SSH script; instead, run the Vertica Setup Script, and manually enter the password as many times as required.

    install_cluster.sh (for cluster setup only)

    This script sets up Vertica Database Server on all of the nodes.

    Do not edit this file.

    sudoer_cfg.sh

    This script generates a sudoers configuration covering specific privileges and access permissions, including
    • allowing only a specific user to run commands

    • allowing that user to run only the specified limited list of commands, required for the setup process

    • enabling commands only from the predefined servers

    The generated configuration is based on the details provided in the parameters file (vertica_parameters.sh).

    The package also contains install_vertica.sh, which you do not use in cluster setup, and the internal directory that contains files, scripts and packages that the setup scripts use. Do not edit these files.

  5. Step 5 Configure the setup by opening vertica_parameters in a plain text editor.
    Edit the setup parameters in the vertica_parameters file
    Parameter Description
    RUN_AS_USER

    Enter the username that you already defined on the other servers in the cluster with sudo root or root privileges.

    Make sure the user defined consistently with the same privileges on all servers in the cluster.

    Before running the script, you must have the exact same user with the same permissions on each computer in the cluster.

    If you run the setup as a user with sudo permissions that requires a password, setup will prompt you for the password. If the sudo user does not require a password, the setup does not interrupt the process with those prompts.

    VERTICA_TZ

    Enter the time zone code for the server. The time zone must be identical to that of all other Aternity components.

    For example, enter VERTICA_TZ="America/New_York" or VERTICA_TZ="UTC"

    To see the time zone of a computer, enter timedatectl | grep "Time zone"

    VERTICA_DATA_DIR

    Enter the full path to the target directory for the Vertica setup. You must create the identical target directory (identical full path) on each computer in the cluster before running the setup script.

    Verify that the disk partition dedicated to the Vertica data folder (Target directory) uses the EXT4 file system. Enter df -Th to verify.

    VERTICA_NODE_1

    Enter the IP v4 address of the computer on which you are performing the setup operations.

    Vertica automatically converts hostnames and FQDNs into IP v4 addresses in its configuration file.

    To ensure successful dashboard publishing, do not use a relative address like localhost or 127.0.0.1. Use the IP address as other computers in the cluster see this one.

    VERTICA_OTHER_NODES

    Enter the IP v4 addresses of all of the other computers in the cluster (not this computer). Each address must be surrounded by quotation marks, and separated by a space, for example:

    VERTICA_OTHER_NODES=("11.0.0.102" "11.0.0.103")

    VERTICA_DB_NAME

    Enter the name of the database (case sensitive). It must begin with an alphabetic character, and can contain letters (upper or lower case), numbers and underscores, and must be up to 30 characters.

    The name already in the properties file is aternity, but you can change if needed.

    VERTICA_DB_ADMIN_USER

    Enter the username required to access the Vertica Database Server. The Vertica DBA user must have bash as its shell, and have read, write and execute permissions on its home folder. (For several nodes in a cluster) It must also be able to SSH to other nodes in the cluster.

    The script creates /opt/vertica/* for Vertica software, and assigns read, write and execute to that folder for the required username.

    You can enter a DBA username, password and group in the parameters file, or keep the name which is predefined (dbadmin), if the username does not exist, the script creates it with these properties.

    VERTICA_DB_ADMIN_GROUP

    If the dbadmin user account exists, provide the group to which dbadmin user belongs. If you specify a user that does not already exist, when the script is executed, it will create the user and assign it to the group provided in the parameters file. If neither group nor the user exist, the script will create both.

    If the user must have a specific group membership according to your enterprise policy, enter the user's group name here. If this group does not exist, the script will create it.

    If you leave this blank, the script assigns this user to the verticadba group (default).

    VERTICA_DB_ADMIN_LINUX_PASSWORD

    Enter the password of this user to give access to the Linux operating system. We define a username in the Linux operating system, with its password, and the same username to access the Vertica database, with its separate password.

    Surround the password with an apostrophe (').

    If the user does not exist on the server, it will be created with this password, and if the user already exists, the value provided here must match the actual password of that user.

    If you leave this blank, the script creates user without a password.

    VERTICA_DB_ADMIN_PASSWORD

    Enter the password of this user to give access to the Vertica database as administrator (like DBADMIN). We define a username in the Linux operating system, with its password, and the same username to access the Vertica database, with its separate password.

    Do not use an apostrophe (') in the password.

    Important

    We strongly recommend replacing the default password with a new one, and making a note of it elsewhere. You will need it later for different operations, like start/stop server.

    VERTICA_FAILURE_THRESHOLD

    Enter the level of the error where the script stops the setup and reports the message, hence ignoring any error lower than this level.

    Use this for specific cases where you need to override the setup's errors, AFTER you confirm that all other errors are resolved. For example, using SELinux in enforcing mode generates a specific FAIL level error which you need to ignore. A local firewall deployed on this computer generates a specific WARN error which you may need to ignore.

    The default value (when empty) is WARN, hence it ignores any error message less important than a warning, which is a HINT.

    Enter:

    • HINT to stop even on the smallest errors like hints.

    • WARN to ignore hints only.

    • FAIL to ignore warnings and hints

    • HALT to ignore fail, warn and hints.

    VERTICA_SIZE

    Enter the size of your Aternity deployment, based on the sizing you chose, to set requirements for the Vertica Database Server:

    • TRIAL for demos only, not for production.

    • SMALL

    • MEDIUM

    • LARGE

    • XLARGE

    • XXLARGE

  6. Step 6 Enter ./sudoer_cfg.sh to get the content of the file, meaning the list of commands and access permissions; alternatively, enter ./sudoer_cfg.sh >>vertica_install_configuration to create a file with the list of commands.
  7. Step 7 Supply your IT representative with the content.
  8. Step 8 (IT people only) Grant access permissions in the sudoers file allowing privileged user(s) to set up and administer Vertica Database Server. Learn more.
    Important

    The sudoers file must be updated on every node.

  9. Step 9 To make the Vertica Setup Script run smoothly without entering a password many times, run the setup_ssh.sh script from this computer just once.

    The SSH script establishes secure communications (SSH) between this computer and the other computers in the cluster using an RSA key, giving it access to those computers as the user which you define in the parameters file. As a result, when you run the Vertica Setup Script, you do not need to enter this user's password multiple times.

    Run the setup_ssh.sh on one computer, to set up SSH to the other computers

    If your IT policy requires that you secure an RSA key with a passphrase, do NOT use this SSH script; instead, run the Vertica Setup Script, and manually enter the password as many times as required.

    Run setup_ssh.sh to create an SSH connection between all of the computers
    Field Description

    Enter passphrase (empty for no passphrase)

    Press Enter here to continue without a passphrase.

  10. Step 10 Run ./install_cluster.sh on this same computer.

    You only need to run the ./install_cluster.sh script one time. It sets up the Vertica Database Server on itself and on all of the other nodes automatically in one step.

    Run install_cluster.sh to install Vertica on all of the cluster computers

    The script runs the Vertica setup for you, and displays Vertica's setup messages as it proceeds.

    The script saves the setup log, installer.log, in the setup directory.

    The script indicates when it has completed the installation successfully

    Vertica Database Server setup checks that free disk space meets the hardware requirements. In deployments with maximum two nodes, the setup checks that free disk space is at least 200GB per node. In deployments with three nodes and above, the setup automatically configures high availability and checks that free disk space is at least 400GB per node.

  11. Step 11 Verify the setup by confirming that Vertica is running.
    1. a Switch to the Vertica database administrator user, like dbadmin, by entering su - dbadmin.
    2. b Run the Vertica administration tool by entering admintools.
      Vertica's database administrator tools
    3. c Select the first option View database cluster state.

      Confirm the State is UP.

      Verify the database is running
      Tip

      If the database is not up, return to the main menu and select 3 to start the database.