Set Up a New Aternity ETL Server

The Aternity ETL Server is a component of Aternity on-premise, which aggregates and converts performance data from the Aternity Oracle Database Server to the Vertica database format, and sends it to the Vertica Database Server.

The Aternity Vertica Database Server stores the performance data from the past 31 days in the Vertica format, which is most efficient for displaying in Aternity dashboards.

The Vertica server provides data for efficient display in Aternity dashboards
Tip

You run the setup as a Python script with a long list of parameters needed. We recommend that you prepare it beforehand in as plain text file, and then run it.

While you choose the destination folder of the setup, note that it also inserts files in /opt/vertica and /opt/sqlcl.

Before you begin

Ensure that you have completed the following:

  • Download the latest Aternity on-premise's main setup package from the Riverbed support site by selecting Software (size).

  • Complete setting up the Aternity Oracle Database Server.

  • Open TCP port 1521 on your Aternity Oracle Database Server and TCP port 5433 on the Aternity Vertica Database Server to enable Aternity ETL Server communication (learn more).

  • Complete setting up one or more Aternity Vertica Database Servers.

  • Complete setting up the Aternity Dashboard Server and the set up of the Dashboard Gateway.

  • Complete setting up the Aternity Management Server.

  • Complete the setup (publishing) of the Aternity dashboard layouts.

  • (Optional) Complete setting up a dedicated Aternity Data Warehouse Server computer.

  • Check that the server conforms to the minimum system requirements:

    Attribute Requirement

    Hardware

    Hardware specifications depend on the size of your Aternity on-premise deployment. Choose the sizing and hardware specifications for your deployment size.

    In addition, check the ports which you must open on this server (learn more).

    Network

    Ensure this server has a static IP v4 address, and that you open the required ports.

    Operating system for Aternity ETL Server

    • Linux CentOS 7.3 - 7.5. To verify your version of CentOS, enter cat /etc/centos-release For the kernel version, enter uname -r and see the first two numbers.

    • (Aternity 10.0.1) Red Hat Enterprise Linux (RHEL) 7.4 or 7.5.

      (Aternity 10.0) Red Hat Enterprise Linux (RHEL) 7.4.

      To verify the RHEL version, enter cat /etc/redhat-release

    Virtual memory settings for Aternity ETL Server

    Verify that your system has a swap file of at least 6GB which is active, by entering free -m.

    To create a 6GB swap file and enable it, enter the following commands, one line at a time. For RHEL, enter:

    dd if=/dev/zero of=/swapfile bs=1024 count=65536
mkswap /swapfile
swapon /swapfile

    For CentOS systems, enter:

    fallocate -l 6G /swapfile
mkswap /swapfile
swapon /swapfile

    To set the swap file as permanent, edit the system partition settings in /etc/fstab in a plain text editor such as vim, and add the following line at the end of the file:

    /swapfile   swap    swap    defaults    0 0

    Then save and exit the text editor. The system enables the new swap file when you restart the server.

    Resource limits for ETL Server

    The user that runs the ETL Server must have an open file limit of 65536 or more.

    Set the number by editing the system's resource limits in etc/security/limits.conf in a plain text editor like vim, and add the following line at the end of the file:

    etl_username -     nofile  65536
    If the username is root, enter 
    root -     nofile  65536

    Now enter ulimit -n to verify the limit.

    Then save and exit the editor.

    Chrony

    The ETL Server setup script automatically uses a utility called Chrony, which comes with CentOS 7, to synchronize time and time zones between the Aternity components.

    Important

    Synchronize all Aternity components to have the same date, time AND time zone.

    Confirm the system runs the Chrony utility, by entering systemctl status -l chronyd

    If your system does not have Chrony, search for Chrony, download the version for CentOS 7, and enable it by entering yum install chrony.

    Permissions

    The user with permissions to run the ETL Server must also be allowed to run scheduled (crontab) tasks. While the user running the setup script must have root or sudo root privileges, the user which runs the Aternity ETL Server process does NOT need to have sudo root privileges.
Important

For best results, we recommend that this machine be dedicated to running ONLY the Aternity ETL Server.

Procedure

  1. Step 1 Access the computer of the Aternity ETL Server.

    Run the setup as a user with root or sudo root privileges on the computer. To verify, enter sudo id.

  2. Step 2 Select the ETL Server setup package, etl.tar, which you downloaded as part of the Aternity on-premise server setup package.

    Copy it to a temporary location on this computer.

  3. Step 3 Unpack the files in the temporary folder by entering tar -xvf ETL.tar.
    Unpack the ETL setup files
    Field Description
    -x

    Use -x to unzip the contents of the package.
    -v

    Use -v to output all messages (verbose).
    -f

    Use -f to specify the filename.
  4. Step 4 Create a destination folder to hold the ETL Server data and program files.
  5. Step 5 Navigate to the temporary folder containing the unpacked setup files and run this command, replacing the parameters with those from your configuration.

    python ETL_Install.py -etl_home_folder /vertica/etl_data -oracle_user aternity -oracle_db_password mypassword -oracle_ip oracledb.mycompany.com -oracle_port 1521 -oracle_service aternity -vertica_db_name aternity -vertica_node verticadb1.mycompany.com -vertica_db_admin_user dbadmin -vertica_db_admin_password mypassword -vertica_cluster_ips verticadb1.mycompany.com -etl_user etluser -etl_user_group_name etluser

    This is a single command line containing all of the parameters needed to run the Aternity ETL Server setup. We recommend that you carefully edit this command in a text editor to ensure that you have correctly entered the parameters, and then paste the text into the command prompt.

    Run the setup script using Python
    Field Description

    etl_home_folder

    Provide a path to a destination folder where setup writes the program files. This folder also stores ETL data files.

    Note

    While you choose the destination folder of the setup, note that it also inserts files in /opt/vertica and /opt/sqlcl.

    oracle_user

    oracle_db_password

    Enter the name of the ATERNITY schema (username) and password of the Aternity Oracle Database Server.

    oracle_ip

    Enter the hostname or FQDN (recommended) or IP v4 address of the Oracle Database Server.

    oracle_port

    Enter the port required to access the Oracle Database Server (default is 1521).

    oracle_service

    Enter the Oracle database service name (usually Aternity). This is the alias to the instance of the Aternity database.

    vertica_db_name

    Enter the name of the Vertica database (not the schema).

    vertica_node

    Enter the hostname, FQDN (recommended), or IP v4 address of any one of the Vertica Database Servers in the Vertica cluster.

    The Aternity ETL Server attempts to connect to this server first, and if it is unable, it tries to connect with the next server in the cluster.

    If you only have one Vertica Database Server, enter its hostname, FQDN (recommended) or IP v4 address.

    vertica_db_admin_user

    vertica_db_admin_password

    Enter the username and password of the Vertica database administrator, with privileges to create schemas and prepare the database.

    vertica_cluster_ips

    Enter a comma-separated list of all of the hostnames or FQDNs (recommended) or IP v4 addresses of all of the Vertica Database Server nodes in the cluster, for example, vertica_db1.mydomain.com,vertica_db2.mydomain.com,vertica_db3.mydomain.com.

    If you only have one Vertica Database Server, enter its name or address.

    etl_user

    Enter the name of the user with permissions to run the ETL Server. This user must have permissions to run scheduled (crontab) tasks. Typically, you would not run the Aternity ETL Server as a root user, which is the default if you do not specify another user.

    etl_user_group_name

    Enter the group of the etl_user.

    The setup notifies you when it completes successfully.

    Your setup of the ETL Server completed successfully

    The setup writes a log file, installer.log to the ETL destination folder.