Table of contents Set Up a New Aternity ETL Server The Aternity ETL Server is a component of Aternity on-premise, which aggregates and converts performance data from the Aternity Oracle Database Server to the Vertica database format, and sends it to the Vertica Database Server. The Aternity Vertica Database Server stores the performance data from the past 31 days in the Vertica format, which is most efficient for displaying in Aternity dashboards. The Vertica server provides data for efficient display in Aternity dashboards Tip You run the setup as a Python script with a long list of parameters needed. We recommend that you prepare it beforehand in as plain text file, and then run it. While you choose the destination folder of the setup, note that it also inserts files in /opt/vertica and /opt/sqlcl. Before you begin Ensure that you have completed the following: Download the latest Aternity on-premise's main setup package from the Riverbed support site by selecting Software (size). Complete setting up the Aternity Oracle Database Server. (Non-DBAs, small deployments only) Use the express database setup. (DBAs only) Use the custom database setup to integrate any sized database properly into your enterprise. Open TCP port 1521 on your Aternity Oracle Database Server and TCP port 5433 on the Aternity Vertica Database Server to enable Aternity ETL Server communication (learn more). Complete setting up one or more Aternity Vertica Database Servers. Complete setting up the Aternity Dashboard Server and the set up of the Dashboard Gateway. Complete setting up the Aternity Management Server. Complete the setup (publishing) of the Aternity dashboard layouts. (Optional) Complete setting up a dedicated Aternity Data Warehouse Server computer. Check that the server conforms to the minimum system requirements: Attribute Requirement Hardware Hardware specifications depend on the size of your Aternity on-premise deployment. Choose the sizing and hardware specifications for your deployment size. In addition, check the ports which you must open on this server (learn more). Network Ensure this server has a static IP v4 address, and that you open the required ports. Operating system for Aternity ETL Server Linux CentOS 7.3 - 7.5. To verify your version of CentOS, enter cat /etc/centos-release For the kernel version, enter uname -r and see the first two numbers. (Aternity 10.0.1) Red Hat Enterprise Linux (RHEL) 7.4 or 7.5. (Aternity 10.0) Red Hat Enterprise Linux (RHEL) 7.4. To verify the RHEL version, enter cat /etc/redhat-release Virtual memory settings for Aternity ETL Server Verify that your system has a swap file of at least 6GB which is active, by entering free -m. To create a 6GB swap file and enable it, enter the following commands, one line at a time. For RHEL, enter: dd if=/dev/zero of=/swapfile bs=1024 count=65536 mkswap /swapfile swapon /swapfile For CentOS systems, enter: fallocate -l 6G /swapfile mkswap /swapfile swapon /swapfile To set the swap file as permanent, edit the system partition settings in /etc/fstab in a plain text editor such as vim, and add the following line at the end of the file: /swapfile swap swap defaults 0 0 Then save and exit the text editor. The system enables the new swap file when you restart the server. Resource limits for ETL Server The user that runs the ETL Server must have an open file limit of 65536 or more. Set the number by editing the system's resource limits in etc/security/limits.conf in a plain text editor like vim, and add the following line at the end of the file: etl_username - nofile 65536 If the username is root, enter root - nofile 65536 Now enter ulimit -n to verify the limit. Then save and exit the editor. Chrony The ETL Server setup script automatically uses a utility called Chrony, which comes with CentOS 7, to synchronize time and time zones between the Aternity components. Important Synchronize all Aternity components to have the same date, time AND time zone. Confirm the system runs the Chrony utility, by entering systemctl status -l chronyd If your system does not have Chrony, search for Chrony, download the version for CentOS 7, and enable it by entering yum install chrony. Permissions The user with permissions to run the ETL Server must also be allowed to run scheduled (crontab) tasks. While the user running the setup script must have root or sudo root privileges, the user which runs the Aternity ETL Server process does NOT need to have sudo root privileges. Important For best results, we recommend that this machine be dedicated to running ONLY the Aternity ETL Server. ProcedureStep 1 Access the computer of the Aternity ETL Server. Run the setup as a user with root or sudo root privileges on the computer. To verify, enter sudo id. Step 2 Select the ETL Server setup package, etl.tar, which you downloaded as part of the Aternity on-premise server setup package. Copy it to a temporary location on this computer. Step 3 Unpack the files in the temporary folder by entering tar -xvf ETL.tar. Unpack the ETL setup files Field Description -x Use -x to unzip the contents of the package. -v Use -v to output all messages (verbose). -f Use -f to specify the filename. Step 4 Create a destination folder to hold the ETL Server data and program files. Step 5 Navigate to the temporary folder containing the unpacked setup files and run this command, replacing the parameters with those from your configuration. python ETL_Install.py -etl_home_folder /vertica/etl_data -oracle_user aternity -oracle_db_password mypassword -oracle_ip oracledb.mycompany.com -oracle_port 1521 -oracle_service aternity -vertica_db_name aternity -vertica_node verticadb1.mycompany.com -vertica_db_admin_user dbadmin -vertica_db_admin_password mypassword -vertica_cluster_ips verticadb1.mycompany.com -etl_user etluser -etl_user_group_name etluser This is a single command line containing all of the parameters needed to run the Aternity ETL Server setup. We recommend that you carefully edit this command in a text editor to ensure that you have correctly entered the parameters, and then paste the text into the command prompt. Run the setup script using Python Field Description etl_home_folder Provide a path to a destination folder where setup writes the program files. This folder also stores ETL data files. Note While you choose the destination folder of the setup, note that it also inserts files in /opt/vertica and /opt/sqlcl. oracle_user oracle_db_password Enter the name of the ATERNITY schema (username) and password of the Aternity Oracle Database Server. oracle_ip Enter the hostname or FQDN (recommended) or IP v4 address of the Oracle Database Server. oracle_port Enter the port required to access the Oracle Database Server (default is 1521). oracle_service Enter the Oracle database service name (usually Aternity). This is the alias to the instance of the Aternity database. vertica_db_name Enter the name of the Vertica database (not the schema). vertica_node Enter the hostname, FQDN (recommended), or IP v4 address of any one of the Vertica Database Servers in the Vertica cluster. The Aternity ETL Server attempts to connect to this server first, and if it is unable, it tries to connect with the next server in the cluster. If you only have one Vertica Database Server, enter its hostname, FQDN (recommended) or IP v4 address. vertica_db_admin_user vertica_db_admin_password Enter the username and password of the Vertica database administrator, with privileges to create schemas and prepare the database. vertica_cluster_ips Enter a comma-separated list of all of the hostnames or FQDNs (recommended) or IP v4 addresses of all of the Vertica Database Server nodes in the cluster, for example, vertica_db1.mydomain.com,vertica_db2.mydomain.com,vertica_db3.mydomain.com. If you only have one Vertica Database Server, enter its name or address. etl_user Enter the name of the user with permissions to run the ETL Server. This user must have permissions to run scheduled (crontab) tasks. Typically, you would not run the Aternity ETL Server as a root user, which is the default if you do not specify another user. etl_user_group_name Enter the group of the etl_user. The setup notifies you when it completes successfully. Your setup of the ETL Server completed successfully The setup writes a log file, installer.log to the ETL destination folder. Parent topic Install Aternity on-premise 10Related tasksSet up Trial POC for Aternity on-premise 10 with VMWare Images (OVA files)Express Setup for the Oracle Database Server for Aternity 10Set Up a New Vertica Database Server for Aternity 10Set Up a New Dashboard Server for Aternity 10Set Up a New Dashboard Gateway for Aternity 10Set Up a New Management Server for Aternity 10Publish or Upload Dashboard Layouts for Aternity 10Set Up a New Data Warehouse Server on a Dedicated ComputerSet Up a New Aggregation Server on a Dedicated ComputerSet Up a New Aternity REST API ServerSet Up a New Aternity Data Source for Portal SavePDF Selected topic Selected topic and subtopics All content Related Links
Set Up a New Aternity ETL Server The Aternity ETL Server is a component of Aternity on-premise, which aggregates and converts performance data from the Aternity Oracle Database Server to the Vertica database format, and sends it to the Vertica Database Server. The Aternity Vertica Database Server stores the performance data from the past 31 days in the Vertica format, which is most efficient for displaying in Aternity dashboards. The Vertica server provides data for efficient display in Aternity dashboards Tip You run the setup as a Python script with a long list of parameters needed. We recommend that you prepare it beforehand in as plain text file, and then run it. While you choose the destination folder of the setup, note that it also inserts files in /opt/vertica and /opt/sqlcl. Before you begin Ensure that you have completed the following: Download the latest Aternity on-premise's main setup package from the Riverbed support site by selecting Software (size). Complete setting up the Aternity Oracle Database Server. (Non-DBAs, small deployments only) Use the express database setup. (DBAs only) Use the custom database setup to integrate any sized database properly into your enterprise. Open TCP port 1521 on your Aternity Oracle Database Server and TCP port 5433 on the Aternity Vertica Database Server to enable Aternity ETL Server communication (learn more). Complete setting up one or more Aternity Vertica Database Servers. Complete setting up the Aternity Dashboard Server and the set up of the Dashboard Gateway. Complete setting up the Aternity Management Server. Complete the setup (publishing) of the Aternity dashboard layouts. (Optional) Complete setting up a dedicated Aternity Data Warehouse Server computer. Check that the server conforms to the minimum system requirements: Attribute Requirement Hardware Hardware specifications depend on the size of your Aternity on-premise deployment. Choose the sizing and hardware specifications for your deployment size. In addition, check the ports which you must open on this server (learn more). Network Ensure this server has a static IP v4 address, and that you open the required ports. Operating system for Aternity ETL Server Linux CentOS 7.3 - 7.5. To verify your version of CentOS, enter cat /etc/centos-release For the kernel version, enter uname -r and see the first two numbers. (Aternity 10.0.1) Red Hat Enterprise Linux (RHEL) 7.4 or 7.5. (Aternity 10.0) Red Hat Enterprise Linux (RHEL) 7.4. To verify the RHEL version, enter cat /etc/redhat-release Virtual memory settings for Aternity ETL Server Verify that your system has a swap file of at least 6GB which is active, by entering free -m. To create a 6GB swap file and enable it, enter the following commands, one line at a time. For RHEL, enter: dd if=/dev/zero of=/swapfile bs=1024 count=65536 mkswap /swapfile swapon /swapfile For CentOS systems, enter: fallocate -l 6G /swapfile mkswap /swapfile swapon /swapfile To set the swap file as permanent, edit the system partition settings in /etc/fstab in a plain text editor such as vim, and add the following line at the end of the file: /swapfile swap swap defaults 0 0 Then save and exit the text editor. The system enables the new swap file when you restart the server. Resource limits for ETL Server The user that runs the ETL Server must have an open file limit of 65536 or more. Set the number by editing the system's resource limits in etc/security/limits.conf in a plain text editor like vim, and add the following line at the end of the file: etl_username - nofile 65536 If the username is root, enter root - nofile 65536 Now enter ulimit -n to verify the limit. Then save and exit the editor. Chrony The ETL Server setup script automatically uses a utility called Chrony, which comes with CentOS 7, to synchronize time and time zones between the Aternity components. Important Synchronize all Aternity components to have the same date, time AND time zone. Confirm the system runs the Chrony utility, by entering systemctl status -l chronyd If your system does not have Chrony, search for Chrony, download the version for CentOS 7, and enable it by entering yum install chrony. Permissions The user with permissions to run the ETL Server must also be allowed to run scheduled (crontab) tasks. While the user running the setup script must have root or sudo root privileges, the user which runs the Aternity ETL Server process does NOT need to have sudo root privileges. Important For best results, we recommend that this machine be dedicated to running ONLY the Aternity ETL Server. ProcedureStep 1 Access the computer of the Aternity ETL Server. Run the setup as a user with root or sudo root privileges on the computer. To verify, enter sudo id. Step 2 Select the ETL Server setup package, etl.tar, which you downloaded as part of the Aternity on-premise server setup package. Copy it to a temporary location on this computer. Step 3 Unpack the files in the temporary folder by entering tar -xvf ETL.tar. Unpack the ETL setup files Field Description -x Use -x to unzip the contents of the package. -v Use -v to output all messages (verbose). -f Use -f to specify the filename. Step 4 Create a destination folder to hold the ETL Server data and program files. Step 5 Navigate to the temporary folder containing the unpacked setup files and run this command, replacing the parameters with those from your configuration. python ETL_Install.py -etl_home_folder /vertica/etl_data -oracle_user aternity -oracle_db_password mypassword -oracle_ip oracledb.mycompany.com -oracle_port 1521 -oracle_service aternity -vertica_db_name aternity -vertica_node verticadb1.mycompany.com -vertica_db_admin_user dbadmin -vertica_db_admin_password mypassword -vertica_cluster_ips verticadb1.mycompany.com -etl_user etluser -etl_user_group_name etluser This is a single command line containing all of the parameters needed to run the Aternity ETL Server setup. We recommend that you carefully edit this command in a text editor to ensure that you have correctly entered the parameters, and then paste the text into the command prompt. Run the setup script using Python Field Description etl_home_folder Provide a path to a destination folder where setup writes the program files. This folder also stores ETL data files. Note While you choose the destination folder of the setup, note that it also inserts files in /opt/vertica and /opt/sqlcl. oracle_user oracle_db_password Enter the name of the ATERNITY schema (username) and password of the Aternity Oracle Database Server. oracle_ip Enter the hostname or FQDN (recommended) or IP v4 address of the Oracle Database Server. oracle_port Enter the port required to access the Oracle Database Server (default is 1521). oracle_service Enter the Oracle database service name (usually Aternity). This is the alias to the instance of the Aternity database. vertica_db_name Enter the name of the Vertica database (not the schema). vertica_node Enter the hostname, FQDN (recommended), or IP v4 address of any one of the Vertica Database Servers in the Vertica cluster. The Aternity ETL Server attempts to connect to this server first, and if it is unable, it tries to connect with the next server in the cluster. If you only have one Vertica Database Server, enter its hostname, FQDN (recommended) or IP v4 address. vertica_db_admin_user vertica_db_admin_password Enter the username and password of the Vertica database administrator, with privileges to create schemas and prepare the database. vertica_cluster_ips Enter a comma-separated list of all of the hostnames or FQDNs (recommended) or IP v4 addresses of all of the Vertica Database Server nodes in the cluster, for example, vertica_db1.mydomain.com,vertica_db2.mydomain.com,vertica_db3.mydomain.com. If you only have one Vertica Database Server, enter its name or address. etl_user Enter the name of the user with permissions to run the ETL Server. This user must have permissions to run scheduled (crontab) tasks. Typically, you would not run the Aternity ETL Server as a root user, which is the default if you do not specify another user. etl_user_group_name Enter the group of the etl_user. The setup notifies you when it completes successfully. Your setup of the ETL Server completed successfully The setup writes a log file, installer.log to the ETL destination folder. Parent topic Install Aternity on-premise 10Related tasksSet up Trial POC for Aternity on-premise 10 with VMWare Images (OVA files)Express Setup for the Oracle Database Server for Aternity 10Set Up a New Vertica Database Server for Aternity 10Set Up a New Dashboard Server for Aternity 10Set Up a New Dashboard Gateway for Aternity 10Set Up a New Management Server for Aternity 10Publish or Upload Dashboard Layouts for Aternity 10Set Up a New Data Warehouse Server on a Dedicated ComputerSet Up a New Aggregation Server on a Dedicated ComputerSet Up a New Aternity REST API ServerSet Up a New Aternity Data Source for Portal
Set Up a New Aternity ETL Server The Aternity ETL Server is a component of Aternity on-premise, which aggregates and converts performance data from the Aternity Oracle Database Server to the Vertica database format, and sends it to the Vertica Database Server. The Aternity Vertica Database Server stores the performance data from the past 31 days in the Vertica format, which is most efficient for displaying in Aternity dashboards. The Vertica server provides data for efficient display in Aternity dashboards Tip You run the setup as a Python script with a long list of parameters needed. We recommend that you prepare it beforehand in as plain text file, and then run it. While you choose the destination folder of the setup, note that it also inserts files in /opt/vertica and /opt/sqlcl. Before you begin Ensure that you have completed the following: Download the latest Aternity on-premise's main setup package from the Riverbed support site by selecting Software (size). Complete setting up the Aternity Oracle Database Server. (Non-DBAs, small deployments only) Use the express database setup. (DBAs only) Use the custom database setup to integrate any sized database properly into your enterprise. Open TCP port 1521 on your Aternity Oracle Database Server and TCP port 5433 on the Aternity Vertica Database Server to enable Aternity ETL Server communication (learn more). Complete setting up one or more Aternity Vertica Database Servers. Complete setting up the Aternity Dashboard Server and the set up of the Dashboard Gateway. Complete setting up the Aternity Management Server. Complete the setup (publishing) of the Aternity dashboard layouts. (Optional) Complete setting up a dedicated Aternity Data Warehouse Server computer. Check that the server conforms to the minimum system requirements: Attribute Requirement Hardware Hardware specifications depend on the size of your Aternity on-premise deployment. Choose the sizing and hardware specifications for your deployment size. In addition, check the ports which you must open on this server (learn more). Network Ensure this server has a static IP v4 address, and that you open the required ports. Operating system for Aternity ETL Server Linux CentOS 7.3 - 7.5. To verify your version of CentOS, enter cat /etc/centos-release For the kernel version, enter uname -r and see the first two numbers. (Aternity 10.0.1) Red Hat Enterprise Linux (RHEL) 7.4 or 7.5. (Aternity 10.0) Red Hat Enterprise Linux (RHEL) 7.4. To verify the RHEL version, enter cat /etc/redhat-release Virtual memory settings for Aternity ETL Server Verify that your system has a swap file of at least 6GB which is active, by entering free -m. To create a 6GB swap file and enable it, enter the following commands, one line at a time. For RHEL, enter: dd if=/dev/zero of=/swapfile bs=1024 count=65536 mkswap /swapfile swapon /swapfile For CentOS systems, enter: fallocate -l 6G /swapfile mkswap /swapfile swapon /swapfile To set the swap file as permanent, edit the system partition settings in /etc/fstab in a plain text editor such as vim, and add the following line at the end of the file: /swapfile swap swap defaults 0 0 Then save and exit the text editor. The system enables the new swap file when you restart the server. Resource limits for ETL Server The user that runs the ETL Server must have an open file limit of 65536 or more. Set the number by editing the system's resource limits in etc/security/limits.conf in a plain text editor like vim, and add the following line at the end of the file: etl_username - nofile 65536 If the username is root, enter root - nofile 65536 Now enter ulimit -n to verify the limit. Then save and exit the editor. Chrony The ETL Server setup script automatically uses a utility called Chrony, which comes with CentOS 7, to synchronize time and time zones between the Aternity components. Important Synchronize all Aternity components to have the same date, time AND time zone. Confirm the system runs the Chrony utility, by entering systemctl status -l chronyd If your system does not have Chrony, search for Chrony, download the version for CentOS 7, and enable it by entering yum install chrony. Permissions The user with permissions to run the ETL Server must also be allowed to run scheduled (crontab) tasks. While the user running the setup script must have root or sudo root privileges, the user which runs the Aternity ETL Server process does NOT need to have sudo root privileges. Important For best results, we recommend that this machine be dedicated to running ONLY the Aternity ETL Server. ProcedureStep 1 Access the computer of the Aternity ETL Server. Run the setup as a user with root or sudo root privileges on the computer. To verify, enter sudo id. Step 2 Select the ETL Server setup package, etl.tar, which you downloaded as part of the Aternity on-premise server setup package. Copy it to a temporary location on this computer. Step 3 Unpack the files in the temporary folder by entering tar -xvf ETL.tar. Unpack the ETL setup files Field Description -x Use -x to unzip the contents of the package. -v Use -v to output all messages (verbose). -f Use -f to specify the filename. Step 4 Create a destination folder to hold the ETL Server data and program files. Step 5 Navigate to the temporary folder containing the unpacked setup files and run this command, replacing the parameters with those from your configuration. python ETL_Install.py -etl_home_folder /vertica/etl_data -oracle_user aternity -oracle_db_password mypassword -oracle_ip oracledb.mycompany.com -oracle_port 1521 -oracle_service aternity -vertica_db_name aternity -vertica_node verticadb1.mycompany.com -vertica_db_admin_user dbadmin -vertica_db_admin_password mypassword -vertica_cluster_ips verticadb1.mycompany.com -etl_user etluser -etl_user_group_name etluser This is a single command line containing all of the parameters needed to run the Aternity ETL Server setup. We recommend that you carefully edit this command in a text editor to ensure that you have correctly entered the parameters, and then paste the text into the command prompt. Run the setup script using Python Field Description etl_home_folder Provide a path to a destination folder where setup writes the program files. This folder also stores ETL data files. Note While you choose the destination folder of the setup, note that it also inserts files in /opt/vertica and /opt/sqlcl. oracle_user oracle_db_password Enter the name of the ATERNITY schema (username) and password of the Aternity Oracle Database Server. oracle_ip Enter the hostname or FQDN (recommended) or IP v4 address of the Oracle Database Server. oracle_port Enter the port required to access the Oracle Database Server (default is 1521). oracle_service Enter the Oracle database service name (usually Aternity). This is the alias to the instance of the Aternity database. vertica_db_name Enter the name of the Vertica database (not the schema). vertica_node Enter the hostname, FQDN (recommended), or IP v4 address of any one of the Vertica Database Servers in the Vertica cluster. The Aternity ETL Server attempts to connect to this server first, and if it is unable, it tries to connect with the next server in the cluster. If you only have one Vertica Database Server, enter its hostname, FQDN (recommended) or IP v4 address. vertica_db_admin_user vertica_db_admin_password Enter the username and password of the Vertica database administrator, with privileges to create schemas and prepare the database. vertica_cluster_ips Enter a comma-separated list of all of the hostnames or FQDNs (recommended) or IP v4 addresses of all of the Vertica Database Server nodes in the cluster, for example, vertica_db1.mydomain.com,vertica_db2.mydomain.com,vertica_db3.mydomain.com. If you only have one Vertica Database Server, enter its name or address. etl_user Enter the name of the user with permissions to run the ETL Server. This user must have permissions to run scheduled (crontab) tasks. Typically, you would not run the Aternity ETL Server as a root user, which is the default if you do not specify another user. etl_user_group_name Enter the group of the etl_user. The setup notifies you when it completes successfully. Your setup of the ETL Server completed successfully The setup writes a log file, installer.log to the ETL destination folder. Parent topic Install Aternity on-premise 10Related tasksSet up Trial POC for Aternity on-premise 10 with VMWare Images (OVA files)Express Setup for the Oracle Database Server for Aternity 10Set Up a New Vertica Database Server for Aternity 10Set Up a New Dashboard Server for Aternity 10Set Up a New Dashboard Gateway for Aternity 10Set Up a New Management Server for Aternity 10Publish or Upload Dashboard Layouts for Aternity 10Set Up a New Data Warehouse Server on a Dedicated ComputerSet Up a New Aggregation Server on a Dedicated ComputerSet Up a New Aternity REST API ServerSet Up a New Aternity Data Source for Portal