Perform a Failover for Aternity 11.0.3 (Disaster Recovery)

You can switch to a standby site in case of disaster, when your primary site becomes unavailable.

Also, it is possible to switch to a standby site if you want to test your deployment readiness.

The below procedure refers to both scenarios and we mention what step is required in each scenario.

CAUTION

To test your deployment readiness, we recommend using a separate testing environment. Please note that there is no rollback once you switch your production environment to a standby. Once switched, your standby environment becomes your primary production site.

Before you begin

  • Make sure you have a FULLY IDENTICAL hot standby site of Aternity on-premise 11.0.3. The hot standby Vertica cluster must be IDENTICAL to the primary cluster (NUMBER OF NODES, STORAGE CAPACITY, VERTICA VERSION, DBADMIN USER, and DBADMIN PASSWORD). Learn more

    You must start with a fully functioning deployment of Aternity on-premise 11.0.3.

  • Make sure you have the password you defined for the GR schema during the primary Aternity Management Server installation (learn more). You will need it to perform a failover.

  • You must validate from time to time that Aternity hot standby site properly functions by adding end user devices and starting all components.

  • Your DBA should be available to make changes to the Oracle Database Server.

  • Create a backup of the advanced dashboards you saved locally. Learn more.

  • Make yourself familiar with the script parameters.

  • It is recommended to use domain names (DNS) instead of IP addresses.

Procedure

  1. Step 1 (For a test failover) On the primary Vertica Database Server, disable the replication process:

    On the Node 1, as dbadmin user, execute the Python vertica_dr.py script. Enter the relevant parameters instead of default values (see the table below) and run the script:

    $ /opt/vertica/oss/python/bin/python /<dbadmin_home_folder>/vertica_dr.py --task disable --dbadmin_user dbadmin --dbadmin_password admin --database_name aternity

    Field Description

    <dbadmin_home_folder>

    Enter the path to the home folder of a DBA user. The default folder is home/dbadmin

    dbadmin user

    The default name in the sample is dbadmin. Enter the name as defined in your deployment.

    This user name is for both Linux and Vertica databases.
    dbadmin_password

    The default password in the sample isadmin. Enter the password as defined in your deployment.

    This is Vertica Database Server password, not Linux.

    database_name

    The default name in the sample is aternity. Enter the name as defined in your deployment.

    Note that the name is case sensitive and should be a lower case! It must begin with an alphabetic character, and can contain letters (lower case), numbers and underscores, and must be up to 30 characters.

  2. Step 2 (For a test failover) Ensure the replication process is disabled:

    Login to Aternity and open the System Health dashboard to check the status of the replication. Hover your mouse over the Vertica Replicator component whose status you check and see the details in the tooltip.

  3. Step 3 (For a test failover) Ensure the primary production site is shut down (shut down all processes of the server before continuing).
    1. a (For a test failover) Stop the AternityPlatform service on these servers, in this order:
      1. Management Server

      2. Aggregation Server

      3. Data Warehouse Server

      To stop the services, you must sign in to each computer as an administrator, and access the Windows Services screen.

    2. b (For a test failover) Stop the Aternity Docker Components Server.

      Log in to the Aternity Docker Components Server as a user configured to run Docker Components as a non-root user. Learn more.

      In the aternity-docker-admin home directory, run the command ./aternity-docker-admin stop. This stops all Docker components.

  4. Step 4 On a standby Aternity Docker Components Server, from the docker installation folder run:
    ./aternity-docker-admin prepare-dr-switch
  5. Step 5 (For a test failover) Stop the AternityPlatform services on the standby Management Server, Aggregation Server, and Data Warehouse Server.
    Ensure that Aternity Dashboard Server, Dashboard Gateway, Vertica Database Server, and Oracle Database Server are up and running.
  6. Step 6 (DBAs only) Disable the replication process on the primary Oracle Database Server.
  7. Step 7 (DBAs only) Change the standby Oracle Database Server READONLY mode to a regular mode.
  8. Step 8 On the standby Vertica Database Server Node 1, as dbadmin user, execute:
    $ /opt/vertica/oss/python/bin/python /<dbadmin_home_folder>/change_vertica_schema_owner.py --dbadmin_user dbadmin --dbadmin_password admin --database_name aternity --aternity_user '<aternity_user>' --aternity_user_password '<aternity_user_password>'

    First of all, enter the relevant parameters instead of default values. See the table below.

    The script resides in the home folder of a DBA user.

    Field Description

    <dbadmin_home_folder>

    Enter the path to the home folder of a DBA user. The default folder is home/dbadmin

    dbadmin user

    The default name in the sample is dbadmin. Enter the name as defined in your deployment.

    This user name is for both Linux and Vertica databases.
    dbadmin_password

    The default password in the sample isadmin. Enter the password as defined in your deployment.

    This is Vertica Database Server password, not Linux.

    database_name

    The default name in the sample is aternity. Enter the name as defined in your deployment.

    Note that the name is case sensitive and should be a lower case! It must begin with an alphabetic character, and can contain letters (lower case), numbers and underscores, and must be up to 30 characters.

    aternity_user

    During setup of the Aternity Management Server you defined the name of the schema. Enter that username in the command.

    aternity_password

    During setup of the Aternity Management Server you defined the username and password for the schema. Enter that password in the command.

    Note

    Execute this command for both schemas (the primary aternity_schema called aternity and the hot standby aternity_schema called aternity_dr). Each time enter the corresponding input parameters (username and password) of a respective schema.

  9. Step 9 On the standby Management Server, open the Command Prompt as administrator and browse to the installation directory (for example, D:\data\aternity).
  10. Step 10 Run switchDR.bat after defining the parameters as explained in the below table:
    switchDR.bat jdbc:oracle:thin:@//<standby_oracle_address>:<oracle_listener_port>/<aternity_service_name> <GR_schema_password> <standby_vertica_node_1_IP>
    Field Description
    standby_oracle_address

    Enter the host name of the computer where the STANDBY Oracle is installed.

    oracle_listener_port

    Enter the port. The default port is 1521.

    aternity_service_name

    Enter the Oracle database Service Name.

    GR_schema_password

    Enter the password you defined for the GR schema during the PRIMARY Aternity Management Server installation

    standby_vertica_node_1_IP

    Enter a STANDBY Vertica Database Server Node 1 IP address

    For example,
    switchDR.bat jdbc:oracle:thin:@//dr-oracle.mycompany.com:1521/aternity prodGRuser 10.50.33.4
  11. Step 11 Start the AternityPlatform services on the standby Management Server, Aggregation Server, and Data Warehouse Server that now serve as the primary.
  12. Step 12 Update the DNS names of all servers in a hot standby site so that the devices that previously connected to the failed primary site, will be able to reach the new standby server that now serves you as the primary.
  13. Step 13 On the standby Docker Components server that should now serve you as the primary, edit the properties.ini file with the details of the original primary site that failed:
    Field Description

    management_url in the [General] section.

    Enter the original primary Aternity Management Server hostname or FQDN (recommended) or IP v4 address, including protocol prefix. For example, https://aternity.mycompany.com.

    external_url in the [portal_ds] section.

    Enter the external URL to the original primary Aternity Data Source for Portal if a load balancer or proxy are to be used, and user requests will be made to an address other than that of the Docker Components host.

    If no value is given, then the FQDN will be used. If no value was given for the FQDN either, then the hostname will be used.

    external_url in the [rest_api] section.

    Enter the external URL to the original primary Aternity REST API Server if a load balancer or proxy are to be used, and user requests will be made to an address other than that of the Docker Components host.

    If no value is given, then the FQDN will be used. If no value was given for the FQDN either, then the hostname will be used.

  14. Step 14 On the standby Docker Components server that should now serve you as the primary, browse to the installation folder and run the following command:
    ./aternity-docker-admin reconfigure
    Now the standby site starts functioning instead of the failed one.