Key Task: Validate a Configuration Change

Validate the expense of an upgrade or migration with the Validate Configuration Change dashboard, which measures the impact by comparing performance before and after the upgrade. You can monitor upgrades for hardware (like moving users to SSD drives, or moving to laptops) or system changes (like migrating to Windows 10). Validate Change provides a unified view of the impact of your change on both hardware performance AND software performance.

Tip

The other validate dashboards show performance of activities only for managed applications around a change in a narrow transition time. In contrast, Validate Change can monitor many kinds of changes, including migration projects for any group of users on any application (not just managed applications) and for hardware or system health events.

Validate Configuration Change dashboard

Compare the best and worst effects of your change in the BEST | WORST sections. It shows the best performers on the left side and the worst performers on the right side.

Compare best and worst effects after the change

You can also manage your own custom-defined group of devices, comparing those with an upgraded a virus-checker, by defining a custom pilot group which covers all the devices you intend to upgrade.

Before you begin

Contact the Aternity Beta Team to enable this dashboard for your account.

To assess the effects of your rollout or migration, you must first plan and set expectations:

  • Define your expectations of the change, like faster launch times of applications, or fewer system crashes.

  • Perform only one change at a time, and then compare the results with your expectations. For example, do not upgrade OS versions and disk types at the same time, so that you can better understand any resulting change in performance.

  • (Optional) If you plan to change only some devices, define the pool of devices which you want to update, and the sub-groups within that pool which you intend to upgrade one at a time (like per department per location).

    For example, if you upgrade all of the hard disk drives (HDDs) to solid state drives (SSDs) on Lenovo T530s running Windows 10, split that device pool into groups, and upgrade one group at a time, like one location at a time, or another attribute that fits your planning. A phased approach enables you to compare the performance of upgraded groups of devices to those who not yet upgraded.

    Define pool of devices for migration, then upgrade one group at a time
    Note

    Choose devices with similar performance and specifications to help you obtain accurate results. For example, don't compare an HDD to SSD upgrade if one set is quad-core and the other is dual-core.

Procedure

  1. Step 1 Open a browser and sign in to Aternity.
  2. Step 2 Select Main Menu > Validate > Configuration Change or VDI Migration.

    Use Validate VDI Migration to validate a migration to virtual desktops. Your migration can be from regular desktops to VDI or from virtual applications to VDI.

    Access Validate Configuration Change
  3. Step 3 Define the change that the dashboard tracks.
    Tip

    If you are validating VDI Migration, Aternity automatically sets the dashboard parameters to compare the performance of physical device types with virtual desktops.

    Describe the change you want to validate
    Field Description
    Type of Change

    Select the type of configuration change you made from the drop-down menu.

    • OS Version: Displays the full name, the exact version number, and the service pack version of the operating system. In Windows 10, it includes the release ID (like Microsoft Windows 10 Enterprise 1507). Use this to differentiate between details of the same operating system. For example, it lists MS Windows Server 2008 R2 Enterprise SP 1.0 separately from MS Windows Server 2008 R2 Enterprise SP 2.0.

    • OS Name: Displays the generic name and version of the operating system (like MS Windows 10, MS Windows Server 2008 R2, MacOS 10.3, iOS 10 or Android 6).

    • Pilot Group: A pilot group is a custom set of users or devices which undergo a change, like migrating to Windows 10, or updating the type of hard disk to SSD.

    • Memory Size: Displays the size of physical RAM of the device.

    • CPU Cores: Displays the number of CPU cores of the device.

    • CPU Frequency: (Windows, Macs only) Displays the speed of the CPU processors of the device.

    • Image Build Number: Displays the ID of the disk image used when creating this device's initial setup and configuration. Only displays if you defined a custom attribute using this predefined name.

    • Model: Displays the name and the model number of the device, like iPhone 6s, GalaxyTab8.

    • Manufacturer: Displays the name of the vendor which created this device, like Samsung, Apple, Dell, Lenovo, and so on.

    • OS Architecture: Displays the type of device reporting performance to Aternity.

    • Device Type: Displays the type of device reporting performance to Aternity.

    • OS Disk Types: (Windows only, Agent 9.0.3 or later) Displays the type of hard disk containing the operating system. Possible values are:

      • HDD for a traditional spinning hard disk drive

      • SSD for a solid state drive

      • Virtual if this is not a physical device.

    Change to

    Select the target of your migration or upgrade. For example, if you selected Type of Change > Memory Size select the new memory size here.

    When did the change start

    Enter the date on which you made the change, to define the timeframe that you want to analyze.

    When you enter the dashboard, it uses one of the preset timeframes which loosely matches the same number of days before and after the date of change.

  4. Step 4 For a quick view of the changes in performance between those before or after the upgrade, see the Summary Bar.

    A green bar to the right indicates improvement, while those in red to the left worsened.

    View a summary of the change effects
    Field Description
    UXI

    Displays the average UXI for all the devices displayed in the dashboard.

    Activity Response

    Displays the average activity response time for all the devices displayed in the dashboard.

    Boot Time

    Displays the average boot time for all of the devices displayed in the dashboard.

    System & Hardware Health

    Displays the total of all the health events displayed in the dashboard (hardware and system) divided by the number of hours in the timeframe.

    Remote Display Latency

    Displays the average remote display latency for all of the devices displayed in the dashboard.

  5. Step 5 See the applications most impacted by the upgrade in the Applications (BEST | WORST) section: those which slowed down the most on the left, vs improved most on the right.View the differences in performance of applications before and after the change,

    For example, see performance by each application's activity response times.

    Tip

    A drop in value is not always worse. It could represent an improvement.

    See apps impacted the most by your migration
    Field Description
    Application type

    Displays the type of application: Windows desktop (), web application () which you run in a web browser, or monitored mobile app (), which has been integrated with Aternity monitoring.

    Name

    Displays the name of the monitored application, as it appears throughout the system. You can customize it when you add it as a managed application.

    From

    Displays the average performance before the change, according to your selection in the Measure menu.

    To

    Displays the average performance after the change.

    Change

    Displays the value of the measurement's change, according to your selection in the Measure menu.

    For example, if the overall activity response time of SAP increased from 3.68s in Windows 7 to 4.39s in Windows 10, it is a 16% increase in response time, which is a decrease in performance.

    View alternative measurements of application performance from the Measure drop-down list.

    Select the measurement to compare
    Field Description
    UXI

    The User Experience Index (UXI) is a value (0-5) which measures the overall performance and health of applications, based on the number of crashes per hour out of the total usage time, the percentage hang time out of the total usage time, and the percentage wait time out of the total usage time. For web applications, it also uses the percentage of web page errors out of all page loads, and the average page load time.

    Activity Response

    (For managed applications only) An activity response is the time taken for an application to complete an activity in seconds.

    Performance Index

    The performance index is a value (0-5) which measures an application's responsiveness. If users must wait frequently or for long periods for an application to respond, its performance index is lower. It is calculated from the usage time and wait time.

    Wait Percent

    Displays the percentage wait time out of the total usage time.

    Web Page Load Time

    The web page load time is the time required for a web page to load and finish rendering in a browser, from sending a URL request to when the page's events finish loading and it has a status of Completed. This measurement does NOT include the time to load additional page elements which occur after the main page has loaded, such as iframes that are embedded separate web pages, AJAX calls after the page is complete, or bookmarks with # in the URL). It does include AJAX calls that the page makes before it is complete.

    Health Index

    The health index is a value (0-5) which measures the time an application hangs, crashes or (for web applications) experiences web errors. If users experience frequent or severe crashes in the application, this index is lower.

    Crash Rate (Per Hour)

    The crash rate of an application is the average number of crashes which occurred in that application during an hour of active usage. It is calculated as the total number of crashes divided by the total usage time in hours.

    Hang Percent

    Displays the percentage of hang time of out the total usage time. This is one of the elements used when calculating the UXI.

    Web Page Error Rate

    The web page error rate is the percentage of errors out of all web page loads which cause the page not to load (HTTP error 40x or 50x).

    Hover over a measurement in a BEST | WORST section to view its details.

    View the details of an application's change in performance
    Field Description
    Usage Time

    Displays the usage time before and after the change, during the timeframe of the dashboard.

    Total Activities

    (For managed applications only) Displays the number of times someone performed an activity in this application, thereby adding impact to its usage. If the same user performs the same activity twice, it counts as two. The icon (if present) represents the overall status, derived from the score.

    For further information, you can drill down to:

  6. Step 6 See the devices impacted most by the upgrade in the System and Hardware Health section.

    See those worst hit by the change (the biggest jump in crashes or other system or hardware health events) on the left, versus those whose stability improved (the biggest drop in crashes or system errors).

    For example, check if your disk drive upgrade resulted in fewer low disk space events, if an OS upgrade caused more memory allocation failures, or if your updated application resolved problems of low disk space. For more information on health events, see the Device Health dashboard.

    View the devices impacted most by the change
    Field Description
    Category

    The type of health event:

    Event

    Displays the name of the health event which occurred.

    From

    Displays the number of health events on average which occurred per hour of active usage before the change, during the dashboard's timeframe, for the pool of devices undergoing the change.

    To

    Displays the number of health events on average which occurred per hour of active usage after the change, during the dashboard's timeframe, for the pool of devices undergoing the change.

    Change

    Displays the number of health events on average which occurred per hour of active usage as a percentage.

    Fewer health events represents better performance, so a drop in events is colored green, while a rise in events is colored red. For example, if disk failures drop from 0.25 to 0.15 events per hour of usage, performance rose by 40%.

    Hover your mouse over a measurement for details.

    View the details of the health impact of your change
    Field Description
    Health Events per Hour of Use

    Displays the number of health events on average which occurred per hour of active usage.

    Health Events

    Displays the total number of health events before and after the change, during the dashboard's timeframe, for the pool of devices undergoing the change.

    Hours of Use

    Displays the total usage time of the selected applications (in the Applications section in the side bar) during the dashboard's timeframe.

  7. Step 7 Narrow the displayed pool of devices by selecting attribute names and their values in the Filters section.

    The filters combine (using AND) to narrow the pool further. For example, if you upgrade all of the hard disk drives (HDDs) to solid state drives (SSDs) on Lenovo T530s running Windows 10, you can limit the dashboard to display only these models, by selecting Model as Lenovo T530 and OS Type as Win 10.

    Important

    Although the dashboard reacts dynamically by updating the data to reflect your filter selections, select Apply to implement ALL of the selections you make.

    Define pool of devices

    You can create your own pilot groups if none of the attributes define the group undergoing the change.

    Tip

    Be careful not to filter out the main attribute from the dashboard display. For example, if you are migrating to Windows 10, if you filter out Windows 10 devices, the dashboard would be empty.

  8. Step 8 In the Applications section, limit the display to show Application Types, such as desktop applications, or choose specific applications from the Application drop-down.

    For example, if you install SSDs to increase the performance of a largely used application, select that application from the Application drop-down list to single out its measurements. If the application's performance index or UXI have not improved enough, you may consider also upgrading the RAM on those computers to achieve the performance improvement.

    Select specific applications to assess their performance change
    Field Description
    Application Type

    Select the types of applications whose performance you want to compare. The options are Desktop, Mobile, Web, and All.

    Applications

    Select one or several applications from the Application list. It includes all of the used managed and discovered applications.

  9. Step 9 Check the impact of ANY attribute in the lower section of the dashboard, to validate any theory where you expected the biggest impact resulting from the change.

    Select the attribute which you expected to improve with the change, and the measurement you want to check. Learn more about the attributes collected by Aternity

    For example, if you upgraded to SSDs in one location to improve the performance of applications, view the section by Business Locations, and select the Applications Performance Index. Alternatively, you can check your performance by device model, and many more device attributes, to verify the upgrade had the desired effect.

    Tip

    If you see a blank value in the From or the To columns, it could mean, for example, that no devices in that specific breakdown received the change, or that nobody used the targeted applications in the dashboard's timeframe.

    View the breakdown of the change of the measured parameter

    The measurements you can compare before and after the change are:

    Field Description
    Applications UXI

    The User Experience Index (UXI) is a value (0-5) which measures the overall performance and health of applications, based on the number of crashes per hour out of the total usage time, the percentage hang time out of the total usage time, and the percentage wait time out of the total usage time. For web applications, it also uses the percentage of web page errors out of all page loads, and the average page load time.

    Applications Activity Response

    An activity response is the time taken for an application to complete an activity in seconds.

    Applications Performance Index

    The performance index is a value (0-5) which measures an application's responsiveness. If users must wait frequently or for long periods for an application to respond, its performance index is lower. It is calculated from the usage time and wait time.

    Applications Wait Percent

    Displays the percentage wait time out of the total usage time.

    Applications Web Page Load Time

    The web page load time is the time required for a web page to load and finish rendering in a browser, from sending a URL request to when the page's events finish loading and it has a status of Completed. This measurement does NOT include the time to load additional page elements which occur after the main page has loaded, such as iframes that are embedded separate web pages, AJAX calls after the page is complete, or bookmarks with # in the URL). It does include AJAX calls that the page makes before it is complete.

    Applications Health Index

    The health index is a value (0-5) which measures the time an application hangs, crashes or (for web applications) experiences web errors. If users experience frequent or severe crashes in the application, this index is lower.

    Applications Crash Rate

    The crash rate of an application is the average number of crashes which occurred in that application during an hour of active usage. It is calculated as the total number of crashes divided by the total usage time in hours.

    Applications Hang Percent

    Displays the percentage of hang time of out the total usage time. This is one of the elements used when calculating the UXI.

    Applications Web Page Error Rate

    The web page error rate is the percentage of errors out of all web page loads which cause the page not to load (HTTP error 40x or 50x).

    Remote Display Latency

    Displays the overall latency for virtual sessions (or lag) in seconds for each type of virtualized environment (virtual desktops and virtual applications), gathered from latency measurements taken every minute. It lists the following protocols (if present in your organization):

    • ICA is the protocol used by Citrix XenApp.

    • RDP is the protocol used by Microsoft products like Remote Desktop Connection. It is also increasingly being adopted by newer versions of Citrix XenApp.

    • PCoIP is the protocol used by VMWare's VDI systems.

    System / Hardware / System and Hardware Health Events per Hour of Use

    Displays the number of health events on average which occurred per hour of active usage.

    Boot Duration

    Displays the average boot time for the pool of devices during the selected timeframe.

  10. Step 10 Save the dashboard to return to it for the duration of your change implementation, and so that you can share its data with others.

    For more information, see Save and Share a Dashboard for Later Viewing.

    Save this migration to return and track progress throughout the project
  11. Step 11 You can limit the display of this dashboard using the menus at the top of the window.
    Change or limit the display in the top menu drop-down lists
    Field Description
    Timeframe

    You can change the start time of the data displayed in this dashboard in the Timeframe menu in the top right corner of the dashboard.

    The default timeframe presented in the dashboard begins a few days before the change date you provide, and extends to the day you view the dashboard.

    You can access data in this dashboard (retention) going back up to 90 days. This dashboard's data refreshes every hour.

    Measure

    Select to view the absolute value or the percentage of the change. Most often, you need both.

    For example, if you change the network infrastructure and expect a two second improvement in response times, the percentage improvement would vary massively depending on the previous measurement. It would be a massive 66% if it used to be three seconds, or just 16% if it used to be 12 seconds. So in this case, you may prefer to see the absolute value.

    In contrast, a different change would yield a proportional improvement across the board, and may therefore be better to see the percentage improvement.

    Usage Level

    Select to limit or widen the number of applications displayed in this dashboard:

    • High (default) displays only the top 10% of applications with the most usage time, or if that is less than 30 applications, it displays the 30 most used applications.

    • High and Medium displays only the top 20% of applications with the most usage time, or if that is less than 60 applications, it displays the 60 most used applications.

    • Managed Applications displays only managed applications in this dashboard.

    • All Applications displays the full list of applications in this dashboard.