Benchmarking Employee Experience

Use the Aternity Industry Benchmark dashboards to compare user experience and device health in your organization with other Aternity customers and their monitored devices.

Aternity monitors over 500 customers globally and millions of end points are reporting to Aternity SaaS platform. The Benchmark feature allows comparing the performance of users’ devices and applications to other Aternity customers and understanding the overall status of your IT. Benchmarking considers the geographical location, the industry and the size of the company and provides different views of the data. Aternity is releasing now the first version of the feature and planning to add more capabilities in future releases allowing to drill down into specific application and into device's hardware.

For example, you can improve your overall score by resolving boot issues and MS Office performance.
See the distribution of various Aternity customers as of device health and application performance
See about Aternity DEM-QTM on our YouTube channel:
View Benchmarking Employee Experience with Aternity DEM-QTM on YouTube

This article covers the following sections:

Main Features and Capabilities

  • 9 available KPIs (Health and Performance)

  • Hints and Scores on DEM-Q

  • Ability to alter KPIs to customize dashboards

  • Filter data by different parameters, including industry, company size, geographical regions, device type, manufacturer, and more

Defining Company Size and Industry to Ensure a Successful Benchmark

To... Do This...

Allow a successful benchmark by assigning your Industry and/or Company size

  1. Sign in to Aternity.

  2. Hover over the panel, click the gear icon , and select My Account.

  3. Select your industry and the amount of employees in your organization.

    Now you can see in the Benchmark dashboard the level of device health and user experience in your organization compared to other companies in this industry. Learn more.
    Define the industry and size of your company to get a precise benchmark
Tip

You are not limited to compare your business only within the same industry or size segment. Regardless of how you defined your company in the My Account screen, you can still compare your own business to business processes and performance of other Aternity customers if you change filters at the top of the DEM-Q screen. This way if you are performing in different business fields or have many subsidiaries you still see where you stand compared to others.

Available Operations in the DEM-Q tab

To... Do This...

Locate your company on the graph

A small red square between all other dots on the graph is your company.
See the scores for your company

Each dot represents a company/customer account that Aternity monitors.

Note

At least three companies of the same industry are necessary for benchmarking. If less, it is not enough for comparison and the graph will not present any data.

The tooltip shows company industry and size, if defined in the My Account administration screen. If not, the tooltip shows Coming Soon.

Discover where you stand compared to other Aternity customers

In the DEM-Q tab, see the scores at the right side of the screen (on gauge widgets).

When data shows on the dashboard, the light bulbs change to the gauge widgets that show your company-specific scores.

Understand the Did You Know? light bulbs

In the DEM-Q tab, see the scores at the right side of the screen. In addition, every time when the console refreshes, see one Did you know? bulb that brings interesting general facts about industry and provides essential facts about computers, software or browsers. Randomly, upon every refresh another light bulb will show.

Understand the colors of your scores and where you are compared to others

Here is the legend for colors of your gauge widgets:
  • Green means you are within the top 70% in industry

  • Yellow means you match 15-30% in industry

  • Red means you are on the low side within 15% in industry

Learn where you stand compared to other companies

Select the Health KPI for comparison

Go to Industry Benchmarks Analyze and select the required health KPI for deeper analysis.
Select Health KPI to see on the graph
To see specific measures, hover over a dot on the graph and see the values for the selected KPI.
View the exact value for each KPI

Compare your company Performance KPI to other companies

Go to Industry Benchmarks Analyze and select the required performance KPI for deeper analysis.
Select performance KPI for the graph

Compare your company with companies in all regions or in a certain region only

In the DEM-Q tab, select the required Region at the top right side of the screen.
Select certain region or all regions to show on the comparison graph

Compare your company with companies in the same industry

In the DEM-Q tab, select the required Industry at the top right side of the screen.
Select your industry or all industries to show on the comparison graph

Compare your company with the companies of the same size

In the DEM-Q tab, select the Company Size at the top right side of the screen.
Select your company size or all sizes to show on the comparison graph

Drill-down to Aternity dashboards

Drill-down from the Benchmark dashboards to your specific dashboards for further analysis by hovering over Your Company and selecting in the tooltip the required dashboard's link (Analyze Applications or Device Health).

Select a dashboard in your account to view the details

Available Operations in the Analyze tab

To... Do This...

Analyze the selected KPIs and compare between required measurements

Go to the Analyze tab and select what Health KPI and Performance KPI to display on the graph. For example, you can see that your apps response time is very high and can be improved, or that crash frequency is low enough which is good.

See the definition for each KPI further in this article.

View the selected health and performance KPIs for devices in your company by any available device attribute

Go to Analyze and compare the KPIs By any available in the list attribute. The colors are randomly assigned to groups and allow a convenient visualization and ease of analysis.

For example, compare the hours between crashes for different manufactures and also see where your company compared to others.

Another example, visualize activity response time in different regions where different colors present different regions.

By default, when you open the dashboard for the first time, it shows the data by company. Once you select a different attribute in the By breakdown, the graph and the legend will change accordingly.

Compare Hours Between Crashes and Wait Time for different device types

Select an attribute to visualize on the graph

Go to the Analyze tab and select an attribute in the By breakdown.

Field Description
Device Type
  • Desktops are monitored Windows devices without a fitted battery, or for Macs, any monitored MacBook running macOS or OS X.

  • Laptops are Windows devices with a battery and a built-in keyboard (including all Windows hybrid tablet/laptop models), or for Macs, any monitored laptop running macOS or OS X.

  • Remote Devices have applications accessed remotely via an RDP protocol, for example, with Microsoft's Remote Desktop Connection.

  • Smartphones run monitored mobile apps on a small touch screen within a mobile operating system environment.

  • Tablets have larger touch screens, and no built-in keyboard, running iOS or Android. If it runs Windows, it is defined as a tablet if it is a known model of a Windows pure tablet (like Microsoft Surface models).

  • Virtual App Servers offer multiple users access to a single instance of an application, for example, with Citrix XenApp.

  • Virtual Desktops offer the ability to run an application within a VDI environment, which is a virtual instance of the entire desktop operating system (usually Windows).

Device Connection Location

This attribute is useful when analyzing remote workforce: when working remotely from home, using VPN (intranet), being completely off-site or logging in remotely to a VDI or a physical device left in the office.

Displays the reported location of the monitored device. The possible values are: Remote (VPN On), Remote (VPN Off), Remote (Non-Corporate VPN), In-Office

User connection location and device connection location may be different when working remotely.

User Connection Location Displays the reported location of where the USER is.
  • In Office: The user is connected to the Microsoft Active Directory, or the subnet is mapped to a valid location using the Subnet2Site file, and no VPN is detected.

  • NA: Aternity cannot detect and report the site to which the user belongs.

  • Remote (Non Corporate VPN) / Remote (VPN On): The user is not in the office, but connected to a VPN.

  • Remote (VPN Off): The user is not in the office and not connected to a VPN.

Manufacturer Displays the name of the vendor which created this device, like Samsung, Apple, Dell, Lenovo, and so on.
Industry

Groups organizations by industry.

OS Name

Displays the generic name and version of the operating system (like MS Windows 10, MS Windows Server 2008 R2, MacOS 10.3, iOS 10 or Android 6).

Company Size

Groups organizations by size (the number of employees per company).

Region

Displays the region of the current location of the device.

View KPIs for different device attributes.

  1. Go to the Analyze tab.

  2. Verify that the required KPIs are already selected.

  3. In the right pane, filter the drop-down menu for each list separately and select the device attribute you want to analyze (three sections allow viewing three attributes in parallel).

The definitions for these attributes are listed in the row above.

For example, analyze the hang percent in different regions, activity response time for different manufacturers or operating systems, or check performance KPIs for different device types, and more. There are many cases for health and performance evaluation that might help you make critical business decisions.
Select what data you wish to see on the widget

Colors Legend: Dark green is the best value and red is the worst. The dashboard automatically adjusts gradient colors for the rest (light green, yellow, orange).

Benchmark KPIs - Health KPIs

Health KPI measures the percentage and averages of errors, hangs, and crashes from all monitored devices.

The graph allows you to compare your Health KPI with other Aternity customers.

Field Description
Stability Index

The stability index (used to be reliability value) is a Windows score (from 1 to 10) of a PC's overall stability (search in Windows for the Windows Reliability Monitor). As the number and severity of errors increases, it lowers the stability index. Aternity displays the average for the previous day, or, if unavailable, it shows the most recent daily average. The server versions of Windows do not have this measurement, and therefore would not report it to Aternity.

Hours Between Crashes

The Hours Between Crashes is the number of hours that passes on average between crashes which occurred in an application during the total usage time.

Aternity monitors the performance of all applications which are automatically discovered. These include managed applications which have their own defined activities.

Hang Percent

This KPI measures the percentage hang time out of the total wait time for each customer.

Hang time measures the time when an application is listed as Not responding in the Windows Task Manager while it is in the foreground (in use). This measurement is used to calculate the wait time of an application, and the overall UXI.

Hours Between BSODs

The Hours Between BSOD is the number of application usage hours that pass on average between BSOD errors which happened during the total usage time. BSOD is a stop error, commonly called the blue screen of death. The Hours Between BSOD are calculated as total usage time divided by number of BSODs.

UXI

The User Experience Index (UXI) is a value (0-5) which measures the overall performance and health of applications, based on the number of crashes per hour out of the total usage time, the percentage hang time out of the total usage time, and the percentage wait time out of the total usage time. For web applications, it also uses the percentage of web page errors out of all page loads, and the average page load time.

Learn more.

Benchmark KPIs - Performance KPIs

Performance KPI measures the percentage and averages of response time, boot time, wait time, and load time from all monitored applications.

The graph allows you to compare your Performance KPI with other Aternity customers. To see specific measures, hover over a dot on the graph and see the values for the selected KPI.

Field Description
Wait Percent

This KPI measures the percentage wait time out of the usage time.

The wait time of a Windows application is defined as the time users spend waiting for the application to respond when it is actively running and in use (part of the usage time).

The total wait time is calculated as the time covered by the following components (which may overlap): the hang time when an application is not responding, or when the mouse pointer has a busy icon (Windows devices). For web applications, the wait time is the web page load time when both the browser window and its tab are in the foreground.

For web applications, the wait time is the web page load time when both the browser window and its tab are in the foreground.

Activity Response

An activity response is the time taken for an application to complete an activity in seconds. The time also has a severity status based on its expected time (measured using baselines or manual or SLA thresholds). For example, if the time required to launch an application is much slower than recent baselined measurements, it might be classed as Major.

An activity is a user action which you monitor for performance, like a mouse click or a key press, to measure the time until the app's GUI responds, known as the activity response time. In Aternity, you can compare response times in any app across the enterprise, and troubleshoot performance by seeing when they perform slower. For example, you can monitor the launch of an application, or the time it takes for an application to respond to a menu choice.

In Aternity, you can compare response times in any app across the enterprise, and troubleshoot performance by seeing when they perform slower.

Average Boot Time

This KPI measures the Average Boot Time which is the average of the Total Boot Time.

The total boot time on a Windows device starts from the time the Windows logo appears until the desktop appears and all components are loaded.

Web Page Load Time

This KPI measures the average of the web page load time from all web pages load time.

The web page load time is the time required for a web page to load and finish rendering in a browser, from sending a URL request to when the page's events finish loading and it has a status of Completed. This measurement does NOT include the time to load additional page elements which occur after the main page has loaded, such as iframes that are embedded separate web pages, AJAX calls after the page is complete, or bookmarks with # in the URL). It does include AJAX calls that the page makes before it is complete.

UXI

The User Experience Index (UXI) is a value (0-5) which measures the overall performance and health of applications, based on the number of crashes per hour out of the total usage time, the percentage hang time out of the total usage time, and the percentage wait time out of the total usage time. For web applications, it also uses the percentage of web page errors out of all page loads, and the average page load time.

Learn more.