Employee Experience Benchmarking

Use the Aternity Digital Experience Management Quadrant (DEM-Q®) to compare employee digital experience and device health in your organization to that of hundreds of companies and millions of devices managed by Aternity. The Benchmark dashboard shows KPI data, performance trends, and the scores of your organization compared to other Aternity customers.

Aternity monitors over 500 customers globally and millions of devices are reporting to Aternity SaaS platform. The Industry Benchmark dashboard allows understanding the overall status of your IT compared to other companies. Benchmarking considers the geographical location, the industry and the size of the company and provides different views of the data.

Use the DEM-Q® dashboard for benchmarking your company against others in the industry and the Analyze dashboard for decision making (what OS type to deploy, what device type to use, and more).

See the distribution of various Aternity customers as of device health and application performance
Here are some examples of ways companies can use benchmarking:
  • Whether working from home boosts or hurts productivity for companies around the world? If performance in your organization has worsened when working from home, view the trend for the rest of the world.

  • Your devices suffer from many BSOD events and you might be interested in replacing the manufacturer to a better one. See what devices other companies use and how these devices perform in general. If you discover a performance problem in your devices, there may be many reasons for that, like old processors, lack of memory, and more. You might find out that the only thing you should do to improve the performance of devices is, for example, to upgrade their processors. The Aternity DEM-Q® provides you the information you need to make a decision.

  • View and improve your overall score by resolving boot issues and MS Office performance.

  • Aternity DEM-Q® helps understanding where your company performance excels and where to focus efforts to improve it.

See about Aternity DEM-Q® and Analyze on our YouTube channel:
View Benchmarking Employee Experience with Aternity DEM-Q® and Analyze on YouTube

This article covers the following sections:

Main Features and Capabilities

  • 9 available KPIs (Health and Performance)

  • Hints and Scores on DEM-Q

  • Ability to alter KPIs to customize dashboards

  • Filter data by different parameters, including industry, company size, geographical regions, device type, manufacturer, and more

Defining Company Size and Industry to Ensure a Successful Benchmark

To... Do This...

Allow a successful benchmark by assigning your Industry and/or Company size

  1. Sign in to Aternity.

  2. Hover over the panel, click the gear icon , and select My Account.

  3. Scroll down to the bottom of the screen and select your industry and the amount of employees in your organization.

    Now, having selected the corresponding Industry and Company Size filter values above, you can see in the dashboard the level of device health and user experience in your organization compared to other similar-size companies in this industry. Learn more.
    Define the industry and size of your company to get a precise benchmark
Tip

You are not limited to compare your business only within the same industry or size segment. Regardless of how you defined your company in the My Account screen, you can still compare your own business to business processes and performance of other Aternity customers if you change filters at the top of the DEM-Q screen. This way if you are performing in different business fields or have many subsidiaries you still see where you stand compared to others.

Available Operations in the DEM-Q tab

To... Do This...

Locate your company on the graph

A small red square between all other dots on the graph is your company.
See the scores for your company

Each dot represents a company/customer account that Aternity monitors.

Note

At least three companies of the same industry are necessary for benchmarking. If less, it is not enough for comparison and the graph will not present any data.

The tooltip shows company industry and size, if defined in the My Account administration screen. If not, the tooltip shows Coming Soon.

Discover where you stand compared to other Aternity customers

In the DEM-Q tab, see the scores at the right side of the screen (on gauge widgets).

When data shows on the dashboard, the light bulbs change to the gauge widgets that show your company-specific scores.

Understand the Did You Know? light bulbs

In the DEM-Q tab, see the scores at the right side of the screen. In addition, every time when the console refreshes, see one Did you know? bulb that brings interesting general facts about industry and provides essential facts about computers, software or browsers. Randomly, upon every refresh another light bulb will show.

Understand the colors of your scores and where you are compared to others

Here is the legend for colors of your gauge widgets:
  • Green means you are within the top 70% in industry

  • Yellow means you match 15-30% in industry

  • Red means you are on the low side within 15% in industry

Learn where you stand compared to other companies

Use Timeframe to compare your organization to others before and after you have made significant changes.

Select the time range to limit the dashboard view to the period you want to investigate.

Important

When you select the time range for which you compare your business against others, note that there is a delay of up to 48 hours for the data to be fully synched. Each account has its own time zone. Daily aggregations are done at midnight, which is different for different time zones. Thus, it takes time to analyze all data from all accounts for a selected time period.

Select the Health KPI for comparison

Go to Industry Benchmarks Analyze and select the required health KPI for deeper analysis.
Select Health KPI to see on the graph
To see specific measures, hover over a dot on the graph and see the values for the selected KPI.
View the exact value for each KPI

Compare your company Performance KPI to other companies

Go to Industry Benchmarks Analyze and select the required performance KPI for deeper analysis.
Select performance KPI for the graph

Compare your company with companies in all regions or in a certain region only

In the DEM-Q tab, select the required Region at the top right side of the screen.
Select certain region or all regions to show on the comparison graph

Compare your company with companies in the same industry

In the DEM-Q tab, select the required Industry at the top right side of the screen.
Select your industry or all industries to show on the comparison graph

Compare your company with the companies of the same size

In the DEM-Q tab, select the Company Size at the top right side of the screen.
Select your company size or all sizes to show on the comparison graph

Drill-down to Aternity dashboards

Drill-down from the Benchmark dashboards to your specific dashboards for further analysis by hovering over Your Company and selecting in the tooltip the required dashboard's link (Analyze Applications or Device Health).

Select a dashboard in your account to view the details

Available Operations in the Analyze tab

To... Do This...

Analyze the selected KPIs and compare between required measurements

Go to the Analyze tab and select what Health KPI and Performance KPI to display on the graph. For example, you can see that your apps response time is very high and can be improved, or that crash frequency is low enough which is good.

See the definition for each KPI further in this article.

Use Timeframe to compare your organization to others before and after you have made significant changes.

Select the time range to limit the dashboard view to the period you want to investigate.

Important

When you select the time range for which you compare your business against others, note that there is a delay of up to 48 hours for the data to be fully synched. Each account has its own time zone. Daily aggregations are done at midnight, which is different for different time zones. Thus, it takes time to analyze all data from all accounts for a selected time period.

View the selected health and performance KPIs for devices in your company by any available device attribute

Go to Analyze and compare the KPIs By any available in the list attribute. Coloring values of the selected attribute makes it easy to visualize different groups of values. The colors are randomly assigned to groups and allow a convenient visualization and ease of analysis.

For example, compare the hours between crashes for different manufactures and see where your company located compared to others, if it performs worse or better compared to other companies.

Another example, visualize activity response time in different regions where different colors present different regions.

By default, when you open the dashboard for the first time, it shows the data by company. Once you select a different attribute in the By breakdown, the graph and the legend will change accordingly.

Compare Hours Between Crashes and Wait Time for different device types

Select an attribute to visualize on the graph

Go to the Analyze tab and select an attribute in the By breakdown.

Field Description
Device Type
  • Desktops are monitored Windows devices without a fitted battery, or for Macs, any monitored MacBook running macOS or OS X.

  • Laptops are Windows devices with a battery and a built-in keyboard (including all Windows hybrid tablet/laptop models), or for Macs, any monitored laptop running macOS or OS X.

  • Remote Devices have applications accessed remotely via an RDP protocol, for example, with Microsoft's Remote Desktop Connection.

  • Smartphones run monitored mobile apps on a small touch screen within a mobile operating system environment.

  • Tablets have larger touch screens, and no built-in keyboard, running iOS or Android. If it runs Windows, it is defined as a tablet if it is a known model of a Windows pure tablet (like Microsoft Surface models).

  • Virtual App Servers offer multiple users access to a single instance of an application, for example, with Citrix XenApp.

  • Virtual Desktops offer the ability to run an application within a VDI environment, which is a virtual instance of the entire desktop operating system (usually Windows).

Device Connection Location

Displays the reported location of the monitored device. The possible values are: Remote (VPN On), Remote (VPN Off), Remote (Non-Corporate VPN), In-Office.

For laptops, Device Connection Location is same as User Connection Location.

This attribute is useful when analyzing remote workforce: when working remotely from home, using VPN (intranet), being completely off-site or logging in remotely to a VDI or a physical device left in the office.

User connection location and device connection location may be different when working remotely.

User Connection Location Displays the reported location of where the USER is.
  • In Office: The user is connected to the Microsoft Active Directory, or the subnet is mapped to a valid location using the Subnet2Site file, and no VPN is detected.

  • N/A: Aternity cannot detect and report the site to which the user belongs.

  • Remote (Non-Corporate VPN) / Remote (VPN On): The user is not in the office, but connected to a VPN.

  • Remote (VPN Off): The user is not in the office and not connected to a VPN.

Manufacturer Displays the name of the vendor who created this device, like Samsung, Apple, Dell, Lenovo, and so on.
Industry

Groups organizations by industry.

OS Name

Displays the generic name and version of the operating system (like MS Windows 10, MS Windows Server 2008 R2, MacOS 10.3, or Android 6).

Company Size

Groups organizations by size (the number of employees per company).

Region

Displays the region of the current location of the device.

View KPIs for different device attributes.

  1. Go to the Analyze tab.

  2. Verify that the required KPIs are already selected.

  3. In the right pane, filter the drop-down menu for each list separately and select the device attribute you want to analyze (three sections allow viewing three attributes in parallel).

The definitions for these attributes are listed in the row above.

For example, analyze the hang percent in different regions, activity response time for different manufacturers or operating systems, or check performance KPIs for different device types, and more. There are many cases for health and performance evaluation that might help you make critical business decisions.
Select what data you wish to see on the widget

Colors Legend: Dark green is the best value and red is the worst. The dashboard automatically adjusts gradient colors for the rest (light green, yellow, orange).

Benchmark KPIs - Health KPIs

Health KPI measures the percentage and averages of errors, hangs, and crashes from all monitored devices.

The graph allows you to compare your Health KPI with other Aternity customers.

Field Description
Stability Index

The stability index (used to be reliability value) is a Windows score (from 1 to 10) of a PC's overall stability (search in Windows for the Windows Reliability Monitor). As the number and severity of errors increases, it lowers the stability index. Aternity displays the average for the previous day, or, if unavailable, it shows the most recent daily average. The server versions of Windows do not have this measurement, and therefore would not report it to Aternity.

Hours Between Crashes

The Hours Between Crashes is the number of hours that passes on average between crashes which occurred in an application during the total usage time.

Aternity monitors the performance of all applications which are automatically discovered. These include managed applications which have their own defined activities.

Hang Percent

This KPI measures the percentage hang time out of the total wait time for each customer.

Hang time measures the time when an application is listed as Not responding in the Windows Task Manager while it is in the foreground (in use). This measurement is used to calculate the wait time of an application, and the overall UXI.

Hours Between BSODs

The Hours Between BSOD is the number of application usage hours that pass on average between BSOD errors which happened during the total usage time. BSOD is a stop error, commonly called the blue screen of death. The Hours Between BSOD are calculated as total usage time divided by number of BSODs.

UXI

The User Experience Index (UXI) is a value (0-5) which measures the overall performance and health of applications, based on the number of crashes per hour out of the total usage time, the percentage hang time out of the total usage time, and the percentage wait time out of the total usage time. For web applications, it also uses the percentage of web page errors out of all page loads, and the average page load time.

Learn more.

Benchmark KPIs - Performance KPIs

Performance KPI measures the percentage and averages of response time, boot time, wait time, and load time from all monitored applications.

The graph allows you to compare your Performance KPI with other Aternity customers. To see specific measures, hover over a dot on the graph and see the values for the selected KPI.

Field Description
Wait Percent

This KPI measures the percentage wait time out of the usage time.

The wait time of a Windows application is defined as the time users spend waiting for the application to respond when it is actively running and in use (part of the usage time).

The total wait time is calculated as the time covered by the following components (which may overlap): the hang time when an application is not responding, or when the mouse pointer has a busy icon (Windows devices). For web applications, the wait time is the web page load time when both the browser window and its tab are in the foreground.

For web applications, the wait time is the web page load time when both the browser window and its tab are in the foreground.

Activity Response

An activity response is the time taken for an application to complete an activity in seconds. The time also has a severity status based on its expected time (measured using baselines or manual or SLA thresholds). For example, if the time required to launch an application is much slower than recent baselined measurements, it might be classed as Major.

An activity is a user action which you monitor for performance, like a mouse click or a key press, to measure the time until the app's GUI responds, known as the activity response time. In Aternity, you can compare response times in any app across the enterprise, and troubleshoot performance by seeing when they perform slower. For example, you can monitor the launch of an application, or the time it takes for an application to respond to a menu choice.

In Aternity, you can compare response times in any app across the enterprise, and troubleshoot performance by seeing when they perform slower.

Average Boot Time

This KPI measures the Average Boot Time which is the average of the Total Boot Time.

The total boot time on a Windows device starts from the time the Windows logo appears until the desktop appears and all components are loaded.

Web Page Load Time

This KPI measures the average of the web page load time from all web pages load time.

The web page load time is the time required for a web page to load and finish rendering in a browser, from sending a URL request to when the page's events finish loading and it has a status of Completed. This measurement does NOT include the time to load additional page elements which occur after the main page has loaded, such as iframes that are embedded separate web pages, AJAX calls after the page is complete, or bookmarks with # in the URL). It does include AJAX calls that the page makes before it is complete.

UXI

The User Experience Index (UXI) is a value (0-5) which measures the overall performance and health of applications, based on the number of crashes per hour out of the total usage time, the percentage hang time out of the total usage time, and the percentage wait time out of the total usage time. For web applications, it also uses the percentage of web page errors out of all page loads, and the average page load time.

Learn more.