Troubleshoot Virtual Application Server (Citrix XenApp or Citrix Virtual Apps)

Use the Aternity Troubleshoot Virtual Application Server dashboard to isolate a problematic virtual application server and locate the session or application that causes slowdown in performance.

The Troubleshoot Virtual Application Server dashboard displays the Citrix XenApp server resources' utilization and top processes running on that server as well as delays to performance caused by remote display latency in virtual server. Use this dashboard to isolate the virtual server and latency issues which are causing the slowdowns in applications hosted on that server. Note that in advanced versions of Citrix the new name is Citrix Virtual Apps.

Troubleshoot the Citrix app server by viewing key support information

If a user calls support about an application slowdown in performance, first find out to what server this user is connected (using User Experience dashboard). In case it is a virtual application server (like Citrix XenApp), use this dashboard to troubleshoot the problem. All your typical device questions are laid out for you: the operating system, network connection, top processes, memory, CPU usage, health events and so on. Correlate high resource usage with Windows processes running on that server or with network attributes to find the reason for poorly-performing virtual application server.

Change the timeframe to view the server at the time the problem occurred, making it a powerful troubleshooting tool.

Note

This dashboard requires Agent for End User Devices 11.0.1 or later. You can troubleshoot virtual application servers after installing the Agent for End User Devices 11.0.1 on them.

The Troubleshoot Virtual Citrix Server Dashboard

The upper line of tabs display different dashboards related to this specific device. The bottom line of tabs display different attributes you can investigate.

Use Troubleshoot > Overview as your first entry point to troubleshoot the server's slowdown in performance. Get a holistic view on attributes of the virtual server and their utilization measurement as well as on hardware spec of that server. View the performance of the attributes over time. In this view you can locate the problem.

Then, browse between tabs to see the details about a specific attribute (CPU, Memory, etc.) to continue the investigation of the root cause of the issue. Investigate a virtual server resources usage, which slow down an application performance. In this view you can determine what happened on the server for each session.

Procedure

  1. Step 1 Open a browser and sign in to Aternity.
  2. Step 2 Locate the required server to troubleshoot.
    With Aternity you can easily determine who is using a particular device, or view all the devices operated by a specific user. There are several ways to do this.
    • Enter the user name who reported an issue in the Search box at the top of the console and drill-down to the User Experience dashboard. Select the Citrix application server in the Devices area.
    • Alternatively, if you know that the user is definitely connected to the Citrix application server and you know the name of the server, use the search box in the top bar to quickly find servers with a specific name; enter the name of the server in the Search box at the top of the console.
  3. Step 3 In the Overview tab, hover your mouse to view the resources usage over time.
    Use this view to troubleshoot the virtual application server for a slowdown in performance. In the Overview tab, look at the statuses of the server resources consumption over time to see when the problem occurred or even if it recurs at regular intervals.
    Have an overall view on the server resources' usage

    View the total number of open sessions into this virtual application server during the timeframe. Hover your mouse pointer over the line to view the number of active and inactive sessions.

    The dashboard displays sessions which connect via the RDP, ICA, Blast or PCoIP protocols.

    Note

    By default, a virtual session only reports data to Aternity while a user is logged in to Windows, and stops when a user logs out. Aternity does not report boot times for virtual sessions.

    Field Description
    Health Events A health event indicates a device or application encountered a significant error which affects the overall health of the user experience, like a corrupted disk or a system crash. There are three types of health events: application health events, hardware health events, and system health events.
    Boot Machine boot is part of a device's boot time, starting a fraction of a second after the Windows logo appears, and ending with the Windows sign in screen.
    CPU Utilization Displays the percentage CPU utilization of the core with the greatest usage at a given time. For example, if the device has four CPU cores, where one is at 80%, one is at 60% and the others are idle, it will display a value of 80%.
    Max CPU Core Utilization (Windows) Displays the individual CPU core processor with the highest percentage usage at a given time. Look for 100% for a length of time (flat line), indicating a process is stuck and hogging that CPU's resources. For example, if the device has four CPU cores, where one is at 100% usage and the others are idle, it will display a value of 100%.
    Physical Memory Displays the processes which utilize the most physical memory (known as the total working set) during the timeframe, and view the maximum physical memory usage for each process.

    Use this to find processes which suffer memory leakage, causing other applications to slow down.

    By default, Aternity collects top processes data if the total physical RAM usage of all processes on the device rises above 90%, or if the disk queue length is more than 1.

    Virtual Memory (Windows only) Displays the processes which reserve the most virtual memory (known as commit size), during the timeframe, and view the peak commit size for each process.

    By default, Aternity collects top processes data if the total commit size usage of all processes on the device rises above 90% of the device's virtual memory, or if the disk queue length is more than 1.

    Disk Queue Length Displays the number of waiting I/O requests to read or write to the hard disk or a logical disk at a given time

    A consistent queue for the disk indicates a bottleneck in hard disk access, which significantly impacts on system performance, either due to excess system demands on the disk, or it can be a hardware disk problem. To check if the problem is hardware, view if the speed (rate of reads and writes to the disk) is low

    Disk IO Read Rate

    Displays the rate at which the device reads from the hard disk in MB per second at any given time.

    For example, if a virus scanner slows performance by issuing many disk read requests, reschedule to off-peak times. Alternatively, if the read rate falls to almost zero, the hard disk may be failing, or its connection to the computer may be unreliable.
    Disk IO Write Rate

    DIsplays the rate at which the device writes to the hard disk in MB per second at any given time.

    For example, a movie editor can perform large disk writes, slowing down the device's performance. Alternatively, if the write rate falls to almost zero, the hard disk may be failing, or its connection to the computer may be unreliable.

    Network IO Read Rate

    Displays the data downloads of this device in KB per second at any given time (the units change dynamically according to the size of information, so that you may see the data in MBps also)..

    For example, if its throughput or usage of bandwidth is low, and the user complains of slow network connections, consider checking the NIC hardware.

    Network IO Write Rate

    Displays the data uploads from this device in KB per second at any given time (the units change dynamically according to the size of information, so that you may see the data in MBps also)..

    For example, if its throughput or usage of bandwidth is low, and the user complains of slow network connections, consider checking the NIC hardware.

    Remote Display Latency

    The remote display latency is the average time taken for the round trip of a network data packet to travel between the front line user and a virtual server (both ways). Practically, it is the time between performing an action in a virtual session on the front line user's machine, then sending that action to the virtual desktop server (VDI) or virtual application server, and then viewing that action back on the front line terminal again. This does NOT measure the time for the application to respond. The Agent retrieves the session latency from Windows every 15 seconds and sends an average to Aternity every minute.

    For example, if a user types the character 'g' in a text editor which runs on a virtual application server, when the remote session sends this action to the virtual server, the remote display latency is the lag time between typing 'g' to seeing the 'g' on the screen.
    Remote display latency is the time in both directions from the front line user to the virtual server

    This dashboard shows the remote display latency for the ICA protocol only. ICA is the protocol used by Citrix.

    Sessions

    (Virtual application servers only) View the total number of open sessions into this virtual application server during the timeframe. Hover your mouse pointer over the line to view the number of active and inactive sessions.

    The dashboard displays sessions which connect via the RDP, ICA, Blast or PCoIP protocols.
  4. Step 4 In the Overview tab, in the Summary area, check this server's key information, including its location, model, IP, operating system, and more.
    Field Description
    Business Location

    Displays the current location of the device, and whether or not it is connected via VPN (by checking for known VPN adapters which are operational).

    For virtual applications (like Citrix XenApp), Aternity always tries to report the location of the end user's front-end device by detecting its subnet.

    CPU Cores

    Displays the number of CPU cores of the device.

    CPU Generation

    (Windows on Intel only) Displays the generation of the Intel Core micro-architecture. For example 6 represents the 6th generation architecture processor, also known as Skylake.

    CPU Model

    (Windows on Intel only) Displays the model and speed of the Intel processor, as displayed in the System control panel. For example Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz or Intel(R) Xeon(R) CPU E5620 @ 2.40GHz.

    CPU Type

    (Windows on Intel only) Displays the core type of the Intel processor, for example i7, E5, and so on).

    Citrix Farm Name If this XenApp server belongs to a farm, it displays the name of the farm. A farm is a set of servers running the Citrix Presentation Server, and configured to be managed as a single unit.
    Citrix Product Edition Displays the type of Citrix XenApp set up on this server. For example, Citrix XenApp, Platinum Edition.
    Citrix Product Version Displays the release or version number of Citrix XenApp deployed on this server. For example, 7.8.0.

    Relevant for Citrix XenApp 7 only.

    In order to collect the Citrix Product Version attribute, Aternity requires the Citrix Director Windows Management Instrumentation Provider service. Currently without the service being installed, the product version will not be populated. It is a known limitation and will be fixed in near future.

    Citrix Zone Name Displays the name of zone within the farm, where all servers use the same data collector, which acts as their load balancer.

    Relevant for Citrix XenApp 6.5 only.

    IP Address Displays the device's internal IP address (including IP v6 if the device runs Agent 10 or later) which it uses to connect to Aternity.
    Manufacturer Displays the name of the vendor which created this device, like Samsung, Apple, Dell, Lenovo, and so on.
    Memory Displays the size of physical RAM of the device.
    Model Displays the name and the model number of the device, like iPhone 6s, GalaxyTab8, MacBook Pro 12.1, Dell Latitude D620.
    OS Architecture Displays whether the operating system of the monitored device is 32-bit or 64-bit.
    OS Disk Type (Windows only, Agent 9.0.3 or later) Displays the type of hard disk containing the operating system. Possible values are:
    • HDD for a traditional spinning hard disk drive

    • SSD for a solid state drive

    • Virtual if this is not a physical device.

    OS Name Displays the generic name and version of the operating system (like MS Windows 10, MS Windows Server 2008 R2, MacOS 10.3, iOS 10 or Android 6).
    OS Version

    Displays the full name, the exact version number, and the service pack version of the operating system. In Windows 10, it includes the release ID (like Microsoft Windows 10 Enterprise 1507). Use this to differentiate between details of the same operating system. For example, it lists MS Windows Server 2008 R2 Enterprise SP 1.0 separately from MS Windows Server 2008 R2 Enterprise SP 2.0.

    Subnet

    Displays the device's subnet configuration used to connect to Aternity (including IP v6 if the device runs Agent 10 or later).

    For virtual applications (like Citrix XenApp), Aternity always tries to report the location of the end user's front-end device by detecting its subnet.

  5. Step 5 Select the tab of the resource you want to investigate and view the correlation between usage of the server resource and Windows processes which utilize the most of the selected resource at the specified timeframe.

    For example, if you select CPU Utilization, view the Top Processes which utilize the most CPU.

    Correlate between CPU utilization and virtual sessions
  6. Step 6 Check Top Processes to view the direct effects of using a particular Windows process.

    Under Top Processes you can see the top five names of the processes that consume the most of selected resource.

    In the Top Processes area, select the process you want to further investigate.

    The list of Top Consumers changes accordingly, showing only those users who consume the most of the selected process.

  7. Step 7 In the Top Consumers area, select the user you want to further investigate.
    For example, you can figure out that the user who called support experiences a poor performance because another user consumes most of the server's CPU.
  8. Step 8 Select this user in the Sessions area and investigate the metrics at a given time.
    Here you can learn about this user experience. For example, maybe it is a virus scanner slows performance by issuing many disk read requests; if so, reschedule it to off-peak times.
    Another example, if you notice high values for ICA Data Sent and Latency, examine the root cause for slow network communication. Often high latency can result from the client and server being physically distant, like on different continents. If your users complain about latency performance, consider investing in a physically closer data center or hypervisor.
    Investigate user experience metrics

    This view presents all attributes and their metrics for the selected user session at a given time. So, you do not have to move between tabs in order to see a different attribute.

    Field Description
    Time

    Displays the date and time of the measurement.

    Client Device Name

    (For virtual deployments only) Displays the hostname of a device which is connecting to a VDI or virtual application server.

    Username Displays the username signed in to the device's operating system.
    Connection State

    Displays Active and Inactive sessions.

    Inactive user session means that the user disconnected from a remote application, but the session is still running on the server allowing that user to reconnect to the session fast. Connection state reflects whether the user is currently viewing the application on the screen (active) or it is kept on the server (inactive). This allows to identify if a specific user was actually using the application at a given time and to estimate the utilization of the server resources.

    ICA Data Received

    The amount of data received by the Client as part of of virtual channel streaming using ICA protocol.

    ICA Data Sent

    The amount of data sent from the Client to Server as part of virtual channel streaming using ICA protocol.

    Average Client Latency

    The average latency of streaming. This is the time interval measured at the Client between user action and graphical response.

    If it is high, users will experience slowness or non-smooth mouse movements or the screen lag while interacting with an application hosted in a session on a XenApp or XenDesktop Server.

    CPU Utilization Displays the percentage CPU utilization of the core with the greatest usage at a given time. For example, if the device has four CPU cores, where one is at 80%, one is at 60% and the others are idle, it will display a value of 80%.
    IO Read Rate (Windows only) Displays the processes with the highest data input rate (read requests) from the network and hard disk during the timeframe, and displays the maximum read rate for each process

    For example, if a virus scanner slows performance by issuing many disk read requests, reschedule to off-peak times. Alternatively, if the read rate falls to almost zero, the hard disk may be failing, or its connection to the computer may be unreliable.

    By default, Aternity collects top processes data if the total read rate from the hard disk exceeds 1 megabyte per second (MBps), or if the disk queue length is more than 1.

    IO Read Write (Windows only) Displays the processes with the highest data output rate (write requests) from the network and hard disk during the timeframe, and displays the maximum write rate for each process

    For example, a movie editor can perform large disk writes, slowing down the device's performance. Alternatively, if the write rate falls to almost zero, the hard disk may be failing, or its connection to the computer may be unreliable.

    By default, Aternity collects top processes data if the total write rate to the hard disk exceeds 1 megabyte per second (MBps), or if the disk queue length is more than 1.

    Physical Memory Displays the processes which utilize the most physical memory (known as the total working set) during the timeframe, and view the maximum physical memory usage for each process.

    Use this to find processes which suffer memory leakage, causing other applications to slow down.

    By default, Aternity collects top processes data if the total physical RAM usage of all processes on the device rises above 90%, or if the disk queue length is more than 1.

    Virtual Memory (Windows only) Displays the processes which reserve the most virtual memory (known as commit size), during the timeframe, and view the peak commit size for each process.

    By default, Aternity collects top processes data if the total commit size usage of all processes on the device rises above 90% of the device's virtual memory, or if the disk queue length is more than 1.

  9. Step 9 You can limit the display of the dashboard using the menus at the top of the window.
    Select the data to display in the dashboard
    Field Description
    Time Zone Selection

    Select the time zone to view the times associated with the data in this dashboard:

    • Default displays the time zone set in most Aternity dashboards.

    • Yours displays the time zone of your computer where you are viewing the dashboard.

    • Device displays the time zone of the monitored device which is the focus of this dashboard's content.

      Tip

      The default is the time zone of the Aternity Management Server. It is possible to change the timezone to a local timezone of your account. Administrator of Aternity can set it in the My Account screen.

    Timeframe

    You can change the start time of the data displayed in the dashboard in the Timeframe menu in the top right corner of the screen. This menu impacts all tabs available in this view.

    You can access data in this dashboard (retention) going back up to seven days.

    This dashboard displays raw data in real time, refreshing every time you access it or whenever you manually refresh the browser page.

  10. Step 10 After investigating user experience in Aternity dashboards, drill-down to AppResponse to further investigate network issues.

    To quickly switch to AppResponse, select View in AppResponse in the right upper side of the Troubleshoot Citrix AppServer dashboard. Enter your AppResponse user name and password.

    Drill down to Citrix network issues to investigate network performance in AppResponse
  11. Step 11 When investigating Remote Display Latency in Aternity, you also can easily and quickly drill-down to AppResponse.
    View dashboards in AppResponse
  12. Step 12 It is possible to drill-down to Aternity from the Insights > Citrix > Citrix Servers.
    Open the Insights > Citrix > Citrix Servers and right-click on the server name.
    Go to Aternity from AppResponse Insights