Understanding Queue Utilization in XDMoD using CPU, GPU, & Jobs Run

This article covers: CPU Hours over Time, GPU Hours over Time, and Number of Jobs Run within a specific Queue.

Description:

This article explains how to use standard XDMoD reports to evaluate queue utilization of CPUs & GPUs, as well as the number of jobs Run over a specific period of time.

Resolution:

Viewing CPU Hours over Time by Queue

1. Open https://xdmod.hpc.wichita.edu
2. Open the Usage tab
3. In the toolbar under the Usage tab, select the appropriate "Duration" or Start/End dates.
4. In the "Metrics and Options" panel on the left, select and expand the "Jobs Summary" bucket
5. Choose the "CPU Hours: Total" option
6. On the provided chart on the right, left-click on any part of the line displayed to "Drilldown to:"
7. Choose the "Queue" option. You will see the top 10 queues utilized for your time frame.
8. If the lines of the various queues are too overlapping to properly select the queue you are looking for, you can narrow down to only the queue(s) you want to look at.
   1. Option 1: Use the Filter on the toolbar to select only the queue(s) that you want to review and select 'ok'.
   2. Option 2: Click on the associated lines for those queues you do not want to view down in the 'key' area under the graph. This will remove them from your view and allow you to see only the queue you want to review.

Viewing GPU Hours over Time by Queue

1. Open https://xdmod.hpc.wichita.edu
2. Open the Usage tab
3. In the toolbar under the Usage tab, select the appropriate "Duration" or Start/End dates.
4. In the "Metrics and Options" panel on the left, select and expand the "Jobs Summary" bucket
5. Choose the "GPU Hours: Total" option
6. On the provided chart on the right, left-click on any part of the line displayed to "Drilldown to:"
7. Choose the "Queue" option. You will see the top 10 queues utilized for your time frame.
8. If the lines of the various queues are too overlapping to properly select the queue you are looking for, you can narrow down to only the queue(s) you want to look at.
   1. Option 1: Use the Filter on the toolbar to select only the queue(s) that you want to review and select 'ok'.
   2. Option 2: Click on the associated lines for those queues you do not want to view down in the 'key' area under the graph. This will remove them from your view and allow you to see only the queue you want to review.

Viewing Number of Jobs Run Within a Queue

1. Open https://xdmod.hpc.wichita.edu
2. Open the Usage tab
3. In the toolbar under the Usage tab, select the appropriate "Duration" or Start/End dates.
4. In the "Metrics and Options" panel on the left, select and expand the "Jobs Summary" bucket
5. Choose the "Number of Jobs Submitted" option
6. On the provided chart on the right, left-click on any part of the line displayed to "Drilldown to:"
7. Choose the "Queue" option. You will see the top 10 queues utilized for your time frame.
8. If the lines of the various queues are too overlapping to properly select the queue you are looking for, you can narrow down to only the queue(s) you want to look at.
   1. Option 1: Use the Filter on the toolbar to select only the queue(s) that you want to review and select 'ok'.
   2. Option 2: Click on the associated lines for those queues you do not want to view down in the 'key' area under the graph. This will remove them from your view and allow you to see only the queue you want to review.

Evaluating Queue Utilization

• Utilization assessment comes from combining all three metrics for the individual queue.
  • Use Number of Jobs Run to assess engagement
  • Use CPU Hours Total to measure compute impact
  • Use GPU Hours Total to assess specialized resource use