Analyzing job queue lifecycle telemetry

APPLIES TO: Business Central 2020 release wave 2, version 17.2, and later

Job queue lifecycle telemetry gathers data about the following operations:

  • A job queue entry was enqueued.
  • A job queue entry was started.
  • A job queue entry finished. Provides information as to whether it was successful or failed.
  • A job queue entry was stopped.
  • A job queue entry was cancelled.
  • A job queue entry was scheduled on login.
  • A job queue entry was run by a user.

Learn more in Use Job Queues to Schedule Tasks in the Business Central application help.

Learn more in How the Job Queue works in the Business Central developer content.

The events below use TelemetryScope::All, except the successful run event AL000082C, which uses TelemetryScope::ExtensionPublisher.

Job queue telemetry events

The following table describes the event IDs that are currently emitted by the job queue system.

EventEvent idJob queue statusDescription
Job queue entry enqueuedAL0000E24EnqueuedOccurs when a job queue entry is enqueued to run at or after the specified alJobQueueEarliestStartDateTime.
Job queue entry failed to be enqueuedAL0000FNYEnqueuedOccurs when a job queue entry fails to enqueue and this happens if no task scheduler is created behind it.
Job queue entry startedAL0000E25StartedOccurs right before the Job Queue Start codeunit is triggered and that codeunit will run the specified Object ID to run.
Job queue request finishedAL0000E26Finished successfullyOccurs when a job queue request completes successfully.
Job queue run finishedAL000082CFinished successfullyOccurs when a job queue run completes successfully. This event is sent only to the extension publisher's telemetry resource.
Job queue entry failed (might be retried)AL0000HE7ErrorOccurs when a job queue errors, and this is the first thing that is triggered before the updating of records.
Job queue entry rescheduled on sign-inAL0000I49N/AEmitted when an eligible recurring job queue entry without a scheduled task is rescheduled after sign-in.
Job queue entry run once by a userAL0000FMGN/AEmitted after a user selects Run once (foreground) on the Job Queue Entries or Job Queue Entry Card page and confirms the action.
Job queue entry failed for the last time and is set to stoppedAL0000JRGErrorEmitted when the job queue entry fails for the very last time before being set in Error state.
Job queue entry cancelledAL0000KZVN/AEmitted when a job queue entry is cancelled. The event has normal severity (1).

Common custom dimensions to all job queue events

The following table explains custom dimensions that are common to all job queue entry telemetry events described below.

DimensionDescription or value
alCategoryAL JobQueueEntries
aadTenantIdSpecifies the Microsoft Entra tenant ID used for Microsoft Entra authentication. For on-premises, if you aren't using Microsoft Entra authentication, this value is common.
environmentNameSpecifies the name of the tenant environment. Learn more in Managing Environments.
environmentTypeSpecifies the environment type for the tenant, such as Production or Sandbox. Learn more in Environment Types.
companyNameThe current company name.
alJobQueueScheduledTaskIdSpecifies the ID of the scheduled task.
alJobQueueEarliestStartDateTimeSpecifies the earliest start date/time for the job queue entry.
alJobQueueIdSpecifies the ID of the job queue entry.
alJobQueueIsRecurringSpecifies whether the job queue is recurring. Yes indicates it's recurring. No indicates it's not recurring. These lifecycle events format values in the default application language.
alJobQueueObjectIdSpecifies the ID of the object that the job queue entry runs.
alJobQueueObjectNameSpecifies the name of the object that the job queue entry runs.

This dimension was introduced in version 22.0.
alJobQueueObjectTypeSpecifies the type of the object that the job queue entry runs, for example Report or Codeunit.
alJobQueueObjectDescriptionSpecifies the description of the object that the job queue entry runs.

This dimension was introduced in version 22.2.
alJobQueueStacktraceSpecifies the AL error stack trace. It's included only in the error events AL0000HE7 and AL0000JRG. This dimension was introduced in version 21.3.
alJobQueueCompanyNameSpecifies the company that contains the job queue entry.
alJobQueueMaxNumberOfAttemptsToRunSpecifies the maximum number of attempts configured for the entry. This dimension was introduced in version 21.5.
alJobQueueNumberOfAttemptsToRunSpecifies the number of attempts to run the entry. This dimension was introduced in version 21.5.
alJobQueueStatusReady indicates it's a non-recurring job queue entry or the first run of a recurring job queue entry that's ready to run. In Process indicates it's being run. Error indicates it encountered an error while running. On Hold indicates it's being queued. Finished indicates it's finished without error. On Hold with Inactivity Timeout indicates it's a recurring job query entry that's ready to run.

Job queue entry enqueued

Occurs when a job queue entry is sent to the job queue to eventually be run. A job queue entry is sent to the queue when its status is changed from On Hold to Ready or if it's a recurring job queue entry. Recurring job queue entries are automatically enqueued after each run.

General dimensions

DimensionDescription or value
messageJob queue entry enqueued: {alJobQueueId}
severityLevel1
user_IdThe user telemetry ID for the user. From the user card, you can use user_Id to identify the user who triggered this telemetry event. Learn more in Assign a telemetry ID to users.

Custom dimensions

The following table explains custom dimensions that are specific to this trace.

DimensionDescription or value
eventIdAL0000E24
alJobQueueIdSpecifies the ID of the job queue entry.
alJobQueueIsRecurringSpecifies whether the job queue is recurring. Yes indicates it's recurring. No indicates it's not recurring.
alJobQueueObjectIdSpecifies the ID of the object that the job queue entry runs.
alJobQueueObjectTypeSpecifies the type of the object that the job queue entry runs, for example Report or Codeunit.
alJobQueueObjectDescriptionSpecifies the description of the object that the job queue entry runs. This dimension was introduced in Business Central 2023 release wave 1, version 22.2.
alJobQueueStatusReady indicates it's a non-recurring job queue entry or the first run of a recurring job queue entry that's ready to run. On Hold with Inactivity Timeout indicates it's a recurring job query entry that's ready to run.

Job queue entry failed to be enqueued

Information if the job queue entry fails to be sent to the queue.

General dimensions

DimensionDescription or value
messageJob queue entry not enqueued: {alJobQueueId}
severityLevel2
user_IdThe user telemetry ID for the user. From the user card, you can use user_Id to identify the user who triggered this telemetry event. Learn more in Assign a telemetry ID to users.

Custom dimensions

The following table explains custom dimensions that are specific to this trace.

DimensionDescription or value
eventIdAL0000FNY
See common custom dimensions

Job queue entry started

Occurs when a job queue entry starts to run.

General dimensions

DimensionDescription or value
messageJob queue entry started: {alJobQueueId}
severityLevel1
user_IdThe user telemetry ID for the user. From the user card, you can use user_Id to identify the user who triggered this telemetry event. Learn more in Assign a telemetry ID to users.

Custom dimensions

The following table explains custom dimensions that are specific to this trace.

DimensionDescription or value
eventIdAL0000E25
See common custom dimensions

Job queue entry finished (request)

Occurs when a request for a job queue entry finishes running.

General dimensions

DimensionDescription or value
messageJob queue entry finished: {alJobQueueId}
severityLevel1
user_IdThe user telemetry ID for the user. From the user card, you can use user_Id to identify the user who triggered this telemetry event. Learn more in Assign a telemetry ID to users.

Custom dimensions

DimensionDescription or value
eventIdAL0000E26
alJobQueueExecutionTimeInMsSpecifies how many milliseconds it took to run the job queue entry.
alJobQueueScheduledTaskIdSpecifies the scheduled task ID for the request that finished, before any recurring entry was rescheduled.
alJobQueueNextScheduledTaskIdSpecifies the scheduled task ID stored on the entry after the request, such as the next task for a recurring entry.
See common custom dimensions

Job queue entry finished (run)

Occurs when a job queue entry finishes running.

General dimensions

DimensionDescription or value
messageJobID = {alJobQueueId}, ObjectType = {alJobQueueObjectType}, ObjectID = {alJobQueueObjectId}, Status = Finished, Result = Success, Company = {alJobQueueCompanyName}, Scheduled Task Id = {alJobQueueScheduledTaskId}
severityLevel1
user_IdThe user telemetry ID for the user. From the user card, you can use user_Id to identify the user who triggered this telemetry event. Learn more in Assign a telemetry ID to users.

Custom dimensions

DimensionDescription or value
eventIdAL000082C
See common custom dimensions

Job queue entry failed with error

Occurs when a job queue entry fails to run.

General dimensions

DimensionDescription or value
messageJob queue entry errored: {alJobQueueId}
severityLevel2
user_IdThe user telemetry ID for the user. From the user card, you can use user_Id to identify the user who triggered this telemetry event. Learn more in Assign a telemetry ID to users.

Custom dimensions

DimensionDescription or value
eventIdAL0000HE7
See common custom dimensions

Sample KQL code

This KQL code can help you get started troubleshooting job queue errors. Note that these failed entries might be restarted.

// Job queue entry run failed
traces
| where timestamp > ago(60d) // adjust as needed
| where customDimensions.eventId == 'AL0000HE7'
| project timestamp
, aadTenantId = customDimensions.aadTenantId
, environmentName = customDimensions.environmentName
, environmentType = customDimensions.environmentType
, companyName = customDimensions.companyName
// Caller app dimensions identify the telemetry caller, not necessarily the app containing the job queue object.
, jobQueueAppId = customDimensions.alCallerAppId
, jobQueueAppName = customDimensions.alCallerAppName
, jobQueueAppPublisher = customDimensions.alCallerPublisher
, jobQueueAppVersion = customDimensions.alCallerAppVersion
// jobQueueObject* dimensions contains the information about the object containing the code being run
, jobQueueObjectId = customDimensions.alJobQueueObjectId
, jobQueueObjectName = customDimensions.alJobQueueObjectName // added in 22.0
, jobQueueObjectType = customDimensions.alJobQueueObjectType
// jobQueueExecution dimensions contain information about the job queue entry definition
, jobQueueEntryId = customDimensions.alJobQueueId
, jobQueueEntryIsRecurring = customDimensions.alJobQueueIsRecurring
, jobQueueEntryDescription = customDimensions.alJobQueueObjectDescription // added in 22.2
, jobQueueEntryMaxNumberOfAttemptsToRun = customDimensions.alJobQueueMaxNumberOfAttemptsToRun // added in 21.5
// jobQueueExecution dimensions contain information about the attempted run of the job queue entry
, jobQueueExecutionNumberOfAttemptsToRun = customDimensions.alJobQueueNumberOfAttemptsToRun // added in 21.5
, jobQueueExecutionStacktrace = customDimensions.alJobQueueStacktrace
, jobQueueExecutionStatus = customDimensions.alJobQueueStatus
, jobQueueExecutionTaskId = customDimensions.alJobQueueScheduledTaskId // you can join to task scheduler telemetry on the taskId

Job queue entry {Job Queue Id} errored after {attempt number} attempts

Occurs when a job queue entry fails to run. This telemetry event was introduced in version 21.5.

General dimensions

DimensionDescription or value
messageJob queue entry {Job Queue Id} errored after {attempt number} attempts
severityLevel2
user_IdThe user telemetry ID for the user. From the user card, you can use user_Id to identify the user who triggered this telemetry event. Learn more in Assign a telemetry ID to users.

Custom dimensions

DimensionDescription or value
eventIdAL0000JRG
See common custom dimensions

Sample KQL code

This KQL code can help you get started troubleshooting job queue errors where the entry has been stopped because it failed for the last time before being set to the Error state. As an administrator, you want to know about job queue entries that are going into the final failed state and require manual intervention, such as restarting.

// Job queue entry {Job Queue Id} errored after {attempt number} attempts
traces
| where timestamp > ago(60d) // adjust as needed
| where customDimensions.eventId == 'AL0000JRG'
| project timestamp
, aadTenantId = customDimensions.aadTenantId
, environmentName = customDimensions.environmentName
, environmentType = customDimensions.environmentType
, companyName = customDimensions.companyName
// Caller app dimensions identify the telemetry caller, not necessarily the app containing the job queue object.
, jobQueueAppId = customDimensions.alCallerAppId
, jobQueueAppName = customDimensions.alCallerAppName
, jobQueueAppPublisher = customDimensions.alCallerPublisher
, jobQueueAppVersion = customDimensions.alCallerAppVersion
// jobQueueObject* dimensions contains the information about the object containing the code being run
, jobQueueObjectId = customDimensions.alJobQueueObjectId
, jobQueueObjectName = customDimensions.alJobQueueObjectName // added in 22.0
, jobQueueObjectType = customDimensions.alJobQueueObjectType
// jobQueueExecution dimensions contain information about the job queue entry definition
, jobQueueEntryId = customDimensions.alJobQueueId
, jobQueueEntryIsRecurring = customDimensions.alJobQueueIsRecurring
, jobQueueEntryDescription = customDimensions.alJobQueueObjectDescription // added in 22.2
, jobQueueEntryMaxNumberOfAttemptsToRun = customDimensions.alJobQueueMaxNumberOfAttemptsToRun // added in 21.5
// jobQueueExecution dimensions contain information about the attempted run of the job queue entry
, jobQueueExecutionNumberOfAttemptsToRun = customDimensions.alJobQueueNumberOfAttemptsToRun // added in 21.5
, jobQueueExecutionStatus = customDimensions.alJobQueueStatus
, jobQueueExecutionStacktrace = customDimensions.alJobQueueStacktrace
, jobQueueExecutionTaskId = customDimensions.alJobQueueScheduledTaskId // you can join to task scheduler telemetry on the taskId

Job queue entry run by user

Occurs when a user selects Run once (foreground) on the Job Queue Entries or Job Queue Entry Card page and confirms the action. This telemetry event was introduced in version 22.2.

General dimensions

DimensionDescription or value
messageRunning job queue once.
severityLevel1
user_IdThe user telemetry ID for the user. From the user card, you can use user_Id to identify the user who triggered this telemetry event. Learn more in Assign a telemetry ID to users.

Custom dimensions

DimensionDescription or value
eventIdAL0000FMG
See common custom dimensions

Job queue entry rescheduled on sign-in

Occurs when a job queue entry was rescheduled when a user signs in. This telemetry event was introduced in version 22.0.

General dimensions

DimensionDescription or value
messageJob queue entry rescheduled on login: {job queue entry id}
severityLevel1
user_IdThe user telemetry ID for the user. From the user card, you can use user_Id to identify the user who triggered this telemetry event. Learn more in Assign a telemetry ID to users.

Custom dimensions

DimensionDescription or value
eventIdAL0000I49
alJobQueueRescheduledNewTaskSpecifies whether rescheduling created a new scheduled task rather than making the existing task ready.
See common custom dimensions

Job queue entry cancelled

Occurs when cancellation of a job queue entry's scheduled task is attempted. This telemetry event was introduced in version 23.0.

General dimensions

DimensionDescription or value
messageJob queue entry task cancelled: {job queue entry id}
severityLevel1
user_IdThe user telemetry ID for the user. From the user card, you can use user_Id to identify the user who triggered this telemetry event. Learn more in Assign a telemetry ID to users.

Custom dimensions

DimensionDescription or value
eventIdAL0000KZV
alJobQueueScheduledTaskExistedOnCancelSpecifies whether cancellation of the scheduled task succeeded.
See common custom dimensions

Job queues and performance

Scheduled tasks or job queue entries that are set to run on a recurring schedule can impact the performance of Business Central under either of the following conditions:

  • They run too frequently.

    Consider how often the task or job needs to run. Especially for polling scenarios, you might have better and more performant ways to react, such as using webhooks.

  • They run heavy jobs while many users are also using Business Central.

    Consider running them outside working hours. This might decrease locking and deadlock issues both for users and for the task or job itself.

Troubleshooting job queue issues with telemetry

The reason for a failed job queue entry could be in the code that is running. You can find the owner (also called publisher) of the code in the AL stack trace dimension in the job queue error event AL0000HE7. If the error comes from an app/extension, you need to contact the owner of the code to get it resolved. If the error comes from the Microsoft application, then you need to open a support request with Microsoft to get it resolved.

Learn more in event AL0000HE7.

If a job queue entry fails after the maximum number of retry attempts has been done, the telemetry event AL0000JRG is emitted and the job queue entry is set to permanently being stopped. As an administrator, you want to know about these job queue entries that are going into the final failed state and require manual intervention (restarting etc).

Learn more in event AL0000JRG.

Alerting on job queue issues with telemetry

If something happens in your environment or app that you need to act on, you can set up a system that sends you an alert. Azure Application Insights makes it easy to define such alerts.

You can use the following tools to define and set up alerts on telemetry events:

  • Azure Application Insights Alerts
  • Azure Logic Apps
  • Power Automate

All three approaches need a Kusto (KQL) query to define the alerting condition.

There are at least three job queue error scenarios where you might want to set up alerts:

  1. Some job queue entries fail
  2. Some job queue entries failed and have been stopped
  3. No job queue entries are running

In the table below, you can read more about the scenarios and get sample KQL code for the alerting conditions in your alerts.

ConditionAreaRelevant forDescriptionEvent ID(s)KQL sample code
Job queue errorsErrorsVARGet alerted when job queue entries fail.AL0000HE7JobQueueFailures.kql
Job queue stopped due to errorsErrorsVARGet alerted on job queue entries that are stopped due to recurring errors (because they have failed for all retry attempts).AL0000JRGJobQueueFailures.kql
Job queue not runningErrorsVARGet alerted if no job queue entries have been started in a given time period.AL0000E25NoJobQueueRuns.kql

Learn more in Alert on Telemetry.

Use Job Queues to Schedule Tasks
How the Job Queue works
Telemetry overview
Enabling Telemetry
Alert on Telemetry