Analyzing database lock timeout trace telemetry

INTRODUCED IN: Business Central 2020 release wave 1, version 16.0

Database lock timeout telemetry gathers information about database locks that have timed out. The telemetry data allows you to troubleshoot what caused these locks.

In the client, when a lock has timed out, the user is presented with a message, similar to the following message:

The operation could not complete because a record in the [table name] table was locked by another user. Please retry the activity.

Two types of trace events are emitted to Application Insights:

  • The first is a Database lock timed out event. This event includes general information about the lock request. This event includes information like the AL object and code that is impacted by the lock, the extension involved, and more.

  • The Database lock timed out event then triggers one or more Database lock snapshot events. Database lock snapshot events provide details about SQL sessions that hold database locks at the time of lock timeout, including the session that caused the lock timeout. These events include specific details about the SQL lock request on the database, like the type, status, mode, and the table.

Note

In later versions of Business Central on-premises, the Business Central Server includes the EnableLockTimeoutMonitoring setting. Use this setting to turn database lock timeout telemetry on or off. By default, it is off. Learn more in Configuring Business Central Server.

Database lock timed out

Occurs when a database lock has timed out for a session.

General dimensions

DimensionDescription or value
messageDatabase lock timed out
severityLevel3

Custom dimensions

DimensionDescription or value
aadTenantIdSpecifies the Microsoft Entra tenant ID used for Microsoft Entra authentication. For on-premises, if you aren't using Microsoft Entra authentication, this value is common.
alObjectIdSpecifies the ID of the running AL object that requested the lock.
alObjectNameSpecifies the name of the running AL object that requested the lock.
alObjectTypeSpecifies the type of the running AL object that requested the lock, such as a page or report.
alStackTraceThe stack trace in AL.
clientTypeSpecifies the type of client that executed the SQL statement, such as Background or WebClient. Learn more in ClientType option type.
companyNameSpecifies the display name of the Business Central company.
componentDynamics 365 Business Central Server
componentVersionSpecifies the version number of the component that emits telemetry (see the component dimension.)
deprecatedKeysSpecifies a comma-separated list of all the keys that have been deprecated. The keys in this list are still supported but will eventually be removed in the next major release. We recommend that you update any queries that use these keys to use the new key name.
eventIdRT0012
environmentNameSpecifies the name of the tenant environment. Learn more in Managing environments.
environmentTypeSpecifies the environment type for the tenant, such as Production, Sandbox, Trial. Learn more in Environment types.
extensionIdSpecifies the AppID of the extension that was involved in the lock.
extensionNameSpecifies the name of the extension that was involved in the lock.
extensionVersionSpecifies the version of the extension that was involved in the lock.
sessionIdSpecifies the ID of the session that requested the lock.
snapshotIdSpecifies the ID of the database snapshot. This ID is used to identify associated Database lock snapshot trace events.
sqlServerSessionIdSpecifies the ID of the SQL server session that requested the lock.
sqlStatementSpecifies the SQL statement whose lock request timed out.
telemetrySchemaVersionSpecifies the version of the Business Central telemetry schema.

Sample KQL code

This KQL code can help you get started troubleshooting lock timeouts.

Note

If these events aren't available on-premises, check whether EnableLockTimeoutMonitoring is enabled. Learn more in Analyzing database lock timeout trace telemetry.

traces
| where timestamp > ago(2d) // adjust to your needs
| where customDimensions has 'RT0012'
| where customDimensions.eventId == 'RT0012'
| project timestamp
, aadTenantId = customDimensions.aadTenantId
, environmentType = customDimensions.environmentType
, environmentName = customDimensions.environmentName
, companyName = customDimensions.companyName
, clientType = customDimensions.clientType
, alObjectId = customDimensions.alObjectId
, alObjectType = customDimensions.alObjectType
, alObjectName = customDimensions.alObjectName
, extensionVersion = customDimensions.extensionVersion
, extensionName = customDimensions.extensionName
, extensionId = customDimensions.extensionId
, alStackTrace = customDimensions.alStackTrace
, sqlStatement = customDimensions.sqlStatement
, sqlServerSessionId = customDimensions.sqlServerSessionId
, snapshotId = customDimensions.snapshotId
, sessionId = customDimensions.sessionId
, usertelemetryId = case(
  // usertelemetryId was introduced in the platform in version 20.0
  toint( substring(customDimensions.componentVersion,0,2)) >= 20, user_Id
, 'N/A'
)

Database lock snapshot (entry)

This telemetry event can occur in two ways:

  1. When a database lock has timed out
  2. When a user or AL process has requested that a snapshot is taken

In the case of a database lock timeout, the BC server also takes a snapshot. Here, the Database lock snapshot trace event is associated with a specific Database lock timed out trace event. The value of the {snapshotId} maps to the snapshotId dimension of the Database lock timed out trace event that triggered this event.

General dimensions

DimensionDescription or value
messageDatabase lock snapshot entry: {snapshotId}
severityLevel3

Custom dimensions

DimensionDescription or value
aadTenantIdSpecifies the Microsoft Entra tenant ID used for Microsoft Entra authentication. For on-premises, if you aren't using Microsoft Entra authentication, this value is common.
alObjectIdSpecifies the ID of the running AL object that requested the lock.
alObjectNameSpecifies the name of the running AL object that requested the lock.
alObjectTypeSpecifies the type of the running AL object that requested the lock, such as a page or report.
alStackTraceThe stack trace in AL.
componentDynamics 365 Business Central Server
componentVersionSpecifies the version number of the component that emits telemetry (see the component dimension.)
deprecatedKeysSpecifies a comma-separated list of all the keys that have been deprecated. The keys in this list are still supported but will eventually be removed in the next major release. We recommend that you update any queries that use these keys to use the new key name.
eventIdRT0013
environmentNameSpecifies the name of the tenant environment. Learn more in Managing environments.
environmentTypeSpecifies the environment type for the tenant, such as Production, Sandbox, Trial. Learn more in Environment types.
extensionIdSpecifies the AppID of the extension that was involved in the lock.
extensionNameSpecifies the name of the extension that was involved in the lock.
extensionVersionSpecifies the version of the extension that was involved in the lock.
sessionIdSpecifies the ID of the session that requested the lock.
snapshotIdSpecifies the ID of the database snapshot. All messages in the snapshot share this ID.
sqlLockRequestModeSpecifies the lock mode that determines how concurrent transactions can access the resource. Learn more in Lock Modes.
sqlLockRequestStatusSpecifies the current status of the lock, which can be one of the following values:<ul><li>CNVRT - means that the lock is transitioning from another mode, but the conversion is blocked by another process that holds a lock with a conflicting mode.</li><li>GRANT - means that the lock is active.</li><li>WAIT- means that the lock is blocked by another process that holds a lock with a conflicting mode.</li></ul>
sqlLockResourceTypeSpecifies the database resource affected by the lock. For example, DATABASE, FILE, OBJECT, PAGE, KEY, and more.
sqlServerSessionIdSpecifies the ID of the SQL server session that requested the lock.
sqlTableNameSpecifies the name of table on which the lock was held.
telemetrySchemaVersionSpecifies the version of the Business Central telemetry schema.

Sample KQL code

This KQL code can help you get started analyzing database lock snapshots:

traces
| where timestamp > ago(2d) // adjust to your needs
| where customDimensions has 'RT0013'
| where customDimensions.eventId == 'RT0013'
| project timestamp
, aadTenantId = customDimensions.aadTenantId
, environmentType = customDimensions.environmentType
, environmentName = customDimensions.environmentName
, companyName = customDimensions.companyName
, snapshotId = customDimensions.snapshotId
, clientTypeHoldingLock = customDimensions.clientType
, alObjectTypeHoldingLock = customDimensions.alObjectType
, alObjectIdHoldingLock = customDimensions.alObjectId
, alObjectNameHoldingLock = customDimensions.alObjectName
, alStackTraceHoldingLock = customDimensions.alStackTrace
, extensionNameHoldingLock = customDimensions.extensionName
, extensionIdHoldingLock = customDimensions.extensionId
, extensionVersionHoldingLock = customDimensions.extensionVersion
, extensionPublisherHoldingLock = customDimensions.extensionPublisher
, sqlTableLocked = customDimensions.sqlTableName
, sqlLockResourceType = customDimensions.sqlLockResourceType
, sqlLockRequestMode = customDimensions.sqlLockRequestMode
, sqlLockRequestStatus = customDimensions.sqlLockRequestStatus
, sqlServerSessionId = customDimensions.sqlServerSessionId
, alSessionIdHoldingLock = customDimensions.sessionId

Database lock snapshot (parent)

Occurs when a database lock snapshot is taken to capture the current locking state. This event is emitted as a header/parent event that groups related lock snapshot entries (RT0013).

APPLIES TO: Business Central 2022 release wave 1, version 20.0, and later

General dimensions

DimensionDescription or value
messageDatabase locks snapshot taken: {snapshotId}
severityLevel3 (Error)

Custom dimensions

DimensionDescription or value
aadTenantIdSpecifies the Microsoft Entra tenant ID used for Microsoft Entra authentication. For on-premises, if you aren't using Microsoft Entra authentication, this value is common.
alObjectIdSpecifies the ID of the AL object that requested the lock snapshot.
alObjectNameSpecifies the name of the AL object that requested the lock snapshot.
alObjectTypeSpecifies the type of the AL object that requested the lock snapshot.
clientTypeSpecifies the type of client that opened the session, such as Background or Web. For a list of the client types, see ClientType Option Type.
companyNameThe current company name.
componentDynamics 365 Business Central Server
componentVersionSpecifies the version number of the component that emits telemetry.
environmentNameSpecifies the name of the tenant environment. Learn more in Managing Environments.
environmentTypeSpecifies the environment type for the tenant, such as Production or Sandbox. Learn more in Environment Types.
eventIdRT0027
extensionIdSpecifies the AppID of the extension that contains the AL object.
extensionNameSpecifies the name of the extension that contains the AL object.
extensionPublisherSpecifies the publisher of the extension that contains the AL object.
extensionVersionSpecifies the version of the extension that contains the AL object.
snapshotIdSpecifies the unique GUID for the lock snapshot. Use this ID to correlate with lock snapshot entries (RT0013).

Sample KQL code

traces
| where timestamp > ago(60d)
| where customDimensions.eventId == 'RT0027'
| project timestamp
, message
, aadTenantId = customDimensions.aadTenantId
, environmentName = customDimensions.environmentName
, companyName = customDimensions.companyName
, snapshotId = customDimensions.snapshotId
, alObjectId = customDimensions.alObjectId
, alObjectName = customDimensions.alObjectName
, alObjectType = customDimensions.alObjectType

Troubleshooting locking issues with telemetry

The telemetry you see for lock timeouts only represents the session that is the victim in a locking issue, not the sessions that hold the locks. When analyzing database lock timeout telemetry, it's therefore useful to look at combined data from the Database lock timed out event and Database lock snapshot events. You might be lucky that the snapshot has captured information from locks that were held when the lock timeout occurred.

You can combine data from different events by using joins in your Kusto queries. Learn more in LockTimeouts.kql in the Microsoft/BCTech repository on GitHub. Learn more in Joins in Azure Monitor log queries in the Microsoft Azure documentation.

Monitoring and Analyzing Telemetry
Enable Sending Telemetry to Application Insights