How to Monitor Azure AD Sign-ins logs and Detect Attacks Proactively

As the Azure cloud administrator, you need to know who is accessing your cloud resources, how they are access it, what they access, what changed, when they access and from where, etc?

Azure AD (Azure Active Directory) provides answers to above by storing the information in two logs, the information stored in them is extremely valuable for troubleshooting, monitoring and for general security related work, the logs are:

  • Azure AD Audit log
  • Azure AD Sign-ins log

Azure Audit log provides you with access to the history of every task performed in your tenant. For example, Information about changes applied to your tenant such as users and group management, updates applied to your tenant’s resources, etc. In my last article, I covered comprehensive monitoring and auditing of the Audit log to ensure you know what has changed regarding who can access your systems and with what privileges, see: https://www.eginnovations.com/blog/monitor-azure-ad-audit-logs/

Azure Sign-ins log helps you to determine who has performed the tasks reported by the Azure Audit log. In this article I will cover how to proactively monitor and historically audit and report on Azure AD Sign-in logs.

The Azure AD sign-ins log is an indispensable tool for troubleshooting and investigating security-related incidents in your tenant. Moreover, proactively, and constantly monitoring the Sign-ins can prevent breaches, alert administrators to malicious attacks and anomalous usage patterns and enable them to reduce their vulnerability by ensuring systems are configured to allow access only to those users and services that need access using up-to-date best practice authentication mechanisms and so on.

With Sign-ins log, you can answer to some of these questions below:

  • What is the sign-in pattern of a particular user, application or service?
  • How many users, apps or services have signed in over a week?
  • What is the status of these sign-ins?
  • What is the status of conditional access defined?
  • Is any legacy authentication mechanism being used for signing in?
  • Has any brute-force signing attempt happened?
  • Has a password spraying attempt has happened?
  • Are Any malicious sign-ins happening?
  • Have there been sign-in attempts with disabled accounts?
  • Has any account been breached recently?

Azure AD provides 4 types of Sign-ins logs now

  • Classic Sign-in (Interactive Sign-ins)
  • Non-Interactive Sign-ins
  • Service Principal Sign-ins
  • Managed Identity Sign-ins
Log Name Sign-ins Detail
Interactive Sign-ins logs provides the sign-in details about the user who uses authentication factor such as a password, MFA (multifactor authentication) token or QR code, etc.
Non-Interactive Sign-ins logs provides the sign-in details of the client application that perform sign-in activity on behalf of the user without any interaction from the user in the form of password or MFA token.
Service Principal Sign-in logs provides the sign-in details of application and services that perfmon sign-in activity on its own behalf to authenticate or access resources.
Managed Identity Sign-in logs provides the sign-in details of Azure resources that have secrets managed by Azure in Key Vault service. Managed identities provide an identity for applications to use when connecting to resources that support Azure Active Directory (Azure AD) authentication.

Required Permissions

You need to be a Global Administrator or the user with one of the following Azure roles to access the above sign-in data and Azure AD workbook reports:

  • Security administrator
  • Security reader
  • Global reader
  • Reports reader

License Required to Access Sign-ins Report

Azure AD Premium P1 or P2 license is required to fetch the above information using the Graph API (Application Programming Interface) programmatically. Azure AD stores the sign-ins data for 30 days for premium P1 or P2 license whereas it stores for 7 days alone for Azure Free license. If you want to store this data for longer period for compliance reasons, you can route it to Azure Storage account.

A screenshot of default report workbook available on Azure AD tenant is shown below

Azure AD Monitoring Tenant Screen

The workbook allows manual examination by an administrator but in practice many Azure AD admins use KQL (Kusto Query Language) queries or import workbooks from the community Git repositories for generating reports that are not available by default. Additionally, you can use Microsoft Sentinel for Azure AD sign-ins reports as an additional paid for service, Microsoft Sentinel is charged based on

  • Ingestion of Sign-in data per GB.
  • Retention of data – Free for first 90 days and charged per GB per month.

The Sign-ins log is a treasure trove for threat hunting actively and a gold mine for security operation centre. Marius Sandbu explained the Azure AD threat hunting in his blogs, “Threat Hunting in Microsoft Azure” and “Auditing and Security Monitoring of Azure Active Directory”. Similarly, Microsoft’s Christiaan Brinkhoff has written about increasing the security level of AVD environment with Azure conditional access: “Learn how to increase the security level of your Azure Virtual Desktop environment (e.g. Windows Client) with Azure MFA and Conditional Access”.

Beyond security though proactive alerting of sign in failures and problems enables IT administrators to rapidly identify users having problems accessing resources or applications, service availability issues allowing rapid remediation to minimise those impacted. Help desk operators can rapidly provide support to individual users reporting problems logging in identifying common user errors or issues such as mis-typing passwords or failing to use MFA.

How eG Enterprise Monitors the Azure AD Sign-ins logs?

eG Enterprise Azure AD monitoring monitors all types of Sign-ins log, records and stores them. So, you can store this data for a longer period for compliance reasons. eG also allows you to slice and dice this data without writing any KQL queries or importing the workbooks from community repositories using custom reports and dashboards.

eG Azure AD Sign-ins monitoring helps you to find answers to questions such as:

  • How many sign-ins happened recently?
  • How many of them succeeded and how many failed?
  • Have any risky sign-ins happened recently?
  • How many people are using single-factor and multi-factor authentication?
  • Is any legacy authentication being used for sign-ins?
  • Does the failed sign-ins happen due to conditional access failure?
  • Have any brute-force and password spraying sign-ins attacks occurred recently?
  • Have there been any malicious sign-ins attempt recently?
  • How many users are registered for MFA and what method(s) are used they registered for?

Out-of-the-box eG proactively scans and monitors the logs, alerting thresholds are set on key metrics and raise alerts on events that may be of concern. Additional alerts can be set via the simple GUI. Beyond this information about the Sign-in logs can be accessed via dashboards and comprehensive reports that can be scheduled to ensure regular traceable auditing. As an AIOPs (Artificial Intelligence for Operations) platform eG Enterprise applies machine learning to your data to learn and baseline normal usage patterns, more details of these capabilities are covered here.

eG Enterprise reports the following metrics for Interactive sign-ins:

Azure Interactive Sign Ins

Figure 2: The magnifying glass icon indicates that deeper diagnostic data is available for that event/metric e.g., the details of the failed sign-ins.

eG Enterprise reports the following metrics for Service Principal sign-ins:

Azure Non Interactive Sign Ins

eG Enterprise reports the following metrics for Non-Interactive sign-ins:

Azure AD Sign Ins

eG Enterprise reports the following metrics for Managed Identity sign-ins:

Azure Managed Identity Sign Ins

The data collected by eG Enterprise includes deep diagnostics that can be accessed via the layer model GUI but also within dashboards and reports. eG Enterprise allows administrators to create and publish their own custom dashboards and reports as required. If you are unfamiliar with the eG Enterprise User interface, you may like to check out some of our short overview videos.

Example Dashboards and Reports of Sign-in Log data

Dashboards providing overviews can be an extremely useful way to overview usage patterns and many common malicious attack mechanisms have distinctive patterns.

Azure AD Monitoring Dashboards

Figure 6: Overall sign-in data overview

Interactive Sign Ins Using Single Factor Authentication

Figure 7: Investigating the details of Interactive Sign-ins using Single factor authentication

Failed Sign Ins Screen

Figure 8: Instant access to the details of Failed Sign-ins

Service Principal Sign Ins

Figure 9: Monitoring Service Principal Sign-ins to ensure those services are up

Users and Applications Screen

Figure 10: Identify issues with specific applications, groups of users or geographical locations.

Brute Force Spraying Attack screen

Figure 11: Brute force and password spraying attacks can be easily identified and the details examined. Here an individual user attempted to log in from numerous geographically distant locations and IP addresses.

Service Failures screen

Figure 12: Steep and sudden spikes in failures often indicate a service failure and such failures often impact users in specific locations. Daily working patterns e.g., the 9 am morning logon or 1pm back from lunch surge become very clear. Anomalous behaviour such as users logging on at 3am from unusual locations should trigger red flags.

Legacy Authentication screen

Figure 13: Proactively audit legacy authentication and whether more secure mechanisms may be appropriate.

Azure data authentication screen

Figure 14: Data on authentication can be used in modernization and transformation projects to demonstrate progress as applications are migrated to better authentication mechanisms, or used to audit the vulnerability of an organization.

Data Vulnerability screen

Figure 15: Investigate Conditional Access

Key Applications Monitoring screen

Figure 16: Are there issues with specific key applications such as Office 365

This blog post is the fifth in a series covering monitoring of various aspects of Azure AD, previously I have covered:

Monitoring and analytics reporting of Azure AD helps our customers comply with many compliance and regulatory standards such as the European GDPR, Australian Privacy Act, PCI DSS, HIPAA, GLBA, FISMA, SOX, and more. Ensuring Azure AD is fully monitored helps our customers detect and respond to insider threat, privilege misuse, and other indicators of compromise, and improve their security processes.

Further Reading

Many of our customers also use eG Enterprise to monitor Azure infrastructures, other clouds, and on-premises infrastructure and applications. Please see the links below for relevant information: