Snowflake monitoring and performance management

  • Proactive monitoring and rapid diagnosis
  • Optimize storage usage to control costs
  • Monitor query performance and much more
Free Trial

Trusted by leading companies

What is Snowflake?

Snowflake is a cloud-based data warehousing platform that provides data storage, analysis, and reporting. It is designed to be highly scalable, secure, and highly available, and supports a wide range of data types and sources.

Snowflake’s cost-per-usage model is revolutionary in the cloud database ecosystem and good tools to optimize performance and costs including those associated with cloud platform, storage and K8s dependencies are essential.

Snowflake uses a unique architecture that separates storage and computation, allowing for high levels of performance and scalability. It also provides a SQL-based interface for querying data, making it easy for analysts and data scientists to work with. Key features are:

  • A fully managed SaaS (software as a service) that provides a single platform for data warehousing, data lakes, data engineering, data science, secure sharing and consumption of real-time / shared data.
  • Out-of-the-box separates storage and compute, has on-the-fly scalable compute, data sharing, data cloning, and third-party tools support.
  • Enables data storage, processing, and analytic solutions that are faster, easier to use, and far more flexible than traditional offerings.
  • A Snowflake database is where an organization’s uploaded structured and semi-structured data sets are held for processing and analysis. Snowflake automatically manages all parts of the data storage process, including organization, structure, metadata, file size, compression, and statistics.

How eG Enterprise meets key
Snowflake monitoring requirements

eG Enterprise’s integration with Snowflake enables complete visibility into the Snowflake architecture and operations, alongside any dependent cloud hosted infrastructures. Leveraging AIOps technologies, eG Enterprise offers an observability platform to help you use Snowflake at scale and assure performance, availability and cost optimizations. With eG Enterprise you can:

  • Quickly pinpoint costly and slow queries and drill into precise execution details to address bottlenecks
  • Optimize the storage requirements by looking at the usage patterns. Identifying spikes in storage used and large files being loaded inefficiently
  • Monitor the credit usage and take necessary actions where cost can be optimized
  • Detect any misconfiguration and security threats and proactively resolve them
  • Monitor the performance of the data warehouse and understand if the warehouse can satisfy user requirements. Predict when to resize the warehouse
  • Know if replication is enabled, and track lag between instances to get early insights into problems with replication
  • Get alerts out-of-the-box from AIOps driven anomaly detection to rapidly detect deviations from historical patterns or fluctuations of key resources such as storage
  • Use rich dashboards, topology maps and reports to break down communication silos across the organization

Monitor Snowflake credit usage across your organization

eG Enterprise allows you to monitor your Snowflake costs in many ways with fine granularity, allowing you to ensure Snowflake credit usage is managed and understood. You can:

  • View and track the credits consumed when using resources, such as virtual warehouse is running, the cloud services layer is performing work, or serverless features are used.
  • Instantly access the credit usage for each service and for each session via the Detailed Diagnosis pane.
  • Monitor Snowflake Credit Usage by Pipes: Snowflake tracks the resource consumption of loads for all pipes in an account, with per-second/per-core granularity, as Snowpipe
  • Monitor Snowflake Credit Usage by Pipes: Snowflake tracks the resource consumption of loads for all pipes in an account, with per-second/per-core granularity, as Snowpipe actively queues and processes data files. Snowflake provides and manages the compute resources, automatically growing or shrinking capacity based on the current Snowpipe load. Accounts are charged based on their actual compute resource usage.
  • actively queues and processes data files. Snowflake provides and manages the compute resources, automatically growing or shrinking capacity based on the current Snowpipe load. Accounts are charged based on their actual compute resource usage.
  • eG Enterprise can also monitor cloud subscription and tenant accounts on AWS and Azure allowing you to consolidate cloud platform costs associated with Snowflake in a single console.

Improve Snowflake warehouse performance and ensure availability

  • Avoid queued queries caused by warehouse overload and monitor key query performance metrics in real-time
  • Proactively adjust warehouse sizes or clusters before they exceed capacity with proactive alerting and dynamic AIOps powered auto-baselining
  • Catch issues early and optimize slow query performance
  • eG Enterprise proactively monitors both Warehouse availability and the load on warehouses
  • Dedicated features also monitor and provide insights into Snowflake’s Automatic allowing you visibility on background re-clustering

Deep insight into queries and query workloads

Query performance is key to any database or data store. Many factors can affect the performance and throughput of queries like the load on the target database, query complexity, blocked queries, etc. If the query performance is not acceptable, it will directly and immediately impact the performance of customers’ applications using Snowflake database and in the worst case might disrupt business critical services.

It is important to monitor each database under the client’s Snowflake service account to provide insights into query workload. eG Enterprise monitors each Snowflake database and provides vital statistics related to query execution like average and maximum time taken for query execution, how long the queries are being queued before execution, for how long were

It is important to monitor each database under the client’s Snowflake service account to provide insights into query workload. eG Enterprise monitors each Snowflake database and provides vital statistics related to query execution like average and maximum time taken for query execution, how long the queries are being queued before execution, for how long were transactions blocked, etc. Administrators can look at these metrics and get to the root cause of degradation in query performance and clear out the blocked queries and transactions.

transactions blocked, etc. Administrators can look at these metrics and get to the root cause of degradation in query performance and clear out the blocked queries and transactions.

Single click detailed diagnostics allow instant visibility on key data such as the top queries by execution time, details of failed queries and more.


Observability on Snowflake data load and Snowpipe

Snowflake data loading is the process of copying and loading data or data sets from a source file, folder or application to a database. It is implemented by copying digital data from a source and loading the data to a data storage or processing utility.

Snowpipe is Snowflake's continuous data ingestion service. Snowpipe loads data within minutes after files are added to a stage and submitted for ingestion.

eG Enterprise allows you to proactively and continuously monitor data loading using COPY INTO (bulk loading) and pipes (continuous loading) within Snowflake. In the event of failures, detailed diagnosis information on error details is instantly available.


Monitor Snowflake tasks

When handling large and complex data pipelines with tasks or task trees, not all of them may execute SQL statements as expected and some may fail.

Snowflake includes a table function named TASK_HISTORY, which gives the task usage detail for your entire Snowflake account or a specific task. This TASK_HISTORY table function exists under the information schema where the task or task tree is created.

eG Enterprise automatically keep track of the Snowflake tasks and

eG Enterprise automatically keep track of the Snowflake tasks and give you visibility into their operation with proactive alerting on failures and instant access to data from TASK_HISTORY with no need to write or action bespoke SQL queries. Detailed diagnostics on the details of failed or skipped tasks are accessible in a single-click.

give you visibility into their operation with proactive alerting on failures and instant access to data from TASK_HISTORY with no need to write or action bespoke SQL queries. Detailed diagnostics on the details of failed or skipped tasks are accessible in a single-click.

Live and historical reporting allows you a comprehensive overview of task usage and success over longer periods.


Integrate Snowflake observability across the enterprise

eG Enterprise allows you to integrate your Snowflake monitoring into automated workflows and best-practice business workflows. Designed for users across the enterprise beyond the Snowflake Administrator including L1/L2 help desk operators, business management and cloud architects. eG Enterprise includes:

  • Live and historical reporting and capacity planning tools. Schedule and email reports automatically
  • Dedicated one-click dashboards for Snowflake
  • Built in configuration change tracking and user audit trails
  • A secure SOC 2 audited architecture with a wealth of security features.
  • RBAC (Role Based Access Controls) limit visibility and operations as required. Non-domain specialist help desk staff
  • RBAC (Role Based Access Controls) limit visibility and operations as required. Non-domain specialist help desk staff can have instant visibility on Snowflake and cloud issues without accessing the Snowflake of cloud service accounts, consoles, and portals
  • can have instant visibility on Snowflake and cloud issues without accessing the Snowflake of cloud service accounts, consoles, and portals
  • Help desk and ITSM tool integrations with ServiceNow, Autotask, JIRA and others allow you to formally track Snowflake issues
  • A PowerBI integration to assist business analytics
  • One-click root cause diagnosis technologies on dashboards avoid alarm storms and allow frontline help desk staff to differentiate between Snowflake and Cloud issues

Get detailed cloud and Snowflake usage and cost insights in a single platform

eG Innovations supports over 250+ technology stacks, including most database technologies and public clouds such as Microsoft Azure and Amazon AWS. eG Enterprise allows you to:

  • Track and compare the performance and costs of different database technologies
  • Evaluate the overall costs of your Snowflake and Cloud Provider usage – see the combined costs of running Snowflake on AWS or Azure
  • Proactively monitor user access and authentication
  • Proactively monitor user access and authentication technologies including Active Directory, Azure AD and AWS access
  • technologies including Active Directory, Azure AD and AWS access
  • Monitor technologies used in conjunction with Snowflake such as Kafka, Docker, SAP, Node.js and more
  • Monitor Cloud services such as Kubernetes (such as Azure AKS or AWS EKS), Networking and Load Balancing, and detect cloud outages and issues in the same console as your Snowflake stack
  • Troubleshoot, optimize and capacity plan Cloud storage services such as AWS S3 or Azure Blob Storage. Metric thresholds, dynamic baselining, and alerting are configured out-of-the-box without the need for manual intervention
  • Comprehensive suite of APM (Application Performance Monitoring) tools for observability on applications or services dependent on your Snowflake usage
  • Synthetic and RUM (Real User Monitoring) tools including protocol, full session and web application simulators allow the performance and behavior of Snowflake to be tested within the full context of applications and services depending on it
  • Transfer licenses across the technologies you monitor, with eG Enterprise you can monitor your on-prem Microsoft SQL Servers deployments before migration to Snowflake and continue to use the same product transferring after migration transferring licenses to avoid shelf-ware
  • A single interface with broad support for all major Database technologies allows you to monitor and benchmark Snowflake alongside other DB technologies to ensure you get the best cost and performance balance for your data workloads

Frequently asked questions (FAQs)
about Snowflake monitoring

Snowflake is designed through three main components:

Cloud services: Snowflake uses ANSI SQL for cloud services empowering users to optimize their data. It eliminates the need for manual data warehouse management and tuning.

Query processing (compute): The compute layer of Snowflake is made up of virtual cloud data warehouses that let you analyze data. Each Snowflake virtual warehouse is an independent cluster and they do not compete for computing resources nor affect the performance of each other — which means workload concurrency is never a problem.

Database storage: The database storage layer holds all data loaded into Snowflake, including structured and semi structured data. Snowflake automatically manages all parts of the data storage process, including organization, structure, metadata, file size, compression, and statistics.

See: Key Concepts & Architecture — Snowflake Documentation for more detail.

Snowflake is provided as a self-managed service that runs completely on cloud infrastructure. This means that all three layers of Snowflake’s architecture (storage, compute, and cloud services) are deployed and managed entirely on a selected cloud platform.

A Snowflake account can be hosted on any of the following cloud platforms:

  • Amazon Web Services (AWS)
  • Google Cloud Platform (GCP)
  • Microsoft Azure (Azure)

eG Enterprise monitors Snowflake in an agentless manner. A read-only monitoring user queries table views to monitor the health of the Snowflake databases.

Snowflake is a cloud-based relational database management system (RDBMS) that supports the SQL language. It is a columnar database, meaning it stores data in columns rather than rows, allowing for more efficient data compression and faster query performance. Snowflake also incorporates elements of NoSQL databases, such as flexible data modeling and semi-structured data storage, making it a hybrid of both traditional relational and NoSQL databases.

Snowflake is a data platform and data warehouse that supports the most common standardized version of SQL: ANSI. This means that all of the most common operations are usable within Snowflake. Snowflake also supports all of the operations that enable data warehousing operations, like create, update, insert, etc.

Snowpipe is Snowflake's continuous data ingestion service. Snowpipe loads data within minutes after files are added to a stage and submitted for ingestion. With Snowpipe's serverless compute model, Snowflake manages load capacity, ensuring optimal compute resources to meet demand.

A list of over 250+ stacks supported by eG Enterprise is available, see: End-to-End Monitoring: Applications, Cloud, Containers (eginnovations.com). All major, on-prem and cloud service, databases are supported as well as many database related products such as Redis. Information on some of the most popular options can be found via our Database Monitoring page, see: Top Database Monitoring Tools | eG Innovations.



Start your free trial or schedule a custom demo with an engineer

  • Works on cloud environments, hybrid cloud setups and on-premises deployments
  • Deploy eG Enterprise using our SaaS platform or on-premises
  • Suitable for monitoring applications, digital workspaces and IT infrastructures
Please Enter Valid Work Email
eG Enterprise