More Shoes for the Cobbler’s Children

When we hear the term ‘embedded analytics’, most people think of business intelligence. The concept of embedded analytics refers to the integration of analytic content and capabilities within a business process application. The business benefits of embedding analytics into a business process include increased visibility, more effective strategic planning and accelerated time to value.

Most of the talk around this topic focuses on business supply chains or sales processes, but of course embedded analytics does not have to be limited to the business. In fact, IT value streams are increasingly critical to the digital business services that are changing our world, and embedding analytics applies in this context as well.

This reminds me of making more shoes for the cobbler’s children (see endnote), since the line between business and IT monitoring is beginning to blur. Having a great business analytics tool may not be worth much if you cannot assure the performance of the underlying IT ecosystems.

When IT’s portfolio of monitoring tools becomes fragmented, with highly specialized monitors based on technology domains, the effects are similar to the business drivers for embedding analytics in business processes as suggested below:

…many of these organizations fall short because they look at business intelligence and analytics access as a separate application from operations, creating silos and gaps into the performance of their business. Business users go about their daily tasks, have a question, and open up a different application to look at a premade dashboard (or create their own) through self-service access. Toggling between applications and not having an integrated way of accessing operational and analytical data creates time delays and overall inefficiencies.

Embedded Analytics, An EMA White Paper

A typical response by the IT organization has been to create a consolidated performance monitoring dashboard, aggregating monitoring data into a ‘single pane of glass’. However, without embedding analytics, effectively using these dashboards requires a significant amount of interpretation by highly skilled staff.

The ‘analytics’ are also sometimes separated from the daily tasks associated with the IT value stream, which results in an overall lack of flow. The familiar results include alert fatigue, bridge calls, war rooms or (much worse) endless maintenance of rule bases, scripts or correlation engines.

Fortunately, we can use the IT4IT Reference Architecture and eG Enterprise to help us build some new shoes for the ‘Cobbler’s Children’. J

Embedding Analytics in a Detect-to-Correct Value Stream

An embedded analytics solution provides relevant information and analytical tools designed for the task at hand, which allows users to work more efficiently. The following will illustrate how eG Enterprise’s converged APM/  with embedded analytics can be used to provide a unified monitoring solution and increase the effectiveness and efficiency of the Detect-to-Correct Value Stream.

IT4IT’s Detect-to-Correct Value Stream

A detailed understanding of the Detect-to-Correct Value Stream (D2C) can be obtained from the IT4ITTM Forum, but for purposes of this post there are several things worth noting:

  • D2C is anchored by a service model which,
    • helps improve the understanding of the inter-relationships and inter-dependencies required to fix operational issues
    • provides business context
  • D2C helps align key activities associated with Event, Incident, Problem, Change and Configuration Management

D2C’s key activities are illustrated below.

Figure 1: Detect-to-Correct Key Activities

eG Enterprise, Unified Monitoring and Embedded Analytics

eG Enterprise provides a converged application and infrastructure monitoring solution that  enables you to manage the performance of your entire service delivery chain from a single console with fully automated, self-healing event correlation and alerting.

As IT infrastructures evolve in scale and complexity, it is no longer sufficient to use a silo-based approach for monitoring, in which each technology tier is monitored independently by a different tool. This universal monitoring technology provides visibility into every layer of every tier of an infrastructure—from hardware to applications, desktops to storage, virtualization to databases—all from a single pane of glass.

At the heart of eG Enterprise are dependency-based analytics that are embedded into the monitoring platform. These analytics automatically correlate measurements across every component of a digital business service, as well as through every layer of each component.

The monitor automatically discovers dynamic virtual resources, learns the norms of all measurements, and adapts data collection in real time based on operational events. This provides broad reach across every layer and tier, deep insights and automated diagnosis of performance issues.

Converged Application Performance Monitoring diagram

Figure 2: Converged Application & Infrastructure Performance Monitoring

It’s also important to note that once new tests and measurements are added to the monitor via an Integration Console, they are automatically available to the monitor users if needed and no rules, scripts or other maintenance of the correlation is needed.

End to End Event Correlation modelThis enables a broad range of staff to focus on adding self-healing correlation that triggers autonomous operations, without the maintenance burden of traditional monitoring solutions.

Finally, since these analytics are embedded into the monitor, they are automatically available in all user interfaces and dashboards.

Correlated alert windows, color-coded service topology diagrams and situationally aware dashboards allow all users to quickly identify cause versus effects, see business and IT impacts, and proactively identify issues before users are impacted.

New tests and measurements can be added with little to no impact on monitor users and automatically take advantage of the embedded analytics.

Business Benefits of Embedded Analytics

The D2C provides us with a business value proposition that is well suited for justifying unified monitoring with eG Enterprise.

D2C Critical Success Factor D2C Key Performance Indicator eG Enterprise Feature(s)
Operational Excellence Events: Increase in breadth and depth of monitoring endpoints, reduction of escalated events (via filtering/correlation/ automated resolution), reduction of false positives, and reduction of the number of security events that cause business disruption.
Incidents: Incident reduction, reduction of escalated Incidents, reduction of false positives, reduction in the total number of security-related Incidents.
Problems: Increase Problems identified, increase Problems eradicated.
Changes: Reduction of change-related outages, reduction of emergency changes, reduction of unplanned changes, and reduction of security vulnerabilities introduced during Change Management.
Knowledge: Increase Known Error availability (enrich Known Error database), increased usage.
  • Universal Agent/Data Collector
  • User Experience, Business Transaction, Application and Infrastructure monitoring reach and depth
  • Automated diagnosis
  • Predictive analytics, trending and reporting
  • Configuration & Change tracking and correlation
  • Fix History / Fix Feedback across every measurement
Customer Satisfaction OLA/SLA: Reduction of failed agreements.
Availability of critical business systems: Increase uptime, decrease MTTR, increase MTBF.
Performance(user experience) of critical business systems: Decrease user complaints.
Incidents: Increased rate of first call resolution.
Self-service: Increased success rate for user self-fix.
  • Services-orientation, service topology
  • Integration of all service components into OLA/SLA reporting
  • Integration with end user experience and business transaction tracing
  • Ability to publish alarm windows, dashboards and scheduled reports
  • Personalized views, multi-tenancy
Staff Effectiveness Events: Increase automatically remediated Events, increase the percentage of Events correlated to a business service.
Incidents: Reduction of re-opened Incidents, increase percentage of first call resolution, reduction in average time to close an Incident, increase automatically remediated Incidents, reduce average handling time, and reduce rejected Incidents.
Changes: Increase automatically remediated changes.
  • All events mapped to relevant services
  • Self-healing correlation; integration with other toolsets
  • Automated diagnosis
Business Alignment Cost: Increase percentage of time invested on business-critical services.
Services: Increase number of business services defined, decrease percentage of business-critical services, decrease number of CIs that are not linked to a business service, increase “quality of service” monitoring for internal and external business services.
SLA/SLO: Increase percentage of business-critical services with defined Service Level targets.
Security: Number of security-related outages to business- critical systems, number of security Incidents causing financial loss, business disruption, or public embarrassment, number of security Incidents resolved without business impact.
  • Unified, real-time view of performance across all stakeholders and all lifecycle stages
  • Use of Integration Console for identifying service dependencies
  • Integrated SLA/OLA reporting
  • Discovery of new applications, components and service topologies

Table: Mapping eG Features to D2C

The business benefits of embedded analytics (in a business context) are well known.

Toggling between applications and not having an integrated way of accessing operational and analytical data creates time delays and overall inefficiencies. People may not get the answers they need or might spend more time looking for information instead of being able to act on it. This creates a disjointed approach to information visibility and business value.

Embedded Analytics, An EMA White Pape

Thanks to movements such as DevOps, new sources of service management guidance such as VeriSMTM and a standard reference architecture for managing the business of IT (IT4ITTM), we are finally making shoes for the Cobblers Children.

While unified monitoring may not be ITSM’s silver bullet, it can offer IT a ‘new pair of shoes’.

Final Thoughts: A unified monitoring solution should do more than converge end user experience, application and infrastructure into a single pane of glass. Unified monitoring is most effective when it automatically provides a single source of actionable truth about the performance of an IT service. It’s what moves us away from diagnosis-related activities and is a more “unifying” activity than troubleshooting.

About eG Innovations

eG Innovations is a provider of performance management solutions. We pioneered unified monitoring in 2001, went on to develop end-to-end business service monitoring solutions, the industry’s first virtualization-aware automated diagnostics, and more recently, code-level application performance correlation with the announcement of our fully converged application and infrastructure monitoring platform.

eG Enterprise takes measurements at every layer of every component across end-to-end IT services, learns the norms of all measurements and, through patented analytics, automatically isolates which layer of which component is the source of a performance issue.

Today we accomplish this across any deployment model— public, private or hybrid cloud—and from code to bare metal.

 IT Service Management Monitoring Architecture illustration

  • Architecture and Patterns for IT: Service Management, Resource Planning, and Governance Making Shoes for the Cobbler’s Children
  • IT4IT Reference Architecture :http://www.opengroup.org/it4it/about