In large-scale online services, crucial metrics, a.k.a., key performance indicators (KPIs), are monitored periodically to check their running statuses. Generally, KPIs are aggregated along multiple dimensions and derived by complex calculations among fundamental metrics from the raw data. Once abnormal KPI values are observed, root cause analysis (RCA) can be applied to identify the reasons for anomalies, so that we can troubleshoot quickly. Recently, several automatic RCA techniques were proposed to localize the related dimensions (or a combination of dimensions) to explain the anomalies. However, their analyses are limited to the data on the abnormal metric and ignore the data of other metrics which may be also related to the anomalies, leading to imprecise or even incorrect root causes. To this end, we propose a cross-metric multi-dimensional root cause analysis method, named CMMD, which consists of two key components: 1) relationship modeling, which utilizes graph neural network (GNN) to model the unknown complex calculation among metrics and aggregation function among dimensions from historical data; 2) root cause localization, which adopts the genetic algorithm to efficiently and effectively dive into the raw data and localize the abnormal dimension(s) once the KPI anomalies are detected. Experiments on synthetic datasets, public datasets and online production environment demonstrate the superiority of our proposed CMMD method compared with baselines. Currently, CMMD is running as an online service in Microsoft Azure.
翻译:在大型在线服务中,对关键指标(a.k.a.a.),即关键业绩指标(KPIs)进行定期监测,以检查其运行状态。一般而言,KPI是按多个层面汇总的,并且从原始数据中基本指标的复杂计算中得出。一旦观察到异常的KPI值,就可以应用根源分析(RCA)来查明异常的原因,这样我们就能迅速排除麻烦。最近,提出了几项自动RCA技术,将相关层面(或多个层面的组合)本地化,以解释异常情况。然而,它们的分析仅限于异常指标的数据,而忽略其他指标的数据,而这些数据也可能与异常情况相关,导致不准确甚至不正确的根源原因。为此,我们建议采用跨度的多维根源分析方法,称为CMMD,由两个关键组成部分组成:(1) 关系建模,利用图形神经网络(GNNN)来模拟历史数据各层面之间未知的复杂计算;(2) 根本原因化,即采用遗传算法,以便高效和有效地向原始数据下潜入,从而将CMMR的原始数据与在线数据进行比较。