Evolving Markov Chains (EMCs)
Motivation
Understanding how real-world systems behave is a challenge. An important part of this challenge comes from their non-stationary nature, that is, they often exhibit time-varying properties. These properties can shift due to various internal and external factors, such as environmental conditions, operational changes, or wear and tear in physical systems. This non-stationarity complicates modeling and predicting system behavior, as assumptions of stability over time often do not hold. Consequently, tracking, adapting to, and anticipating these shifts is essential for effective monitoring and control in areas such as healthcare, industrial machinery, and environmental monitoring.
In this context, an important task is to discover and recognize the modes of a system during runtime. A mode represents a distinct behavioral state, and there could be fully/partially operational or failure modes. For instance, an industrial automation system may operate in a normal mode or encounter various anomalous modes, which may not always be fully known or predictable. Identifying these modes enables better diagnostics and decision-making in response to changing system conditions.
Approach
In this project, we formally define the Online Mode Discovery and Recognition (OMDR) problem, and propose Evolving Markov Chains (EMC) for it. EMC in an unsupervised online algorithm that models system behavior with Markov chains, and updates the transition matrix after each observation obtained from a target system. By tracking how the maintained Markov chain changes over time, the underlying modes of the system are discovered. By saving the models for discovered modes, a recurring mode can be recognized.
EMC is shown to be effective across a range of applications in diverse domains, including human activity recognition, electric motor condition monitoring, and eye-state detection from EEG signals.
Live Demo
Here is a live demo of how EMC works. In this case, there are three modes (Mode 0, Mode 1 and Mode 2) with different transition matrices. The sequence length and EMC’s parameters (lambda and tau) can be edited directly. When the run button is clicked, the sequence gets processed by EMC and live updates on the state diagram, the estimated transition matrix and the calculated Hellinger distance is visible.