zur Startseite

Masterarbeit

Adaptive Check Pointing in Speculative Parallel Complex Event Processing
Betreuer M. Sc. Ahmad Slo
Prüfer Prof. Dr. rer. nat. Dr. h. c. Kurt Rothermel
Ende
Beschreibung

Thesis Description

The tremendous increase in data volume and the need to interpret this data in realtime, to extract useful information have motivated many research communities to develop technologies that process such huge data online. Complex event processing (CEP) is one effective paradigm to process such stream of data.

Data parallel CEP is a well defined paradigm that processes such huge data stream, where each CEP operator is represented by three components, namely splitter, operator instances and merger. The splitter partitions the event stream into different windows of events. These windows are processed by different operator instances in parallel. The merger reorders the produced complex events before emitting them.

In distributed complex event processing systems, different sources of inconsistencies can affect the quality of the detected complex events. An instance of an inconsistency source is out of order events where the events arrive to the operator in out of time stamp order. Processing these events can lead to false positives and false negatives.

We already have developed a framework for the data-parallel CEP which speculatively processes the out of order events. To prevent the reprocessing of a window from the start, in case of wrong speculation, the operator’s state is check-pointed. Later, if an out of order is detected the state is restored to a previous consistent state. However, the check pointing impacts the system throughput if it is taken when it is not necessary. Currently, we are using a simple model that triggers the action to take a check point. In this thesis, we intend to develop different check pointing models that can maximize the system throughput.

Tasks

  • Define the parameters that might have an impact on the check pointing decision.
  • Propose different check pointing models that can be used by our framework. As an instance, the model can depend on the probability of speculative events in a window.
  • Implement the proposed check pointing models.
  • Integrate the developed models in our CEP framework.
  • Extensive evaluation with real world and synthetic data.
  • Document the developed concepts, algorithms and the evaluations in written form.
  • Present your results in VS colloquium.

Requirements

  • Good background in probability theory.
  • Very good programming knowledge in Java.
  • Good background in parallel and mutlithreaded programming.
Download the description