Esper - Java Event Stream Processor

The Esper engine has been developed to address the requirements of applications that analyze and react to events. Some typical examples of applications are:

What these applications have in common is the requirement to process events (or messages) in real-time or near real-time. This is sometimes referred to as complex event processing (CEP) and event stream analysis. Key considerations for these types of applications are throughput, latency and the complexity of the logic required.

The Esper engine was designed to make it easier to build and extend CEP applications.

Relational databases and the standard query language (SQL) are designed for applications in which most data is fairly static and complex queries are less frequent. Also, most databases store all data on disks (except for in-memory databases) and are therefore optimized for disk access.

To retrieve data from a database an application must issue a query. If an application need the data 10 times per second it must fire the query 10 times per second. This does not scale well to hundreds or thousands of queries per second.

Database triggers can be used to fire in response to database update events. However database triggers tend to be slow and often cannot easily perform complex condition checking and implement logic to react.

In-memory databases may be better suited to CEP applications then traditional relational database as they generally have good query performance. Yet they are not optimized to provide immediate, real-time query results required for CEP and event stream analysis.

The Esper engine works a bit like a database turned upside-down. Instead of storing the data and running queries against stored data, the Esper engine allows applications to store queries and run the data through. Response from the Esper engine is real-time when conditions occur that match queries. The execution model is thus continuous rather then only when a query is submitted.

Esper provides two principal methods or mechanisms to process events: event patterns and event stream queries.

Esper offers an event pattern language to specify expression-based event pattern matching. Underlying the pattern matching engine is a state machine implementation. This method of event processing matches expected sequences of presence or absence of events or combinations of events. It includes time-based correlation of events.

Esper also offers event stream queries that address the event stream analysis requirements of CEP applications. Event stream queries provide the windows, aggregation, joining and analysis functions for use with streams of events. These queries are following the EQL syntax. EQL has been designed for similarity with the SQL query language but differs from SQL in its use of views rather then tables. Views represent the different operations needed to structure data in an event stream and to derive data from an event stream.

Esper provides these two methods as alternatives through the same API.

Esper requires the following 3rd-party libraries at runtime:

Esper requires the following 3rd-party libraries at compile-time and for running the test suite:

An event is an immutable record of a past occurrence of an action or state change. Event properties capture the state information for an event. An event is represented by either a POJO (plain-old Java object), a java.util.Map or a XML document via org.w3c.dom.Node.

In Esper, an event can be represented by any of the following underlying Java objects:

Event properties capture the state information for an event. Event properties be simple as well as indexed, mapped and nested event properties. The table below outlines the different types of properties and their syntax in an event expression. This syntax allows statements to query deep JavaBean objects graphs, XML structures and Map events.

name

sensorId

name[index]

sensor[0]

name('key')

sensor('light')

name.nestedname

sensor.value

Combinations are also possible. For example, a valid combination could be person.address('home').street[0].

Dynamic (unchecked) properties are event properties that need not be known at statement compilation time. Such properties are resolved during runtime.

The idea behind dynamic properties is that for a given underlying event representation we don't always know all properties in advance. An underlying event may have additional properties that are not known at statement compilation time, that we want to query on. The concept is especially useful for events that represent rich, object-oriented domain models.

The syntax of dynamic properties consists of the property name and a question mark. Indexed, mapped and nested properties can also be dynamic properties:

name?

name[index]?

name('key')?

name?.nestedPropertyName

Dynamic properties always return the java.lang.Object type. Also, dynamic properties return a null value if the dynamic property does not exist on events processed at runtime.

As an example, consider an OrderEvent event that provides an "item" property. The "item" property is of type Object and holds a reference to an instance of either a Service or Product.

Assume that both Service and Product classes provide a property named "price". Via a dynamic property we can specify a query that obtains the price property from either object (Service or Product):

select item.price? from OrderEvent

As a second example, assume that the Service class contains a "serviceName" property that the Product class does not possess. The following query returns the value of the "serviceName" property for Service objects. It returns a null-value for Product objects that do not have the "serviceName" property:

select item.serviceName? from OrderEvent

Consider the case where OrderEvent has multiple implementation classes, some of which have a "timestamp" property. The next query returns the timestamp property of those implementations of the OrderEvent interface that feature the property:

select timestamp? from OrderEvent

The query as above returns a single column named "timestamp?" of type Object.

When dynamic properties are nested, then all properties under the dynamic property are also considered dynamic properties. In the below example the query asks for the "direction" property of the object returned by the "detail" dynamic property:

select detail?.direction from OrderEvent
// equivalent to 
select detail?.direction? from OrderEvent

The functions that are often useful in conjunction with dynamic properties are:

Dynamic event properties work with all event representations outlined next: Java objects, Map-based and XML DOM-based events.

Plain-old Java object events are object instances that expose event properties through JavaBeans-style getter methods. Events classes or interfaces do not have to be fully compliant to the JavaBean specification; however for the Esper engine to obtain event properties, the required JavaBean getter methods must be present.

Esper supports JavaBeans-style event classes that extend a superclass or implement one or more interfaces. Also, Esper event pattern and EQL statements can refer to Java interface classes and abstract classes.

Classes that represent events should be made immutable. As events are recordings of a state change or action that occurred in the past, the relevant event properties should not be changeable. However this is not a hard requirement and the Esper engine accepts events that are mutable as well.

The hashCode and equals methods do not need to be implemented. The implementation of these methods by a Java event class does not affect the behavior of the engine in any way.

Please see Chapter 10, Configuration on options for naming event types represented by Java object event classes.

public class EmployeeEvent {
	public String getFirstName();
	public Address getAddress(String type);
	public Employee getSubordinate(int index);
	public Employee[] getAllSubordinates();
}

every EmployeeEvent(firstName='myName')
every EmployeeEvent(address('home').streetName='Park Avenue')
every EmployeeEvent(subordinate[0].name='anotherName')
every EmployeeEvent(allSubordinates[1].name='thatName')
every EmployeeEvent(subordinate[0].address('home').streetName='Water Street')

select firstName, address('work'), subordinate[0].name, subordinate[1].name
from EmployeeEvent
where address('work').streetName = 'Park Ave'

select price from MyEvent

select NAME from MyEvent

select itemDesc from MyEvent

select q from MyEvent

select QN from MyEvent

select qn from MyEvent

select s from MyEvent

Event properties that are enumeration values can be compared by means of the Enumeration valueOf method on the java.lang.Enum class. An example could look as follows:

every MyEvent(enumProp=EnumClass.valueOf('ENUM_VALUE_1'))

Java classes that do not follow JavaBean conventions, such as legacy Java classes that expose public fields, or methods not following naming conventions, require additional configuration. Via configuration it is also possible to control case sensitivity in property name resolution. The relevant section in the chapter on configuration is Section 10.4.1.3, “Non-JavaBean and Legacy Java Event Classes”.

Events can also be represented by objects that implement the java.util.Map interface. Event properties of Map events are the values in the map accessible through the get method exposed by the java.util.Map interface.

The engine can process java.util.Map events via the sendEvent(Map map, String eventTypeAlias) method on the EPRuntime interface. Entries in the Map represent event properties. Keys must be of type java.util.String for the engine to be able to look up event property names specified by pattern or EQL statements.

Map event properties can be of any type. Map event properties that are Java application objects or that are of type java.util.Map offer additional power:

In order to use Map events, the event type name and property names and types must be made known to the engine via Configuration. Please see the examples in Section 10.4.2, “Events represented by java.util.Map”.

The code snippet below creates and processes a Map event. The example assumes the CarLocationUpdateEvent event type alias has been configured.

Map event = new HashMap();
event.put("carId", carId);
event.put("direction", direction);
epRuntime.sendEvent(event, "CarLocUpdateEvent");

The CarLocUpdateEvent can now be used in a statement:

select carId from CarLocUpdateEvent.win:time(1 min) where direction = 1

The engine can also query Java objects as values in a Map event via the nested property syntax. Thus Map events can be used to aggregate multiple data structures into a single event and query the composite information in a convenient way. The example below demonstrates a Map event with a transaction and an account object.

Map event = new HashMap();
event.put("txn", txn);
event.put("account", account);
epRuntime.sendEvent(event, "TxnEvent");

An example statement could look as follows.

select account.id, account.rate * txn.amount from TxnEvent.win:time(60 sec) group by account.id

Events can also be represented as org.w3c.dom.Node instances and send into the engine via the sendEvent method on EPRuntime. Please note that configuration is required for allowing the engine to map the event type alias to Node element names. See Chapter 10, Configuration.

Esper allows configuring XPath expressions as event properties. You can specify arbitrary XPath functions or expressions and provide a property name by which their result values will be available for use in expressions. For XML documents that follow an XML schema, Esper can load and interrogate your schema and validate event property names and types against the schema information.

Nested, mapped and indexed event properties are also supported in expressions against org.w3c.dom.Node events. Thus XML trees can conveniently be interrogated using the existing event property syntax for querying JavaBean objects, JavaBean object graphs or java.util.Map events.

Let's look at how a sample XML document could be queried, given the sample XML below.

<?xml version="1.0" encoding="UTF-8"?>
<Sensor>
	<ID>urn:epc:1:4.16.36<ID>
	<Observation Command="READ_PALLET_TAGS_ONLY">
		<ID>00000001<ID>
		<Tag>
			<ID>urn:epc:1:2.24.400<ID>
		</Tag>
		<Tag>
			<ID>urn:epc:1:2.24.401<ID>
		</Tag>
	</Observation>
</Sensor>

To configure the engine for processing Sensor documents, simply configure a SensorEvent event type alias for the Sensor element name via Configuration. Now the document can be queried as below.

select ID, Observation.ID, Observation.Command, Observation.Tag[0], countTags
from SensorEvent.win:time(30 sec)

The equivalent XPath expressions to each of the properties are listed below.

By specifying an event property such below:

nestedElement.mappedElement('key').indexedElement[1]

The equivalent XPath expression is as follows:

/simpleEvent/nestedElement/mappedElement[@id='key']/indexedElement[position() = 2]

The Esper processing model is continuous: Update listeners to statements receive updated data as soon as the engine processes events for that statement, according to the statement's choice of event streams, views, filters and output rates.

As outlined in Chapter 9, API Reference the interface for listeners is net.esper.client.UpdateListener. Implementations must provide a single update method that the engine invokes when results become available:

The engine provides statement results to update listeners by placing results in net.esper.event.EventBean instances. A typical listener implementation queries the EventBean instances via getter methods to obtain the statement-generated results.

The get method on the EventBean interface can be used to retrieve result columns by name. The property name supplied to the get method can also be used to query nested, indexed or array properties of object graphs as discussed in more detail in Chapter 2, Event Representations.

The getUnderlying method on the EventBean interface allows update listeners to obtain the underlying event object. For wildcard selects, the underlying event is the event object that was sent into the engine via the sendEvent method. For joins and select clauses with expressions, the underlying object implements java.util.Map.

In this section we look at the output of a very simple EQL statement. The statement selects an event stream without using a data window and without applying any filtering, as follows:

select * from Withdrawal

This statement selects all Withdrawal events. Every time the engine processes an event of type Withdrawal or any sub-type of Withdrawal, it invokes all update listeners, handing the new event to each of the statement's listeners.

The term insert stream denotes the new events arriving, and entering a data window or aggregation. The insert stream in this example is the stream of arriving Withdrawal events, and is posted to listeners as new events.

The diagram below shows a series of Withdrawal events 1 to 6 arriving over time. The number in parenthesis is the withdrawal amount, an event property that is used in the examples that discuss filtering.

Figure 3.1. Output example for a simple statement

The example statement above results in only new events and no old events posted by the engine to the statement's listeners.

A length window instructs the engine to only keep the last N events for a stream. The next statement applies a length window onto the Withdrawal event stream. The statement serves to illustrate the concept of data window and events entering and leaving a data window:

select * from Withdrawal.win:length(5)

The size of this statement's length window is five events. The engine enters all arriving Withdrawal events into the length window. When the length window is full, the oldest Withdrawal event is pushed out the window. The engine indicates to listeners all events entering the window as new events, and all events leaving the window as old events.

While the term insert stream denotes new events arriving, the term remove stream denotes events leaving a data window, or changing aggregation values. In this example, the remove stream is the stream of Withdrawal events that leave the length window, and such events are posted to listeners as old events.

The next diagram illustrates how the length window contents change as events arrive and shows the events posted to an update listener.

Figure 3.2. Output example for a length window

As before, all arriving events are posted as new events to listeners. In addition, when event W₁ leaves the length window on arrival of event W₆, it is posted as an old event to listeners.

Similar to a length window, a time window also keeps the most recent events up to a given time period. A time window of 5 seconds, for example, keeps the last 5 seconds of events. As seconds pass, the time window actively pushes the oldest events out of the window resulting in one or more old events posted to update listeners.

Note EQL supports optional istream and rstream keywords on select-clauses and on insert-into clauses. These instruct the engine to only forward events that enter or leave data windows, or select only current or prior aggregation values, i.e. the insert stream or the remove stream.

Filters to event streams allow filtering events out of a given stream before events enter a data window. The statement below shows a filter that selects Withdrawal events with an amount value of 200 or more.

select * from Withdrawal(amount>=200).win:length(5)

With the filter, any Withdrawal events that have an amount of less then 200 do not enter the length window and are therefore not passed to update listeners. Filters are discussed in more detail in Section 4.4.1, “Filter-based event streams” and Section 5.4, “Filter Expressions In Patterns”.

Figure 3.3. Output example for a statement with an event stream filter

The where-clause and having-clause in statements eliminate potential result rows at a later stage in processing, after events have been processed into a statement's data window or other views.

The next statement applies a where-clause to Withdrawal events. Where-clauses are discussed in more detail in Section 4.5, “Specifying Search Conditions: the Where Clause”.

select * from Withdrawal.win:length(5) where amount >= 200

The where-clause applies to both new events and old events. As the diagram below shows, arriving events enter the window however only events that pass the where-clause are handed to update listeners. Also, as events leave the data window, only those events that pass the conditions in the where-clause are posted to listeners as old events.

Figure 3.4. Output example for a statement with where-clause

The where-clause can contain complex conditions while event stream filters are more restrictive in the type of filters that can be specified. The next statement's where-clause applies the ceil function of the java.lang.Math Java library class in the where clause. The insert-into clause makes the results of the first statement available to the second statement:

insert into WithdrawalFiltered select * from Withdrawal where Math.ceil(amount) >= 200

select * from WithdrawalFiltered

In this section we explain the output model of statements employing a time window view and a time batch view.

3.5.1. Time Window

A time window is a moving window extending to the specified time interval into the past based on the system time. Time windows enable us to limit the number of events considered by a query, as do length windows.

As a practical example, consider the need to determine all accounts where the average withdrawal amount per account for the last 4 seconds of withdrawals is greater then 1000. The statement to solve this problem is shown below.

select account, avg(amount) 
from Withdrawal.win:time(4 sec) 
group by account
having amount > 1000

The next diagram serves to illustrate the functioning of a time window. For the diagram, we assume a query that simply selects the event itself and does not group or filter events.

select * from Withdrawal.win:time(4 sec)

The diagram starts at a given time t and displays the contents of the time window at t + 4 and t + 5 seconds and so on.

Figure 3.5. Output example for a statement with a time window

The activity as illustrated by the diagram:

1. At time t + 4 seconds an event W₁ arrives and enters the time window. The engine reports the new event to update listeners.

2. At time t + 5 seconds an event W₂ arrives and enters the time window. The engine reports the new event to update listeners.

3. At time t + 6.5 seconds an event W₃ arrives and enters the time window. The engine reports the new event to update listeners.

4. At time t + 8 seconds event W₁ leaves the time window. The engine reports the event as an old event to update listeners.

3.5.2. Time Batch

The time batch view buffers events and releases them every specified time interval in one update. Time windows control the evaluation of events, as does the length batch window.

The next diagram serves to illustrate the functioning of a time batch view. For the diagram, we assume a simple query as below:

select * from Withdrawal.win:time_batch(4 sec)

The diagram starts at a given time t and displays the contents of the time window at t + 4 and t + 5 seconds and so on.

Figure 3.6. Output example for a statement with a time batch view

The activity as illustrated by the diagram:

1. At time t + 1 seconds an event W₁ arrives and enters the batch. No call to inform update listeners occurs.

2. At time t + 3 seconds an event W₂ arrives and enters the batch. No call to inform update listeners occurs.

3. At time t + 4 seconds the engine processes the batched events and a starts a new batch. The engine reports events W₁ and W₂ to update listeners.

4. At time t + 6.5 seconds an event W₃ arrives and enters the batch. No call to inform update listeners occurs.

5. At time t + 8 seconds the engine processes the batched events and a starts a new batch. The engine reports the event W₃ as new data to update listeners. The engine reports the events W₁ and W₂ as old data (prior batch) to update listeners.

The engine posts events to UpdateListener implementations as net.esper.event.EventBean instances. The EventBean represents a row (event) in your continuous query's result set.

Use the iterator method on EPStatement statements to poll or read data out of statements, if you require read-based access to statement result sets. Statement iterators also return EventBean instances.

The EventBean interface offers property type metadata via the getEventType method returning an EventType. The EventType provides property name, property type and underlying type information. This information can be useful to dynamically interrogate query results. The underlying event that an EventBean represents can be obtained via the getUnderlying method. Please see Chapter 2, Event Representations for more information on different event underlying objects.

Consider a statement that returns the symbol, count of events per symbol and average price per symbol for tick events. Our sample statement may declare a fully-qualified Java class name as the event type: org.sample.StockTickEvent. Assume that this class exists and exposes a symbol property of type String, and a price property of type (Java primitive) double.

select symbol, avg(price) as avgprice, count(*) as mycount 
from org.sample.StockTickEvent 
group by symbol

The next table summarizes the property names and types as posted by the statement above:

eventBean.get("symbol")

eventBean.get("avgprice")

eventBean.get("mycount")

A code snippet out of a possible UpdateListener implementation to this statement may look as below:

String symbol = (String) newEvents[0].get("symbol");
Double price= (Double) newEvents[0].get("avgprice");
Long count= (Long) newEvents[0].get("mycount");

The engine supplies the boxed java.lang.Double and java.lang.Long types as property values rather then primitive Java types. This is because aggregated values can return a null value to indicate that no data is available for aggregation. Also, in a select statement that computes expressions, the underlying event objects to EventBean instances are of type java.util.Map.

Consider the next statement that specifies a wildcard selecting the same type of event:

select * from org.sample.StockTickEvent where price > 100

The property names and types provided by an EventBean query result row, as posted by the statement above are as follows:

eventBean.get("symbol")

eventBean.get("price")

As an alternative to querying individual event properties via the get methods, the getUnderlying method on EventBean returns the underlying object representing the query result. In the sample statement that features a wildcard-select, the underlying event object is of type org.sample.StockTickEvent:

StockTickEvent tick = (StockTickEvent) newEvents[0].getUnderlying();

The Event Query Language (EQL) is a SQL-like language with SELECT, FROM, WHERE, GROUP BY, HAVING and ORDER BY clauses. Streams replace tables as the source of data with events replacing rows as the basic unit of data. Since events are composed of data, the SQL concepts of correlation through joins, filtering and aggregation through grouping can be effectively leveraged. The INSERT INTO clause is recast as a means of forwarding events to other streams for further downstream processing. External data accessible through JDBC may be queried and joined with the stream data. Additional clauses such as the PATTERN and OUTPUT clauses are also available to provide the missing SQL language constructs specific to event processing.

EQL statements are used to derive and aggregate information from one or more streams of events, and to join or merge event streams. This section outlines EQL syntax. It also outlines the built-in views, which are the building blocks for deriving and aggregating information from event streams.

EQL statements contain definitions of one or more views. Similar to tables in an SQL statement, views define the data available for querying and filtering. Some views represent windows over a stream of events. Other views derive statistics from event properties, group events or handle unique event property values. Views can be staggered onto each other to build a chain of views. The Esper engine makes sure that views are reused among EQL statements for efficiency.

The built-in set of views is:

Esper can be extended by plugging-in custom developed views.

EQL queries are created and stored in the engine, and publish results as events are received by the engine or timer events occur that match the criteria specified in the query. Events can also be pulled from running EQL queries.

The select clause in an EQL query specifies the event properties or events to retrieve. The from-clause in an EQL query specifies the event stream definitions and stream names to use. The where-clause in an EQL query specifies search conditions that specify which event or event combination to search for. For example, the following statement returns the average price for IBM stock ticks in the last 30 seconds.

select avg(price) from StockTick.win:time(30 sec) where symbol='IBM'

EQL queries follow the below syntax. EQL queries can be simple queries or more complex queries. A simple select contains only a select clause and a single stream definition. Complex EQL queries can be build that feature a more elaborate select list utilizing expressions, may join multiple streams, may contain a where clause with search conditions and so on.

[insert into insert_into_def]
select select_list
from stream_def [as name] [, stream_def [as name]] [,...]
[where search_conditions]
[group by grouping_expression_list]
[having grouping_search_conditions]
[output output_specification]
[order by order_by_expression_list]

time-period : [day-part] [hour-part] [minute-part] [seconds-part] [milliseconds-part]

day-part : number ("days" | "day")
hour-part : number ("hours" | "hour")
minute-part : number ("minutes" | "minute" | "min")
seconds-part : number ("seconds" | "second" | "sec")
milliseconds-part : number ("milliseconds" | "millisecond" | "msec")

10 seconds
10 minutes 30 seconds
20 sec 100 msec
1 day 2 hours 20 minutes 15 seconds 110 milliseconds
0.5 minutes

// This comment extends to the end of the line.
// Two forward slashes with no whitespace between them begin such comments.

select * from MyEvent  // this is a slash-slash comment

// All of this text together is a valid statement.

/* This comment is a "slash-star" comment that spans multiple lines.
 * It begins with the slash-star sequence with no space between the '/' and '*' characters.
 * By convention, subsequent lines can begin with a star and are aligned, but this is 
 * not required.
 */		
select * from MyEvent  /* this also works */

Comments styles can also be mixed:

select field1, // first comment
  /* second comment*/  field2
  from MyEvent

The select clause is required in all EQL statements. The select clause can be used to select all properties via the wildcard *, or to specify a list of event properties and expressions. The select clause defines the event type (event property names and types) of the resulting events published by the statement, or pulled from the statement.

The select clause also offers optional istream and rstream keywords to control how events are posted to UpdateListener instances listening to the statement.

The syntax for the select clause is summarized below.

select [rstream | istream] * | expression_list ... 

select * from stream_def

select * from StockTick(symbol='IBM').win:time(30 sec).stat:uni('price')

select *, price * volume as pricevolume from StockTick(symbol='IBM')

select event_property [, event_property] [, ...] from stream_def

select count, stdev from StockTick(symbol='IBM').win:length(100).stat:uni('volume')

select expression [, expression] [, ...] from stream_def

select volume * price from StockTick.win:time_batch(30 sec)

select [event property | expression] as identifier [, ...]

select volume * price as volPrice from StockTick.win:length(100)

select rstream * from StockTick.win:time(30 sec)

The from clause is required in all EQL statements. It specifies one or more event streams. Each event stream can optionally be given a name by means of the as syntax.

from stream_def [as name] [, stream_def [as name]] [, ...]

The event stream definition stream_def as shown in the syntax above can consists of either a filter-based event stream definition or a pattern-based event stream definition.

For joins and outer joins, specify two or more event streams. Joins between pattern-based and filter-based event streams are also supported.

Esper supports joins against relational databases for access to historical or reference data as explained in Section 4.13, “Joining Relational Data via SQL”.

event_type ( [filter_criteria] ) [.view_spec] [.view_spec] [...]

select * from 
  org.esper.example.StockTick(symbol='IBM').win:length(100).stat:uni('volume') as volumeStats

select * from com.mypackage.myevents.RfidEvent

select * from RfidEvent

select * from org.myorg.rfid.IRfidReadable

select * from RfidEvent(category="Perishable")

select * from RfidEvent(MyRFIDLib.isInRange(x, y) or (x < 0 and y < 0))

select * from RfidEvent(zone=1, category=10)
...is equivalent to...
select * from RfidEvent(zone=1 and category=10)

mypackage.RfidEvent(x in [100:200], y in [0:100])

mypackage.RfidEvent(x between 100 and 200, y between 0 and 50)

mypackage.RfidEvent(x not in [0:100])

mypackage.RfidEvent(x not between 0 and 100)

mypackage.RfidEvent(category in ('Perishable', 'Container'))

mypackage.RfidEvent(category not in ('Household', 'Electrical'))

pattern [pattern_expression] [.view_spec] [.view_spec] [...]

select * from pattern [every tick=StockTickEvent or every trade=TradeEvent]

select tick.price as tickPrice, trade.price as tradePrice, 
       sum(tick.price) + sum(trade.price) as total
  from pattern [every tick=StockTickEvent or every trade=TradeEvent].win:time(30 sec)

select * from CarLocEvent.std:groupby('carId').win:length(4).
  std:groupby({'expressway', 'direction', 'segment'}).std:size()

The where clause is an optional clause in EQL statements. Via the where clause event streams can be joined and events can be filtered.

Comparison operators =, < , > , >=, <=, !=, <>, is null, is not null and logical combinations via and and or are supported in the where clause. The where clause can also introduce join conditions as outlined in Section 4.10, “Joining Event Streams”. Where-clauses can also contain expressions. Some examples are listed below.

...where fraud.severity = 5 and amount > 500
...where (orderItem.orderId is null) or (orderItem.class != 10)		 
...where (orderItem.orderId = null) or (orderItem.class <> 10)		 
...where itemCount / packageCount > 10