Chapter 14. Configuration

Esper engine configuration is entirely optional. Esper has a very small number of configuration parameters that can be used to simplify event pattern and EPL statements, and to tune the engine behavior to specific requirements. The Esper engine works out-of-the-box without configuration.

An application can supply configuration at the time of engine allocation using the Configuration class, and can also use XML files to hold configuration. Configuration can be changed at runtime via the ConfigurationOperations interface available from EPAdministrator via the getConfiguration method.

The difference between using a Configuration object and the ConfigurationOperations interface is that for the latter, all configuration including event types added through that interface are considered runtime configurations. This means they will be discarded when calling the initialize method on an EPServiceProvider instance.

14.1. Programmatic Configuration

An instance of com.espertech.esper.client.Configuration represents all configuration parameters. The Configuration is used to build an EPServiceProvider, which provides the administrative and runtime interfaces for an Esper engine instance.

You may obtain a Configuration instance by instantiating it directly and adding or setting values on it. The Configuration instance is then passed to EPServiceProviderManager to obtain a configured Esper engine.

Configuration configuration = new Configuration();
configuration.addEventType("PriceLimit", PriceLimit.class.getName());
configuration.addEventType("StockTick", StockTick.class.getName());
configuration.addImport("org.mycompany.mypackage.MyUtility");
configuration.addImport("org.mycompany.util.*");

EPServiceProvider epService = EPServiceProviderManager.getProvider("sample", configuration);

Note that Configuration is meant only as an initialization-time object. The Esper engine represented by an EPServiceProvider does not retain any association back to the Configuration.

The ConfigurationOperations interface provides runtime configuration options as further described in Section 13.3.7, “Runtime Configuration”. Through this interface applications can, for example, add new event types at runtime and then create new statements that rely on the additional configuration. The getConfiguration method on EPAdministrator allows access to ConfigurationOperations.

14.2. Configuration via XML File

An alternative approach to configuration is to specify a configuration in a XML file.

The default name for the XML configuration file is esper.cfg.xml. Esper reads this file from the root of the CLASSPATH as an application resource via the configure method.

Configuration configuration = new Configuration();		
configuration.configure();

The Configuration class can read the XML configuration file from other sources as well. The configure method accepts URL, File and String filename parameters.

Configuration configuration = new Configuration();		
configuration.configure("myengine.esper.cfg.xml");

14.3. XML Configuration File

Here is an example configuration file. The schema for the configuration file can be found in the etc folder and is named esper-configuration-4-0.xsd. It is also available online at http://www.espertech.com/schema/esper/esper-configuration-2.0.xsd so that IDE can fetch it automatically. The namespace used is http://www.espertech.com/schema/esper.

<?xml version="1.0" encoding="UTF-8"?>
<esper-configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns="http://www.espertech.com/schema/esper"
    xsi:schemaLocation="
http://www.espertech.com/schema/esper
http://www.espertech.com/schema/esper/esper-configuration-2.0.xsd">
  <event-type name="StockTick" class="com.espertech.esper.example.stockticker.event.StockTick"/>
  <event-type name="PriceLimit" class="com.espertech.esper.example.stockticker.event.PriceLimit"/>
  <auto-import import-name="org.mycompany.mypackage.MyUtility"/>
  <auto-import import-name="org.mycompany.util.*"/>
</esper-configuration>		

The example above is only a subset of the configuration items available. The next chapters outline the available configuration in greater detail.

14.4. Configuration Items

14.4.1. Events represented by Java Classes

14.4.1.1. Package of Java Event Classes

Via this configuration an application can make the Java package or packages that contain an application's Java event classes known to an engine. Thereby an application can simply refer to event types in statements by using the simple class name of each Java class representing an event type.

For example, consider an order-taking application that places all event classes in package com.mycompany.order.event. One Java class representing an event is the class OrderEvent. The application can simply issue a statement as follows to select OrderEvent events:

select * from OrderEvent

The XML configuration for defining the Java packages that contain Java event classes is:

<event-type-auto-name package-name="com.mycompany.order.event"/>

The same configuration but using the Configuration class:

Configuration config = new Configuration();
config.addEventTypeAutoName("com.mycompany.order.event");
// ... or ...
config.addEventTypeAutoName(MyEvent.getPackage().getName());

14.4.1.2. Event type name to Java class mapping

This configuration item can be used to allow event pattern statements and EPL statements to use an event type name rather then the fully qualified Java class name. Note that Java Interface classes and abstract classes are also supported as event types via the fully qualified Java class name, and an event type name can also be defined for such classes.

The example pattern statement below first shows a pattern that uses the name StockTick. The second pattern statement is equivalent but specifies the fully-qualified Java class name.

every StockTick(symbol='IBM')"
every com.espertech.esper.example.stockticker.event.StockTick(symbol='IBM')

The event type name can be listed in the XML configuration file as shown below. The Configuration API can also be used to programatically specify an event type name, as shown in an earlier code snippet.

<event-type name="StockTick" class="com.espertech.esper.example.stockticker.event.StockTick"/>

14.4.1.3. Non-JavaBean and Legacy Java Event Classes

Esper can process Java classes that provide event properties through other means then through JavaBean-style getter methods. It is not necessary that the method and member variable names in your Java class adhere to the JavaBean convention - any public methods and public member variables can be exposed as event properties via the below configuration.

A Java class can optionally be configured with an accessor style attribute. This attribute instructs the engine how it should expose methods and fields for use as event properties in statements.

Table 14.1. Accessor Styles

Style NameDescription
javabeanAs the default setting, the engine exposes an event property for each public method following the JavaBean getter-method conventions
publicThe engine exposes an event property for each public method and public member variable of the given class
explicitThe engine exposes an event property only for the explicitly configured public methods and public member variables

Using the public setting for the accessor-style attribute instructs the engine to expose an event property for each public method and public member variable of a Java class. The engine assigns event property names of the same name as the name of the method or member variable in the Java class.

For example, assuming the class MyLegacyEvent exposes a method named readValue and a member variable named myField, we can then use properties as shown.

select readValue, myField from MyLegacyEvent

Using the explicit setting for the accessor-style attribute requires that event properties are declared via configuration. This is outlined in the next chapter.

When configuring an engine instance from a XML configuration file, the XML snippet below demonstrates the use of the legacy-type element and the accessor-style attribute.

<event-type name="MyLegacyEvent" class="com.mycompany.mypackage.MyLegacyEventClass">
  <legacy-type accessor-style="public"/>
</event-type>

When configuring an engine instance via Configuration API, the sample code below shows how to set the accessor style.

Configuration configuration = new Configuration();
ConfigurationEventTypeLegacy legacyDef = new ConfigurationEventTypeLegacy();
legacyDef.setAccessorStyle(ConfigurationEventTypeLegacy.AccessorStyle.PUBLIC);
config.addEventType("MyLegacyEvent", MyLegacyEventClass.class.getName(), legacyDef);

EPServiceProvider epService = EPServiceProviderManager.getProvider("sample", configuration);

14.4.1.4. Specifying Event Properties for Java Classes

Sometimes it may be convenient to use event property names in pattern and EPL statements that are backed up by a given public method or member variable (field) in a Java class. And it can be useful to declare multiple event properties that each map to the same method or member variable.

We can configure properties of events via method-property and field-property elements, as the next example shows.

<event-type name="StockTick" class="com.espertech.esper.example.stockticker.event.StockTickEvent">
	<legacy-type accessor-style="javabean" code-generation="enabled">
		<method-property name="price" accessor-method="getCurrentPrice" />
		<field-property name="volume" accessor-field="volumeField" />
	</legacy-type>
</event-type>

The XML configuration snippet above declared an event property named price backed by a getter-method named getCurrentPrice, and a second event property named volume that is backed by a public member variable named volumeField. Thus the price and volume properties can be used in a statement:

select avg(price * volume) from StockTick

As with all configuration options, the API can also be used:

Configuration configuration = new Configuration();
ConfigurationEventTypeLegacy legacyDef = new ConfigurationEventTypeLegacy();
legacyDef.addMethodProperty("price", "getCurrentPrice");
legacyDef.addFieldProperty("volume", "volumeField");
config.addEventType("StockTick", StockTickEvent.class.getName(), legacyDef);

14.4.1.5. Turning off Code Generation

Esper employes the CGLIB library for very fast read access to event property values. For certain legacy Java classes it may be desirable to disable the use of this library and instead use Java reflection to obtain event property values from event objects.

In the XML configuration, the optional code-generation attribute in the legacy-type section can be set to disabled as shown next.

<event-type name="MyLegacyEvent" class="com.mycompany.package.MyLegacyEventClass">
	<legacy-type accessor-style="javabean" code-generation="disabled" />
</event-type>

The sample below shows how to configure this option via the API.

Configuration configuration = new Configuration();
ConfigurationEventTypeLegacy legacyDef = new ConfigurationEventTypeLegacy();
legacyDef.setCodeGeneration(ConfigurationEventTypeLegacy.CodeGeneration.DISABLED);
config.addEventType("MyLegacyEvent", MyLegacyEventClass.class.getName(), legacyDef);

14.4.1.6. Case Sensitivity and Property Names

By default the engine resolves Java event properties case sensitive. That is, property names in statements must match JavaBean-convention property names in name and case. This option controls case sensitivity per Java class.

In the configuration XML, the optional property-resolution-style attribute in the legacy-type element can be set to any of these values:

Table 14.2. Property Resolution Case Sensitivity Styles

Style NameDescription
case_sensitive (default)As the default setting, the engine matches property names for the exact name and case only.
case_insensitiveProperties are matched if the names are identical. A case insensitive search is used and will choose the first property that matches the name exactly or the first property that matches case insensitively should no match be found.
distinct_case_insensitiveProperties are matched if the names are identical. A case insensitive search is used and will choose the first property that matches the name exactly case insensitively. If more than one 'name' can be mapped to the property an exception is thrown.

The sample below shows this option in XML configuration, however the setting can also be changed via API:

<event-type name="MyLegacyEvent" class="com.mycompany.package.MyLegacyEventClass">
  <legacy-type property-resolution-style="case_insensitive"/>
</event-type>

14.4.1.7. Factory and Copy Method

The insert into clause and directly instantiate and populate your event object. By default the engine invokes the default constructor to instantiate an event object. To change this behavior, you may configure a factory method. The factory method is a method name or a class name plus a method name (in the format class.method) that returns an instance of the class.

The update clause can change event properties on an event object. For the purpose of maintaining consistency, the engine may have to copy your event object via serialization (implement the java.io.Serializable interface). If instead you do not want any copy operations to occur, or your application needs to control the copy operation, you may configure a copy method. The copy method is the name of a method on the event object that copies the event object.

The sample below shows this option in XML configuration, however the setting can also be changed via ConfigurationEventTypeLegacy:

<event-type name="MyLegacyEvent" class="com.mycompany.package.MyLegacyEventClass">
  <legacy-type factory-method="com.mycompany.myapp.MySampleEventFactory.createMyLegacyTypeEvent" copy-method="myCopyMethod"/>
</event-type>

The copy method should be a public method that takes no parameters and returns a new event object. The copy method may not be a static method and may not take parameters.

14.4.1.8. Start and End Timestamp

For use with date-time interval methods, for example, you may let the engine know which property of your class carries the start and end timestamp value.

The sample below shows this option in XML configuration, however the setting can also be changed via API. The sample sets the name of the property providing the start timestamp to startts and the name of the property providing the end timestamp endts:

<event-type name="MyLegacyEvent" class="com.mycompany.package.MyLegacyEventClass">
  <legacy-type start-timestamp-property-name="startts" end-timestamp-property-name="endts"/>
</event-type>

14.4.2. Events represented by java.util.Map

The engine can process java.util.Map events via the sendEvent(Map map, String eventTypeName) method on the EPRuntime interface. Entries in the Map represent event properties. Keys must be of type java.util.String for the engine to be able to look up event property names in pattern or EPL statements. Values can be of any type. JavaBean-style objects as values in a Map can be processed by the engine, and strongly-typed nested maps are also supported. Please see the Chapter 2, Event Representations section for details on how to use Map events with the engine.

Via configuration we provide an event type name for Map events for use in statements, and the event property names and types enabling the engine to validate properties in statements.

The below snippet of XML configuration configures an event named MyMapEvent.

<event-type name="MyMapEvent">
  <java-util-map>
    <map-property name="carId" class="int"/>
    <map-property name="carType" class="string"/>
    <map-property name="assembly" class="com.mycompany.Assembly"/>    
  </java-util-map>
</event-type>

This configuration defines the carId property of MyMapEvent events to be of type int, and the carType property to be of type java.util.String. The assembly property of the Map event will contain instances of com.mycompany.Assembly for the engine to query.

The valid types for the class attribute are listed in Section 14.5, “Type Names”. In addition, any fully-qualified Java class name that can be resolved via Class.forName is allowed.

You can also use the configuration API to configure Map event types, as the short code snippet below demonstrates:

Map<String, Object> properties = new Map<String, Object>();
properties.put("carId", "int");
properties.put("carType", "string");
properties.put("assembly", Assembly.class.getName());

Configuration configuration = new Configuration();
configuration.addEventType("MyMapEvent", properties);

For strongly-typed nested maps (maps-within-maps), the configuration API method addEventType can also used to define the nested types. The XML configuration does not provide the capability to configure nested maps.

Finally, here is a sample EPL statement that uses the configured MyMapEvent map event. This statement uses the chassisTag and numParts properties of Assembly objects in each map.

select carType, assembly.chassisTag, count(assembly.numParts) from MyMapEvent.win:time(60 sec)

A Map event type may also become a subtype of one or more supertypes that must also be Map event types. The java-util-map element provides an optional attribute supertype-names that accepts a comma-separated list of names of Map event types that are supertypes to the type:

<event-type name="AccountUpdate">
<java-util-map supertype-names="BaseUpdate, AccountEvent">
...

For initialization time configuration, the addMapSuperType method can be used to add Map hierarchy information. For runtime configuration, pass the supertype names to the addEventType method in ConfigurationOperations.

A Map event type may declare a start and end timestamp property name. The XML shown next instructs the engine that the startts property carries the event start timestamp and the endts property carries the event end timestamp:

<event-type name="AccountUpdate">
<java-util-map start-timestamp-property-name="startts" end-timestamp-property-name="endts">
...

14.4.3. Events represented by org.w3c.dom.Node

Via this configuration item the Esper engine can natively process org.w3c.dom.Node instances, i.e. XML document object model (DOM) nodes. Please see the Chapter 2, Event Representations section for details on how to use Node events with the engine.

Esper allows configuring XPath expressions as event properties. You can specify arbitrary XPath functions or expressions and provide a property name by which their result values will be available for use in expressions.

For XML documents that follow a XML schema, Esper can load and interrogate your schema and validate event property names and types against the schema information.

Nested, mapped and indexed event properties are also supported in expressions against org.w3c.dom.Node events. Thus XML trees can conveniently be interrogated using the existing event property syntax for querying JavaBean objects, JavaBean object graphs or java.util.Map events.

In the simplest form, the Esper engine only requires a configuration entry containing the root element name and the event type name in order to process org.w3c.dom.Node events:

<event-type name="MyXMLNodeEvent">
  <xml-dom root-element-name="myevent" />
</event-type>

You can also use the configuration API to configure XML event types, as the short example below demonstrates. In fact, all configuration options available through XML configuration can also be provided via setter methods on the ConfigurationEventTypeXMLDOM class.

Configuration configuration = new Configuration();
ConfigurationEventTypeXMLDOM desc = new ConfigurationEventTypeXMLDOM();
desc.setRootElementName("myevent");
desc.addXPathProperty("name1", "/element/@attribute", XPathConstants.STRING);
desc.addXPathProperty("name2", "/element/subelement", XPathConstants.NUMBER);
configuration.addEventType("MyXMLNodeEvent", desc);

The next example presents configuration options in a sample configuration entry.

<event-type name="AutoIdRFIDEvent">
  <xml-dom root-element-name="Sensor" schema-resource="data/AutoIdPmlCore.xsd" 
       default-namespace="urn:autoid:specification:interchange:PMLCore:xml:schema:1">
    <namespace-prefix prefix="pmlcore" 
       namespace="urn:autoid:specification:interchange:PMLCore:xml:schema:1"/>
    <xpath-property property-name="countTags" 
       xpath="count(/pmlcore:Sensor/pmlcore:Observation/pmlcore:Tag)" type="number"/>
  </xml-dom>
</event-type>

This example configures an event property named countTags whose value is computed by an XPath expression. The namespace prefixes and default namespace are for use with XPath expressions and must also be made known to the engine in order for the engine to compile XPath expressions. Via the schema-resource attribute we instruct the engine to load a schema file. You may also use schema-text instead to provide the actual text of the schema.

Here is an example EPL statement using the configured event type named AutoIdRFIDEvent.

select ID, countTags from AutoIdRFIDEvent.win:time(30 sec)

14.4.3.1. Schema Resource

The schema-resource attribute takes a schema resource URL or classpath-relative filename. The engine attempts to resolve the schema resource as an URL. If the schema resource name is not a valid URL, the engine attempts to resolve the resource from classpath via the ClassLoader.getResource method using the thread context class loader. If the name could not be resolved, the engine uses the Configuration class classloader. Use the schema-text attribute instead when it is more practical to provide the actual text of the schema.

By configuring a schema file for the engine to load, the engine performs these additional services:

  • Validates the event properties in a statement, ensuring the event property name matches an attribute or element in the XML

  • Determines the type of the event property allowing event properties to be used in type-sensitive expressions such as expressions involving arithmetic (Note: XPath properties are also typed)

  • Matches event property names to either element names or attributes

If no schema resource is specified, none of the event properties specified in statements are validated at statement creation time and their type defaults to java.lang.String. Also, attributes are not supported if no schema resource is specified and must thus be declared via XPath expression.

14.4.3.2. Explicit XPath Property

The xpath-property element adds explicitly-names event properties to the event type that are computed via an XPath expression. In order for the XPath expression to compile, be sure to specify the default-namespace attribute and use the namespace-prefix to declare namespace prefixes.

XPath expression properties are strongly typed. The type attribute allows the following values. These values correspond to those declared by javax.xml.xpath.XPathConstants.

  • number (Note: resolves to a double)

  • string

  • boolean

  • node

  • nodeset

In case you need your XPath expression to return a type other then the types listed above, an optional cast-to type can be specified. If specified, the operation firsts obtains the result of the XPath expression as the defined type (number, string, boolean) and then casts or parses the returned type to the specified cast-to-type. At runtime, a warning message is logged if the XPath expression returns a result object that cannot be casted or parsed.

The next line shows how to return a long-type property for an XPath expression that returns a string:

desc.addXPathProperty("name", "/element/sub", XPathConstants.STRING, "long");

The equivalent configuration XML is:

<xpath-property property-name="name"  xpath="/element/sub" type="string" cast="long"/>

See Section 14.5, “Type Names” for a list of cast-to type names.

14.4.3.3. Absolute or Deep Property Resolution

This setting indicates that when properties are compiled to XPath expressions that the compilation should generate an absolute XPath expression or a deep (find element) XPath expression.

For example, consider the following statement against an event type that is represented by a XML DOM document, assuming the event type GetQuote has been configured with the engine as a XML DOM event type:

select request, request.symbol from GetQuote

By default, the engine compiles the "request" property name to an XPath expression "/GetQuote/request". It compiles the nested property named "request.symbol" to an XPath expression "/GetQuote/request/symbol", wherein the root element node is "GetQuote".

By setting absolute property resolution to false, the engine compiles the "request" property name to an XPath expression "//request". It compiles the nested property named "request.symbol" to an XPath expression "//request/symbol". This enables these elements to be located anywhere in the XML document.

The setting is available in XML via the attribute resolve-properties-absolute.

The configuration API provides the above settings as shown here in a sample code:

ConfigurationEventTypeXMLDOM desc = new ConfigurationEventTypeXMLDOM();
desc.setRootElementName("GetQuote");
desc.setDefaultNamespace("http://services.samples/xsd");
desc.setRootElementNamespace("http://services.samples/xsd");
desc.addNamespacePrefix("m0", "http://services.samples/xsd");
desc.setResolvePropertiesAbsolute(false);
configuration.addEventType("GetQuote", desc);

14.4.3.4. XPath Variable and Function Resolver

If your XPath expressions require variables or functions, your application may provide the class name of an XPathVariableResolver and XPathFunctionResolver. At type initialization time the engine instantiates the resolver instances and provides these to the XPathFactory.

This example shows the API to set this configuration.

ConfigurationEventTypeXMLDOM desc = new ConfigurationEventTypeXMLDOM();
desc.setXPathFunctionResolver(MyXPathFunctionResolver.class.getName());
desc.setXPathVariableResolver(MyXPathVariableResolver.class.getName());

14.4.3.5. Auto Fragment

This option is for use when a XSD schema is provided and determines whether the engine automatically creates an event type when a property expression transposes a property that is a complex type according to the schema.

An example:

ConfigurationEventTypeXMLDOM desc = new ConfigurationEventTypeXMLDOM();
desc.setAutoFragment(false);

14.4.3.6. XPath Property Expression

By default Esper employs the built-in DOM walker implementation to evaluate XPath expressions, which is not namespace-aware.

This configuration setting, when set to true, instructs the engine to rewrite property expressions into XPath.

An example:

ConfigurationEventTypeXMLDOM desc = new ConfigurationEventTypeXMLDOM();
desc.setXPathPropertyExpr(true);

14.4.3.7. Event Sender Setting

By default an EventSender for a given XML event type validates the root element name for which the type has been declared against the one provided by the org.w3c.Node sent into the engine.

This configuration setting, when set to false, instructs an EventSender to not validate.

An example:

ConfigurationEventTypeXMLDOM desc = new ConfigurationEventTypeXMLDOM();
desc.setEventSenderValidatesRoot(false);

14.4.3.8. Start and End Timestamp

You may configure the name of the properties that provides the event start timestamp and the event end timestamp as part of the configuration.

An example that configures startts as the property name providing the start timestamp and endts as the property name providing the end timestamp:

ConfigurationEventTypeXMLDOM desc = new ConfigurationEventTypeXMLDOM();
desc.setStartTimestampPropertyName("startts");
desc.setEndTimestampPropertyName("endts");

14.4.4. Events represented by Plug-in Event Representations

As part of the extension API plug-in event representations allows an application to create new event types and event instances based on information available elsewhere. Please see Section 16.8, “Event Type And Event Object” for details.

The configuration examples shown next use the configuration API to select settings. All options are also configurable via XML, please refer to the sample configuration XML in file esper.sample.cfg.xml.

14.4.4.1. Enabling an Custom Event Representation

Use the method addPlugInEventRepresentation to enable a custom event representation, like this:

URI rootURI = new URI("type://mycompany/myproject/myname");
config.addPlugInEventRepresentation(rootURI, 
    MyEventRepresentation.class.getName(), null);

The type:// part of the URI is an optional convention for the scheme part of an URI.

If your event representation takes initialization parameters, these are passed in as the last parameter. Initialization parameters can also be stored in the configuration XML, in which case they are passed as an XML string to the plug-in class.

14.4.4.2. Adding Plug-in Event Types

To register event types that your plug-in event representation creates in advance of creating an EPL statement (either at runtime or at configuration time), use the method addPlugInEventType:

URI childURI = new URI("type://mycompany/myproject/myname/MyEvent");
configuration.addPlugInEventType("MyEvent", new URI[] {childURI}, null);

Your plug-in event type may take initialization parameters, these are passed in as the last parameter. Initialization parameters can also be stored in the configuration XML.

14.4.4.3. Setting Resolution URIs

The engine can invoke your plug-in event representation when an EPL statement is created with an event type name that the engine has not seen before. Plug-in event representations can resolve such names to an actual event type. In order to do this, you need to supply a list of resolution URIs. Use the method setPlugInEventTypeNameResolutionURIs, at runtime or at configuration time:

URI childURI = new URI("type://mycompany/myproject/myname");
configuration.setPlugInEventTypeNameResolutionURIs(new URI[] {childURI});

14.4.5. Class and package imports

Esper allows invocations of static Java library functions in expressions, as outlined in Section 9.1, “Single-row Function Reference”. This configuration item can be set to allow a partial rather than a fully qualified class name in such invocations. The imports work in the same way as in Java files, so both packages and classes can be imported.

select Math.max(priceOne, PriceTwo)
// via configuration equivalent to
select java.lang.Math.max(priceOne, priceTwo)

Esper auto-imports the following Java library packages if no other configuration is supplied. This list is replaced with any configuration specified in a configuration file or through the API.

  • java.lang.*

  • java.math.*

  • java.text.*

  • java.util.*

In a XML configuration file the auto-import configuration may look as below:

<auto-import import-name="com.mycompany.mypackage.*"/>
<auto-import import-name="com.mycompany.myapp.MyUtilityClass"/>

Here is an example of providing imports via the API:

Configuration config = new Configuration();
config.addImport("com.mycompany.mypackage.*");	// package import
config.addImport("com.mycompany.mypackage.MyLib");   // class import

14.4.6. Cache Settings for From-Clause Method Invocations

Method invocations are allowed in the from clause in EPL, such that your application may join event streams to the data returned by a web service, or to data read from a distributed cache or object-oriented database, or obtain data by other means. A local cache may be placed in front of such method invocations through the configuration settings described herein.

The LRU cache is described in detail in Section 14.4.8.6.1, “LRU Cache”. The expiry-time cache documentation can be found in Section 14.4.8.6.2, “Expiry-time Cache”

The next XML snippet is a sample cache configuration that applies to methods provided by the classes 'MyFromClauseLookupLib' and 'MyFromClauseWebServiceLib'. The XML and API configuration understand both the fully-qualified Java class name, as well as the simple class name:

<method-reference class-name="com.mycompany.MyFromClauseLookupLib">
  <expiry-time-cache max-age-seconds="10" purge-interval-seconds="10" ref-type="weak"/>
</method-reference> 	
<method-reference class-name="MyFromClauseWebServiceLib">
  <lru-cache size="1000"/>
</method-reference> 

14.4.7. Variables

Variables can be created dynamically in EPL via the create variable syntax but can also be configured at runtime and at configuration time.

A variable is declared by specifying a variable name, the variable type and an optional initialization value. The initialization value can be of the same or compatible type as the variable type, or can also be a String value that, when parsed, is compatible to the type declared for the variable.

In a XML configuration file the variable configuration may look as below. The Configuration API can also be used to configure variables.

<variable name="var_threshold" type="long" initialization-value="100"/>
<variable name="var_key" type="string"/>

Please find the list of valid values for the type attribute in Section 14.5, “Type Names”.

14.4.8. Relational Database Access

Esper has the capability to join event streams against historical data sources, such as a relational database. This section describes the configuration entries that the engine requires to access data stored in your database. Please see Section 5.13, “Accessing Relational Data via SQL” for information on the use of EPL queries that include historical data sources.

EPL queries that poll data from a relational database specify the name of the database as part of the EPL statement. The engine uses the configuration information described here to resolve the database name in the statement to database settings. The required and optional database settings are summarized below.

  • Database connections can be obtained via JDBC javax.xml.DataSource, via java.sql.DriverManager and via data source factory. Either one of these methods to obtain database connections is a required configuration.

  • Optionally, JDBC connection-level settings such as auto-commit, transaction isolation level, read-only and the catalog name can be defined.

  • Optionally, a connection lifecycle can be set to indicate to the engine whether the engine must retain connections or must obtain a new connection for each lookup and close the connection when the lookup is done (pooled).

  • Optionally, define a cache policy to allow the engine to retrieve data from a query cache, reducing the number of query executions.

Some of the settings can have important performance implications that need to be carefully considered in relationship to your database software, JDBC driver and runtime environment. This section attempts to outline such implications where appropriate.

The sample XML configuration file in the "etc" folder can be used as a template for configuring database settings. All settings are also available by means of the configuration API through the classes Configuration and ConfigurationDBRef.

14.4.8.1. Connections obtained via DataSource

This configuration causes Esper to obtain a database connection from a javax.sql.DataSource available from your JNDI provider.

The setting is most useful when running within an application server or when a JNDI directory is otherwise present in your Java VM. If your application environment does not provide an available DataSource, the next section outlines how to use Apache DBCP as a DataSource implementation with connection pooling options and outlines how to use a custom factory for DataSource implementations.

If your DataSource provides connections out of a connection pool, your configuration should set the collection lifecycle setting to pooled.

The snippet of XML below configures a database named mydb1 to obtain connections via a javax.sql.DataSource. The datasource-connection element instructs the engine to obtain new connections to the database mydb1 by performing a lookup via javax.naming.InitialContext for the given object lookup name. Optional environment properties for the InitialContext are also shown in the example.

<database-reference name="mydb1">
  <datasource-connection context-lookup-name="java:comp/env/jdbc/mydb">
    <env-property name="java.naming.factory.initial" value ="com.myclass.CtxFactory"/>
    <env-property name="java.naming.provider.url" value ="iiop://localhost:1050"/>
  </datasource-connection>
</database-reference>

To help you better understand how the engine uses this information to obtain connections, we have included the logic below.

if (envProperties.size() > 0) {
  initialContext = new InitialContext(envProperties);
}
else {
  initialContext = new InitialContext();
}
DataSource dataSource = (DataSource) initialContext.lookup(lookupName);
Connection connection = dataSource.getConnection();

In order to plug-in your own implementation of the DataSource interface, your application may use an existing JNDI provider as provided by an application server if running in a J2EE environment.

In case your application does not have an existing JNDI implementation to register a DataSource to provide connections, you may set the java.naming.factory.initial property in the configuration to point to your application's own implementation of the javax.naming.spi.InitialContextFactory interface that can return your application DataSource though the javax.naming.Context provided by the factory implementation. Please see Java Naming and Directory Interface (JNDI) API documentation for further information.

14.4.8.2. Connections obtained via DataSource Factory

This section describes how to use Apache Commons Database Connection Pooling (Apache DBCP) with Esper. We also explain how to provide a custom application-specific DataSource factory if not using Apache DBCP.

If your DataSource provides connections out of a connection pool, your configuration should set the collection lifecycle setting to pooled.

Apache DBCP provides comprehensive means to test for dead connections or grow and shrik a connection pool. Configuration properties for Apache DBCP can be found at Apache DBCP configuration. The listed properties are passed to Apache DBCP via the properties list provided as part of the Esper configuration.

The snippet of XML below is an example that configures a database named mydb3 to obtain connections via the pooling DataSource provided by Apache DBCP BasicDataSourceFactory.

The listed properties are passed to DBCP to instruct DBCP how to manage the connection pool. The settings below initialize the connection pool to 2 connections and provide the validation query select 1 from dual for DBCP to validate a connection before providing a connection from the pool to Esper:

<database-reference name="mydb3">
  <!-- For a complete list of properties see Apache DBCP. -->
  <datasourcefactory-connection class-name="org.apache.commons.dbcp.BasicDataSourceFactory">	
    <env-property name="username" value ="myusername"/>
    <env-property name="password" value ="mypassword"/>
    <env-property name="driverClassName" value ="com.mysql.jdbc.Driver"/>
    <env-property name="url" value ="jdbc:mysql://localhost/test"/>
    <env-property name="initialSize" value ="2"/>
    <env-property name="validationQuery" value ="select 1 from dual"/>
  </datasourcefactory-connection>
  <connection-lifecycle value="pooled"/>
</database-reference>

The same configuration options provided through the API:

Properties props = new Properties();
props.put("username", "myusername");
props.put("password", "mypassword");
props.put("driverClassName", "com.mysql.jdbc.Driver");
props.put("url", "jdbc:mysql://localhost/test");
props.put("initialSize", 2);
props.put("validationQuery", "select 1 from dual");

ConfigurationDBRef configDB = new ConfigurationDBRef();
// BasicDataSourceFactory is an Apache DBCP import
configDB.setDataSourceFactory(props, BasicDataSourceFactory.class.getName());
configDB.setConnectionLifecycleEnum(ConfigurationDBRef.ConnectionLifecycleEnum.POOLED);

Configuration configuration = new Configuration();;
configuration.addDatabaseReference("mydb3", configDB);

Apache Commons DBCP is a separate download and not provided as part of the Esper distribution. The Apache Commons DBCP jar file requires the Apache Commons Pool jar file.

Your application can provide its own factory implementation for DataSource instances: Set the class name to the name of the application class that provides a public static method named createDataSource which takes a single Properties object as parameter and returns a DataSource implementation. For example:

configDB.setDataSourceFactory(props, MyOwnDataSourceFactory.class.getName());
...
class MyOwnDataSourceFactory {
  public static DataSource createDataSource(Properties properties) {
    return new MyDataSourceImpl(properties);
  }
}

14.4.8.3. Connections obtained via DriverManager

The next snippet of XML configures a database named mydb2 to obtain connections via java.sql.DriverManager. The drivermanager-connection element instructs the engine to obtain new connections to the database mydb2 by means of Class.forName and DriverManager.getConnection using the class name, URL and optional username, password and connection arguments.

<database-reference name="mydb2">
  <drivermanager-connection class-name="my.sql.Driver" 
        url="jdbc:mysql://localhost/test?user=root&amp;password=mypassword" 
        user="myuser" password="mypassword">
    <connection-arg name="user" value ="myuser"/>
    <connection-arg name="password" value ="mypassword"/>
    <connection-arg name="somearg" value ="someargvalue"/>
  </drivermanager-connection>
</database-reference>

The username and password are shown in multiple places in the XML only as an example. Please check with your database software on the required information in URL and connection arguments.

14.4.8.4. Connections-level settings

Additional connection-level settings can optionally be provided to the engine which the engine will apply to new connections. When the engine obtains a new connection, it applies only those settings to the connection that are explicitly configured. The engine leaves all other connection settings at default values.

The below XML is a sample of all available configuration settings. Please refer to the Java API JavaDocs for java.sql.Connection for more information to each option or check the documentation of your JDBC driver and database software.

<database-reference name="mydb2">
... configure data source or driver manager settings...
  <connection-settings auto-commit="true" catalog="mycatalog" 
      read-only="true" transaction-isolation="1" />
</database-reference>

The read-only setting can be used to indicate to your database engine that SQL statements are read-only. The transaction-isolation and auto-commit help you database software perform the right level of locking and lock release. Consider setting these values to reduce transactional overhead in your database queries.

14.4.8.5. Connections lifecycle settings

By default the engine retains a separate database connection for each started EPL statement. However, it is possible to override this behavior and require the engine to obtain a new database connection for each lookup, and to close that database connection after the lookup is completed. This often makes sense when you have a large number of EPL statements and require pooling of connections via a connection pool.

In the pooled setting, the engine obtains a database connection from the data source or driver manager for every query, and closes the connection when done, returning the database connection to the pool if using a pooling data source.

In the retain setting, the engine retains a separate dedicated database connection for each statement and does not close the connection between uses.

The XML for this option is below. The connection lifecycle allows the following values: pooled and retain.

<database-reference name="mydb2">
... configure data source or driver manager settings...
    <connection-lifecycle value="pooled"/>
</database-reference>

14.4.8.6. Cache settings

Cache settings can dramatically reduce the number of database queries that the engine executes for EPL statements. If no cache setting is specified, the engine does not cache query results and executes a separate database query for every event.

Caches store the results of database queries and make these results available to subsequent queries using the exact same query parameters as the query for which the result was stored. If your query returns one or more rows, the cache keep the result rows of the query keyed to the parameters of the query. If your query returns no rows, the cache also keeps the empty result. Query results are held by a cache until the cache entry is evicted. The strategies available for evicting cached query results are listed next.

14.4.8.6.1. LRU Cache

The least-recently-used (LRU) cache is configured by a maximum size. The cache discards the least recently used query results first once the cache reaches the maximum size.

The XML configuration entry for a LRU cache is as below. This entry configures an LRU cache holding up to 1000 query results.

<database-reference name="mydb">
... configure data source or driver manager settings...
    <lru-cache size="1000"/>
</database-reference>
14.4.8.6.2. Expiry-time Cache

The expiry time cache is configured by a maximum age in seconds, a purge interval and an optional reference type. The cache discards (on the get operation) any query results that are older then the maximum age so that stale data is not used. If the cache is not empty, then every purge interval number of seconds the engine purges any expired entries from the cache.

The XML configuration entry for an expiry-time cache is as follows. The example configures an expiry time cache in which prior query results are valid for 60 seconds and which the engine inspects every 2 minutes to remove query results older then 60 seconds.

<database-reference name="mydb">
... configure data source or driver manager settings...
    <expiry-time-cache max-age-seconds="60" purge-interval-seconds="120" />
</database-reference>

By default, the expiry-time cache is backed by a java.util.WeakHashMap and thus relies on weak references. That means that cached SQL results can be freed during garbage collection.

Via XML or using the configuration API the type of reference can be configured to not allow entries to be garbage collected, by setting the ref-type property to hard:

<database-reference name="mydb">
... configure data source or driver manager settings...
    <expiry-time-cache max-age-seconds="60" purge-interval-seconds="120" ref-type="hard"/>
</database-reference>

The last setting for the cache reference type is soft: This strategy allows the garbage collection of cache entries only when all other weak references have been collected.

14.4.8.7. Column Change Case

This setting instructs the engine to convert to lower- or uppercase any output column names returned by your database system. When using Oracle relational database software, for example, column names can be changed to lowercase via this setting.

A sample XML configuration entry for this setting is:

<column-change-case value="lowercase"/>

14.4.8.8. SQL Types Mapping

By providing a mapping of SQL types (java.sql.Types) to Java built-in types your code can avoid using sometimes awkward default database types and can easily change the way Esper returns Java types for columns returned by a SQL query.

The mapping maps a constant as defined by java.sql.Types to a Java built-in type of any of the following Java type names: String, BigDecimal, Boolean, Byte, Short, Int, Long, Float, Double, ByteArray, SqlDate, SqlTime, SqlTimestamp. The Java type names are not case-sensitive.

A sample XML configuration entry for this setting is shown next. The sample maps Types.NUMERIC which is a constant value of 2 per JDBC API to the Java int type.

<sql-types-mapping sql-type="2" java-type="int" />

14.4.8.9. Metadata Origin

This setting controls how the engine retrieves SQL statement metadata from JDBC prepared statements.

Table 14.3. Syntax and results of aggregate functions

OptionDescription
default

By default, the engine detects the driver name and queries prepared statement metadata if the driver is not an Oracle database driver. For Oracle drivers, the engine uses lexical analysis of the SQL statement to construct a sample SQL statement and then fires that statement to retrieve statement metadata.

metadata

The engine always queries prepared statement metadata regardless of the database driver used.

sample

The engine always uses lexical analysis of the SQL statement to construct a sample SQL statement, and then fires that statement to retrieve statement metadata.

14.4.9. Engine Settings related to Concurrency and Threading

14.4.9.1. Preserving the order of events delivered to listeners

In multithreaded environments, this setting controls whether dispatches of statement result events to listeners preserve the ordering in which a statement processes events. By default the engine guarantees that it delivers a statement's result events to statement listeners in the order in which the result is generated. This behavior can be turned off via configuration as below.

The next code snippet shows how to control this feature:

Configuration config = new Configuration();
config.getEngineDefaults().getThreading().setListenerDispatchPreserveOrder(false);
engine = EPServiceProviderManager.getDefaultProvider(config);

And the XML configuration file can also control this feature by adding the following elements:

<engine-settings>
  <defaults>
    <threading>
      <listener-dispatch preserve-order="true" timeout-msec="1000" locking="spin"/>
    </threading>
  </defaults>
</engine-settings>

As discussed, by default the engine can temporarily block another processing thread when delivering result events to listeners in order to preserve the order in which results are delivered to a given statement. The maximum time the engine blocks a thread can also be configured, and by default is set to 1 second.

As such delivery locks are typically held for a very short amount of time, the default blocking technique employs a spin lock (There are two techniques for implementing blocking; having the operating system suspend the thread until it is awakened later or using spin locks). While spin locks are CPU-intensive and appear inefficient, a spin lock can be more efficient than suspending the thread and subsequently waking it up, especially if the lock in question is held for a very short time. That is because there is significant overhead to suspending and rescheduling a thread.

The locking technique can be changed to use a blocking strategy that suspends the thread, by means of setting the locking property to 'suspend'.

14.4.9.2. Preserving the order of events for insert-into streams

In multithreaded environments, this setting controls whether statements producing events for other statements via insert-into preserve the order of delivery within the producing and consuming statements, allowing statements that consume other statement's events to behave deterministic in multithreaded applications, if the consuming statement requires such determinism. By default, the engine makes this guarantee (the setting is on).

Take, for example, an application where a single statement (S1) inserts events into a stream that another statement (S2) further evaluates. A multithreaded application may have multiple threads processing events into statement S1. As statement S1 produces events for consumption by statement S2, such results may need to be delivered in the exact order produced as the consuming statement may rely on the order received. For example, if the first statement counts the number of events, the second statement may employ a pattern that inspects counts and thus expect the counts posted by statement S1 to continuously increase by 1 even though multiple threads process events.

The engine may need to block a thread such that order of delivery is maintained, and statements that require order (such as pattern detection, previous and prior functions) receive a deterministic order of events. The settings available control the blocking technique and parameters. As described in the section immediately prior, the default blocking technique employs spin locks per statement inserting events for consumption, as the locks in questions are typically held a very short time. The 'suspend' blocking technique can be configured and a timeout value can also defined.

The XML configuration file may change settings via the following elements:

<engine-settings>
  <defaults>
    <threading>
      <insert-into-dispatch preserve-order="true" timeout-msec="100" locking="spin"/>
    </threading>
  </defaults>
</engine-settings>

14.4.9.3. Internal Timer Settings

This option can be used to disable the internal timer thread and such have the application supply external time events, as well as to set a timer resolution.

The next code snippet shows how to disable the internal timer thread via the configuration API:

Configuration config = new Configuration();
  config.getEngineDefaults().getThreading().setInternalTimerEnabled(false);

This snippet of XML configuration leaves the internal timer enabled (the default) and sets a resolution of 200 milliseconds (the default is 100 milliseconds):

<engine-settings>
  <defaults>
    <threading>
      <internal-timer enabled="true" msec-resolution="200"/>
    </threading>
  </defaults>
</engine-settings>

We recommend that when disabling the internal timer, applications send an external timer event setting the start time before creating statements, such that statement start time is well-defined.

14.4.9.4. Advanced Threading Options

The settings described herein are for enabling advanced threading options for inbound, outbound, timer and route executions.

Take the next snippet of XML configuration as an example. It configures all threading options to 2 threads, which may not be suitable to your application, however demonstrates the configuration:

<engine-settings>
  <defaults>
    <threading>
      <threadpool-inbound enabled="true" num-threads="2"/>
      <threadpool-outbound enabled="true" num-threads="2" capacity="1000"/>
      <threadpool-timerexec enabled="true" num-threads="2"/>
      <threadpool-routeexec enabled="true" num-threads="2"/>
    </threading>
  </defaults>
</engine-settings>

By default, queues are unbound and backed by java.util.concurrent.LinkedBlockingQueue. The optional capacity attribute can be set to instruct the threading option to configure a capacity-bound queue with a sender-wait (blocking put) policy, backed ArrayBlockingQueue.

This example uses the API for configuring inbound threading :

Configuration config = new Configuration();
config.getEngineDefaults().getThreading().setThreadPoolInbound(true);
config.getEngineDefaults().getThreading().setThreadPoolInboundNumThreads(2);

With a bounded work queue, the queue size and pool size should be tuned together. A large queue coupled with a small pool can help reduce memory usage, CPU usage, and context switching, at the cost of potentially constraining throughput.

14.4.9.5. Engine Fair Locking

By default Esper configures the engine-level lock without fair locking. The engine-level lock coordinates event processing threads (threads that send events) with threads that perform administrative functions (threads that start or destroy statements, for example). A fair lock is generally less performing that an unfair lock thus the default configuration is an unfair lock.

If your application is multi-threaded and multiple threads sends events without gaps and if the per-event processing time is significant, then configuring a fair lock can help prioritize administrative functions. Administrative functions exclude event-processing threads until the administrative function completed. You may need to set this flag to prevent lock starvation to perform an administrative function in the face of concurrent event processing. Please consult the Java API documentation under ReentrantReadWriteLock and Fair Mode for more information.

The XML configuration to enable fair locking, which is disabled by default, is as follows:

<engine-settings>
  <defaults>
    <threading engine-fairlock="true"/>
  </defaults>
</engine-settings>

The API to change the setting:

Configuration config = new Configuration();
config.getEngineDefaults().getThreading().setEngineFairlock(true);

14.4.10. Engine Settings related to Event Metadata

14.4.10.1. Java Class Property Names, Case Sensitivity and Accessor Style

The engine-wide settings discussed here are used when you want to control case sensitivity or accessor style for all event classes as a default. The two settings are found under class-property-resolution under event-meta in the XML configuration.

To control the case sensitivity as discussed in Section 14.4.1.6, “Case Sensitivity and Property Names”, add the style attribute in the XML configuration to set a default case sensitivity applicable to all event classes unless specifically overridden by class-specific configuration. The default case sensitivity is case_sensitive (case sensitivity turned on).

To control the accessor style as discussed in Section 14.4.1.3, “Non-JavaBean and Legacy Java Event Classes”, add the accessor-style attribute in the XML configuration to set a default accessor style applicable to all event classes unless specifically overridden by class-specific configuration. The default accessor style is javabean JavaBean accessor style.

The next code snippet shows how to control this feature via the API:

Configuration config = new Configuration();
config.getEngineDefaults().getEventMeta().setClassPropertyResolutionStyle(
    Configuration.PropertyResolutionStyle.CASE_INSENSITIVE);
config.getEngineDefaults().getEventMeta().setDefaultAccessorStyle(
    ConfigurationEventTypeLegacy.AccessorStyle.PUBLIC);

14.4.11. Engine Settings related to View Resources

14.4.11.1. Sharing View Resources between Statements

The engine by default attempts to optimize resource usage and thus re-uses or shares views between statements that declare same views. However, in multi-threaded environments, this can lead to reduced concurrency as locking for shared view resources must take place. Via this setting this behavior can be turned off for higher concurrency in multi-threaded processing.

The next code snippet outlines the API to turn off view resource sharing between statements:

Configuration config = new Configuration();
config.getEngineDefaults().getViewResources().setShareViews(false);

14.4.11.2. Configuring Multi-Expiry Policy Defaults

By default, when combining multiple data window views, Esper applies an intersection of the data windows unless the retain-union keyword is provided which instructs to apply an union. The setting described herein may be used primarily for backward compatibility to instruct that intersection should not be the default.

Here is a sample statement that specifies multiple expiry policies:

select * from MyEvent.std:unique(price).std:unique(quantity)

By default Esper applies intersection as described in Section 5.4.4, “Multiple Data Window Views”.

Here is the setting to allow multiple data windows without the intersection default:

Configuration config = new Configuration();
config.getEngineDefaults().getViewResources().setAllowMultipleExpiryPolicies(true);

When setting this option to true, and when using multiple data window views for a given stream, the behavior is as follows: The top-most data window receives an insert stream of events. It passes each insert stream event to each further data window view in the chain. Each data window view may remove events according to its expiry policy. Such remove stream events are only passed to data window views further in the chain, and are not made available to data window views earlier in the chain.

It is recommended to leave the default setting at false.

14.4.12. Engine Settings related to Logging

14.4.12.1. Execution Path Debug Logging

By default, the engine does not produce debug output for the event processing execution paths even when Log4j or Logger configurations have been set to output debug level logs. To enable debug level logging, set this option in the configuration as well as in your Log4j configuration file.

Statement-level processing information can be output via the @Audit annotation, please see Section 15.3.1, “@Audit Annotation”.

When debug-level logging is enabled by setting the flag as below and by setting DEBUG in the Log4j configuration file, then the timer processing may produce extensive debug output that you may not want to have in the log file. The timer-debug setting in the XML or via API as below disables timer debug output which is enabled by default.

The API to use to enable debug logging and disable timer event output is shown here:

Configuration config = new Configuration();
config.getEngineDefaults().getLogging().setEnableExecutionDebug(true);
config.getEngineDefaults().getLogging().setEnableTimerDebug(false);

Note: this is a configuration option that applies to all engine instances of a given Java module or VM.

The XML snippet is:

<esper-configuration>
  <engine-settings>
    <defaults>
      <logging>
        <execution-path enabled="true"/>
        <timer-debug enabled="false"/>
    </logging>
    </defaults>
  </engine-settings>
</esper-configuration>

14.4.12.2. Query Plan Logging

By default, the engine does not produce query plan output unless logging at debug-level. To enable query plan logging, set this option in the configuration. When enabled, the engine reports, at INFO level, any query plans under the log name com.espertech.esper.queryplan.

The API to use to enable query plan logging is shown here:

Configuration config = new Configuration();
config.getEngineDefaults().getLogging().setEnableQueryPlan(true);

The XML snippet is:

<esper-configuration>
  <engine-settings>
    <defaults>
      <logging>
        <query-plan enabled="true"/>
    </logging>
    </defaults>
  </engine-settings>
</esper-configuration>

14.4.12.3. JDBC Logging

By default, the engine does not measure JDBC query execution times or report the number of rows returned from a JDBC query through logging. To enable JDBC logging, set this option in the configuration. When enabled, the engine reports, at INFO level, any JDBC query performance and number of rows returned under the log name com.espertech.esper.jdbc.

The API to use to enable query plan logging is shown here:

Configuration config = new Configuration();
config.getEngineDefaults().getLogging().setEnableJDBC(true);

The XML snippet is:

<esper-configuration>
  <engine-settings>
    <defaults>
      <logging>
        <jdbc enabled="true"/>
    </logging>
    </defaults>
  </engine-settings>
</esper-configuration>

14.4.12.4. Audit Logging

The settings herein control the output format of @Audit logs.

This setting applies to all engine instances in the same JVM. Please also see the API documentation for information on pattern conversion characters.

Table 14.4. Audit Log Conversion Characters

CharacterDescription
mAudit message.
sStatement name.
uEngine URI.

The API to use to set am audit log format is shown here:

Configuration config = new Configuration();
config.getEngineDefaults().getLogging().setAuditPattern("[%u] [%s] %m");

The XML snippet is:

<esper-configuration>
  <engine-settings>
    <defaults>
      <logging>
        <audit pattern="[%u] [%s]%m"/>
    </logging>
    </defaults>
  </engine-settings>
</esper-configuration>

14.4.13. Engine Settings related to Variables

14.4.13.1. Variable Version Release Interval

This setting controls the length of time that the engine retains variable versions for use by statements that use variables and that execute, within the same statement for the same event, longer then the time interval. By default, the engine retains 15 seconds of variable versions.

For statements that use variables and that execute (in response to a single timer or other event) longer then the time period, the engine returns the current variable version at the time the statement executes, thereby softening the guarantee of consistency of variable values within the long-running statement. Please see Section 5.18.3, “Using Variables” for more information.

The XML configuration for this setting is shown below:

<engine-settings>
  <defaults>
    <variables>
      <msec-version-release value="15000"/>
    </variables>
  </defaults>
</engine-settings>

14.4.14. Engine Settings related to Patterns

14.4.14.1. Followed-By Operator Maximum Subexpression Count

You may use this setting to limit the total engine-wide number of pattern sub-expressions that all followed-by operators may manage. When the limit is reached, a condition is raised by the engine through the condition callback API.

By default, when the limit is reached, the engine also prevents the start of new pattern sub-expressions, until pattern sub-expressions end and the limit is no longer reached. By setting the prevent-start flag to false you can instruct the engine to only raise a condition and continue to allow the start of new pattern sub-expressions.

The implications of the settings described herein are also detailed in Section 6.5.8.2, “Limiting Engine-wide Sub-Expression Count”.

A sample XML configuration for this setting is shown below:

<engine-settings>
  <defaults>
    <patterns>
      <max-subexpression value="100" prevent-start="false"/>
    </patterns>
  </defaults>
</engine-settings>

The limit can be changed and disabled or enabled at runtime via the runtime configuration API. Pass a null value as the limit to disable limit checking.

A sample code snippet that sets a new limit is:

epService.getEPAdministrator().getConfiguration().setPatternMaxSubexpressions(100L);

14.4.15. Engine Settings related to Scripts

You may configure a default script dialect as described herein. The default script dialect is js which stands for JavaScript, since most JVM ship with an integrated JavaScript engine.

A sample XML configuration for this setting is shown below:

<engine-settings>
<defaults>
  <scripts default-dialect="js"/>
</defaults>
</engine-settings>

A sample code snippet that sets a new script dialect is:

config.getEngineDefaults().getScripts().setDefaultDialect("js");

14.4.16. Engine Settings related to Stream Selection

14.4.16.1. Default Statement Stream Selection

Statements can produce both insert stream (new data) and remove stream (old data) results. Remember that insert stream refers to arriving events and new aggregation values, while remove stream refers to events leaving data windows and prior aggregation values. By default, the engine delivers only the insert stream to listeners and observers of a statement.

There are keywords in the select clause that instruct the engine to not generate insert stream and/or remove stream results if your application does not need either one of the streams. These keywords are the istream, rstream and the irstream keywords.

By default, the engine only generates insert stream results equivalent to using the optional istream keyword in the select clause. If you application requires insert and remove stream results for many statements, your application can add the irstream keyword to the select clause of each statement, or you can set a new default stream selector via this setting.

The XML configuration for this setting is shown below:

<engine-settings>
  <defaults>
    <stream-selection>
      <stream-selector value="irstream" />
    </stream-selection>
  </defaults>
</engine-settings>

The equivalent code snippet using the configuration API is here:

Configuration config = new Configuration();
config.getEngineDefaults().getStreamSelection()
    .setDefaultStreamSelector(StreamSelector.RSTREAM_ISTREAM_BOTH);

14.4.17. Engine Settings related to Time Source

14.4.17.1. Default Time Source

This setting only applies if internal timer events control engine time (default). If external timer events provide engine clocking, the setting does not apply.

By default, the internal timer uses the call System.currentTimeMillis() to determine engine time in milliseconds. Via this setting the internal timer can be instructed to use System.nanoTime() instead. Please see Section 13.8, “Time Resolution” for more information.

Note: This is a Java VM global setting. If running multiple engine instances in a Java VM, the timer setting is global and applies to all engine instances in the same Java VM, for performance reasons.

A sample XML configuration for this setting is shown below, whereas the sample setting sets the time source to the nanosecond time provider:

<engine-settings>
  <defaults>
    <time-source>
      <time-source-type value="nano" />
    </time-source>
  </defaults>
</engine-settings>

The equivalent code snippet using the configuration API is here:

Configuration config = new Configuration();
config.getEngineDefaults().getTimeSource().
      setTimeSourceType(ConfigurationEngineDefaults.TimeSourceType.NANO);

14.4.18. Engine Settings related to Metrics Reporting

This section explains how to enable and configure metrics reporting, which is by default disabled. Please see Section 13.14, “Engine and Statement Metrics Reporting” for more information on the metrics data reported to your application.

The flag that enables metrics reporting is global to a Java virtual machine. If metrics reporting is enabled, the overhead incurred for reporting metrics is carried by all engine instances per Java VM.

Metrics reporting occurs by an engine-controlled separate daemon thread that each engine instance starts at engine initialization time, if metrics reporting and threading is enabled (threading enabled is the default).

Engine and statement metric intervals are in milliseconds. A negative or zero millisecond interval value may be provided to disable reporting.

To control statement metric reporting for individual statements or groups of statements, the engine provides a facility that groups statements by statement name. Each such statement group may have different reporting intervals configured, and intervals can be changed at runtime through runtime configuration. A statement group is assigned a group name at configuration time to identify the group.

Metrics reporting configuration is part of the engine default settings. All configuration options are also available via the Configuration API.

A sample XML configuration is shown below:

<engine-settings>
  <defaults>
    <metrics-reporting enabled="true" engine-interval="1000" statement-interval="1000" 
        threading="true"/>
  </defaults>
</engine-settings>

The engine-interval setting (defaults to 10 seconds) determines the frequency in milliseconds at which the engine reports engine metrics, in this example every 1 second. The statement-interval is for statement metrics. The threading flag is true by default since reporting takes place by a dedicated engine thread and can be set to false to use the external or internal timer thread instead.

The next example XML declares a statement group: The statements that have statement names that fall within the group follow a different reporting frequency:

<metrics-reporting enabled="true" statement-interval="0">
  <stmtgroup name="MyStmtGroup" interval="2000" default-include="true" num-stmts="100" 
        report-inactive="true">
    <exclude-regex>.*test.*</exclude-regex>
  </stmtgroup>
</metrics-reporting>

The above example configuration sets the statement-interval to zero to disable reporting for all statements. It defines a statement group by name MyStmtGroup and specifies a 2-second interval. The example sets the default-include flag to true (by default false) to include all statements in the statement group. The example also sets report-inactive to true (by default false) to report inactive statements.

The exclude-regex element may be used to specify a regular expression that serves to exclude statements from the group. Any statement whose statement name matches the exclude regular expression is not included in the group. In the above example, all statements with the characters 'test' inside their statement name are excluded from the group.

Any statement not belonging to any of the statement groups follow the configured statement interval.

There are additional elements available to include and exclude statements: include-regex, include-like and exclude-like. The latter two apply SQL-like matching. All patterns are case-sensitive.

Here is a further example of a possible statement group definition, which includes statements whose statement name have the characters @REPORT or @STREAM, and excludes statements whose statement name have the characters @IGNORE or @METRICS inside.

<metrics-reporting enabled="true">
  <stmtgroup name="MyStmtGroup" interval="1000">
    <include-like>%@REPORT%</include-like>
    <include-regex>.*@STREAM.*</include-like>
    <exclude-like>%@IGNORE%</exclude-like>
    <exclude-regex>.*@METRICS.*</exclude-regex>
  </stmtgroup>
</metrics-reporting>

14.4.19. Engine Settings related to Language and Locale

Locale-dependence in Esper can be present in the sort order of string values by the order by clause and by the sort view.

By default, Esper sorts string values using the compare method that is not locale dependent. To enable local dependent sorting you must set the configuration flag as described below.

The XML configuration sets the locale dependent sorting as shown below:

<engine-settings>
  <defaults>
    <language sort-using-collator="true"/>
  </defaults>
</engine-settings>

The API to change the setting:

Configuration config = new Configuration();
config.getEngineDefaults().getLanguage().setSortUsingCollator(true);

14.4.20. Engine Settings related to Expression Evaluation

14.4.20.1. Integer Division and Division by Zero

By default Esper returns double-typed values for divisions regardless of operand types. Division by zero returns positive or negative double infinity.

To have Esper use Java-standard integer division instead, use this setting as described here. In Java integer division, when dividing integer types, the result is an integer type. This means that if you divide an integer unevenly by another integer, it returns the whole number part of the result, does not perform any rounding and the fraction part is dropped. If Java-standard integer division is enabled, when dividing an integer numerator by an integer denominator, the result is an integer number. Thus the expression 1 / 4 results in an integer zero. Your EPL must then convert at least one of the numbers to a double value before the division, for example by specifying 1.0 / 4 or by using cast(myint, double).

When using Java integer division, division by zero for integer-typed operands always returns null. However division by zero for double-type operands still returns positive or negative double infinity. To also return null upon division by zero for double-type operands, set the flag to true as below (default is false).

The XML configuration is as follows:

<engine-settings>
  <defaults>
    <expression integer-division="false" division-by-zero-is-null="false"/>
  </defaults>
</engine-settings>

The API to change the setting:

Configuration config = new Configuration();
config.getEngineDefaults().getExpression().setIntegerDivision(true);
config.getEngineDefaults().getExpression().setDivisionByZeroReturnsNull(true);

14.4.20.2. Subselect Evaluation Order

By default Esper updates sub-selects with new events before evaluating the enclosing statement. This is relevant for statements that look for the same event in both the from clause and subselects.

To have Esper evaluate the enclosing clauses before updating the subselect in a subselect expression, set the flag as indicated herein.

The XML configuration as below sets the same as the default value:

<engine-settings>
  <defaults>
    <expression self-subselect-preeval="true"/>
  </defaults>
</engine-settings>

Here is a sample statement that utilitzes a sub-select against the same-events:

select * from MyEvent where prop not in (select prop from MyEvent.std:unique(otherProp))

By default the subselect data window updates first before the where clause is evaluated, thereby above statement never returns results.

Changing the setting described here causes the where clause to evaluate before the subselect data window updates, thereby the statement does post results.

14.4.20.3. User-Defined Function or Static Method Cache

By default Esper caches the result of an user-defined function if the parameter set to that function is empty or all parameters are constant values. Results of custom plug-in single-row functions are not cached according to the default configuration, unless the single-row function is explicitly configured with value cache enabled.

To have Esper evaluate the user-defined function regardless of constant parameters, set the flag to false as indicated herein.

The XML configuration as below sets the same as the default value:

<engine-settings>
  <defaults>
    <expression udf-cache="true"/>
  </defaults>
</engine-settings>

14.4.20.4. Extended Built-in Aggregation Functions

By default Esper provides a number of additional aggregation functions over the SQL standards. To have Esper only allow the standard SQL aggregation functions and not the additional ones, disable the setting as described here.

The XML configuration as below sets the same as the default value:

<engine-settings>
  <defaults>
    <expression extend-agg="true"/>
  </defaults>
</engine-settings>

14.4.20.5. Duck Typing

By default Esper validates method references when using the dot operator syntax at time of statement creation. With duck typing, the engine resolves method references at runtime.

The XML configuration as below sets the same as the default value:

<engine-settings>
  <defaults>
    <expression ducktyping="false"/>
  </defaults>
</engine-settings>

14.4.21. Engine Settings related to Execution of Statements

14.4.21.1. Prioritized Execution

By default Esper ignores @Priority and @Drop annotations and executes unprioritized, that is the engine does not attempt to interpret assigned priorities and reorder executions based on priority. Use this setting if your application requires prioritized execution.

By setting this configuration, the engine executes statements, when an event or schedule matches multiple statements, according to the assigned priority, starting from the highest priority value. See built-in EPL annotations in Section 5.2.6.6, “@Priority”.

By enabling this setting view sharing between statements as described in Section 14.4.11.1, “Sharing View Resources between Statements” is disabled.

The XML configuration to enable the flag, which is disabled by default, is as follows:

<engine-settings>
  <defaults>
    <execution prioritized="true"/>
  </defaults>
</engine-settings>

The API to change the setting:

Configuration config = new Configuration();
config.getEngineDefaults().getExecution().setPrioritized(true);

14.4.21.2. Context Partition Fair Locking

By default Esper configures context partition locks without fair locking. If your application is multi-threaded and performs very frequent reads via iterator or fire-and-forget queries, you may need to set this flag to prevent lock starvation in the face of concurrent reads and writes. Please consult the Java API documentation under ReentrantReadWriteLock and Fair Mode for more information.

The XML configuration to enable fair locking, which is disabled by default, is as follows:

<engine-settings>
  <defaults>
    <execution fairlock="true"/>
  </defaults>
</engine-settings>

The API to change the setting:

Configuration config = new Configuration();
config.getEngineDefaults().getExecution().setFairlock(true);

14.4.21.3. Disable Locking

By default Esper configures context partition locks as required after analyzing your EPL statements. You may disable context partition locks engine-wide using the setting described here. Use the @NoLock annotation instead to disable locking for a given statement or named window only.

CAUTION: We provide this setting for the purpose of identifying locking overhead, or when your application is single-threaded, or when using an external mechanism for concurrency control. Setting disable-locking to true may have unpredictable results unless your application is taking concurrency under consideration.

The XML configuration to disable context level locking is as follows:

<engine-settings>
  <defaults>
    <execution disable-locking="false"/>
  </defaults>
</engine-settings>

The API to change the setting:

Configuration config = new Configuration();
config.getEngineDefaults().getExecution().setDisableLocking(true);

14.4.21.4. Threading Profile

This setting is for performance tuning when using a large number of threads, such as 100 or more threads.

The configuration provides a setting that instructs the engine to reduce the use of thread-local variables. As the engine may use thread-local variables to reduce temporary object allocation, it can potentially use too much memory when a large number of threads are used with the engine. By setting the threading profile to large, the engine will allocate temporary objects instead of using thread-local variables for certain logic. Currently this setting applies exclusively to patterns and their filter expressions.

The XML configuration to set a large threading profile is as follows:

<engine-settings>
  <defaults>
    <execution threading-profile="large"/>
  </defaults>
</engine-settings>

The API to change the setting:

Configuration config = new Configuration();
config.getEngineDefaults().getExecution().setThreadingProfile(
    ConfigurationEngineDefaults.ThreadingProfile.LARGE);

14.4.22. Engine Settings related to Exception Handling

Use the settings as described here to register an exception handler factory class that provides an exception handler. The engine invokes exception handlers in the order they are listed to handle a continues-query unchecked exception, as further described in Section 13.10, “Exception Handling”.

Please provide the full-qualified class name of each class that implements the com.espertech.esper.client.hook.ExceptionHandlerFactory interface in the engine defaults configuration as below.

The XML configuration is as follows:

<engine-settings>
  <defaults>
    <exceptionHandling>
      <handlerFactory class="my.company.cep.MyCEPEngineExceptionHandlerFactory"/>
    </exceptionHandling>
  </defaults>
</engine-settings>

The API calls to register an exception handler factory are as follows:

Configuration config = new Configuration();
config.getEngineDefaults().getExceptionHandling().addClass(MyCEPEngineExceptionHandlerFactory.class);

14.4.23. Engine Settings related to Condition Handling

Use the settings as described here to register a condition handler factory class that provides a condition handler. The engine invokes condition handlers in the order they are listed to indicate conditions, which is the term used for notification when certain predefined limits are reached, as further described in Section 13.11, “Condition Handling”.

Please provide the full-qualified class name of each class that implements the com.espertech.esper.client.hook.ConditionHandlerFactory interface in the engine defaults configuration as below.

The XML configuration is as follows:

<engine-settings>
  <defaults>
    <conditionHandling>
      <handlerFactory class="my.company.cep.MyCEPEngineConditionHandlerFactory"/>
    </conditionHandling>
  </defaults>
</engine-settings>

The API calls to register a condition handler factory are as follows:

Configuration config = new Configuration();
config.getEngineDefaults().getConditionHandling().addClass(MyCEPEngineConditionHandlerFactory.class);

14.4.24. Revision Event Type

Revision event types reflect a versioning relationship between events of same or different event types. Please refer to Section 2.9, “Updating, Merging and Versioning Events” and Section 5.15.11, “Versioning and Revision Event Type Use with Named Windows”.

The configuration consists of the following:

  • An name of an event type whose events are base events.

  • Zero, one or more names of event types whose events are delta events.

  • One or more property names that supply the key values that tie base and delta events to existing revision events. Properties must exist on the event type as simple properties. Nested, indexed or mapped properties are not allowed.

  • Optionally, a strategy for overlaying or merging properties. The default strategy is Overlay Declared as described below.

The XML configuration for this setting is shown below:

<revision-event-type name="UserProfileRevisions">
  <base-event-type name="ProfileCreation"/>
  <delta-event-type name="ProfileUpdate"/>
  <key-property name="userid"/>
</revision-event-type>

If configuring via runtime or initialization-time API, this code snippet explains how:

Configuration config = new Configuration();
ConfigurationRevisionEventType configRev = new ConfigurationRevisionEventType();
configRev.setNameBaseEventType("ProfileCreation");
configRev.addNameDeltaEventType("ProfileUpdate");
configRev.setKeyPropertyNames(new String[] {"userid"});
config.addRevisionEventType("UserProfileRevisions", configRev);

As the configuration provides names of base and delta event types, such names must be configured for JavaBean, Map or XML events as the previous sections outline.

The next table outlines the available strategies:

Table 14.5. Property Revision Strategies

NameDescription
Overlay Declared (default)

A fast strategy for revising events that groups properties provided by base and delta events and overlays contributed properties to compute a revision.

For use when there is a limited number of combinations of properties that change on an event, and such combinations are known in advance.

The properties available on the output revision events are all properties of the base event type. Delta event types do not add any additional properties that are not present on the base event type.

Any null values or non-existing property on a delta (or base) event results in a null values for the same property on the output revision event.

Merge Declared

A strategy for revising events by merging properties provided by base and delta events, considering null values and non-existing (dynamic) properties as well.

For use when there is a limited number of combinations of properties that change on an event, and such combinations are known in advance.

The properties available on the output revision events are all properties of the base event type plus all additional properties that any of the delta event types provide.

Any null values or non-existing property on a delta (or base) event results in a null values for the same property on the output revision event.

Merge Non-null

A strategy for revising events by merging properties provided by base and delta events, considering only non-null values.

For use when there is an unlimited number of combinations of properties that change on an event, or combinations are not known in advance.

The properties available on the output revision events are all properties of the base event type plus all additional properties that any of the delta event types provide.

Null values returned by delta (or base) event properties provide no value to output revision events, i.e. null values are not merged.

Merge Exists

A strategy for revising events by merging properties provided by base and delta events, considering only values supplied by event properties that exist.

For use when there is an unlimited number of combinations of properties that change on an event, or combinations are not known in advance.

The properties available on the output revision events are all properties of the base event type plus all additional properties that any of the delta event types provide.

All properties are treated as dynamic properties: If an event property does not exist on a delta event (or base) event the property provides no value to output revision events, i.e. non-existing property values are not merged.

14.4.25. Variant Stream

A variant stream is a predefined stream into which events of multiple disparate event types can be inserted, and which can be selected from in patterns and the from clause.

The name of the variant stream and, optionally, the type of events that the stream may accept, are part of the stream definition. By default, the variant stream accepts only the predefined event types. The engine validates your insert into clause which inserts into the variant stream against the predefined types.

A variant stream can be set to accept any type of event, in which case all properties of the variant stream are effectively dynamic properties. Set the type variance flag to ANY to indicate the variant stream accepts any type of event.

The following XML configuration defines a variant stream by name OrderStream that carries only PartsOrder and ServiceOrder events:

<variant-stream name="OrderStream">
  <variant-event-type name="PartsOrder"/>
  <variant-event-type name="ServiceOrder"/>
</variant-stream>

This code snippet sets up a variant stream by name OutgoingEvent:

Configuration config = new Configuration();
ConfigurationVariantStream variant = new ConfigurationVariantStream();
variant.setTypeVariance(ConfigurationVariantStream.TypeVariance.ANY);
config.addVariantStream("OutgoingEvent", variant);

If specifying variant event type names, make sure such names have been configured for JavaBean, Map or XML events.

14.5. Type Names

Certain configuration values accept type names. Type names can occur in the configuration of variable types, Map-event property types as well as XPath cast types, for example. Types names are not case-sensitive.

The table below outlines all possible type names:

Table 14.6. Variable Type Names

Type NameType
string, varchar, varchar2 or java.lang.StringA string value
int, integer or java.lang.IntegerAn integer value
long or java.lang.LongA long value
bool, boolean or java.lang.BooleanA boolean value
double or java.lang.DoubleA double value
float or java.lang.FloatA float value
short or java.lang.ShortA short value
char, character or java.lang.CharacterA character value
byte or java.lang.ByteA byte value

14.6. Runtime Configuration

Certain configuration changes are available to perform on an engine instance while in operation. Such configuration operations are available via the getConfiguration method on EPAdministrator, which returns an ConfigurationOperations object. Please consult the JavaDoc documentation for more detail.

14.7. Logging Configuration

Esper logs all messages to Apache commons logging under an appropriate log level. To output log messages you can add Log4j to classpath and configure Log4j as below.

Esper's only direct dependency for logging is the Apache Commons Logging interfaces. You may use Log4j as described here, or you may use SLF4J instead (for example) as described in http://www.slf4j.org/legacy.html. You can also redirect SLF4J to any backend you want - nop, logback, jul as needed.

Statement-level processing information can be output, please see Section 15.3.1, “@Audit Annotation”.

For performance reasons, Esper does not log any debug-level or informational-level messages for event execution unless explicitly configured via Section 14.4.12.1, “Execution Path Debug Logging”.

A callback API for receiving certain critical engine reports is available as described in Section 13.10, “Exception Handling”.

More information on configuring engine-level settings for logging are at Section 14.4.12, “Engine Settings related to Logging”.

Apache Commons Logging uses an automatic discovery mechanism to find a log framework that it will delegate to. If your application has another component using logback, please inspect you logback configuration and add com.espertech.

The next table explains the log levels:

Table 14.7. Log Levels

Log LevelUse
DebugDisplays detailed engine-internal information that may not be easy to understand for application developers but are useful for engine support.
InfoUsed for a few critical engine-level log messages.
WarnCertain important warning or informational messages are displayed under the warning level.
ErrorExceptions reported within the engine or by plug-in components are reported under the error level. When users enter invalid EPL statements such validation errors are not reported as error logs and are indicated via API exception instead.

14.7.1. Log4j Logging Configuration

Log4j is the default logging component. Please find additional information for Log4j configuration and extension in http://logging.apache.org/log4j.

The easiest way to configure Log4j is by providing a Log4J configuration file, similar to the log4j.xml file shipped in the etc folder of the distribution.

Add the log4j.configuration system property to the java command line and provide the file name of the Log4j configuration file, making sure your classpath also includes the directory of the file:

java -Dlog4j.configuration=log4j.xml ...

© 2011 EsperTech Inc. All Rights Reserved