Introduction

General

Event Machine (EM) is an architectural abstraction and framework of an event driven multicore optimized processing concept originally developed for networking data plane. It offers an easy programming concept for scalable and dynamically load balanced multicore applications with a very low overhead run-to-completion principle.

Main elements in the concept are events, queues, scheduler, dispatcher and the execution objects (EO). Event is an application specific piece of data (like a message or a network packet) describing work, something to do. Any processing in EM must be triggered by an event. Events are sent to asynchronous application specific queues. A single thread on all cores of an EM instance runs a dispatcher loop (a "core" is used here to refer to a core or one HW thread on multi-threaded cores). Dispatcher interfaces with the scheduler and asks for an event. Scheduler evaluates the state of all queues and gives the highest priority event to the dispatcher, which forwards it to the EO mapped to the queue the event came from by calling the registered receive function. As the event is handled and receive function returns, the next event is received from the scheduler and again forwarded to the mapped EO. This happens in parallel on all cores included. Everything is highly efficient run to completion single thread, no context switching nor pre-emption (priorities are handled by the scheduler). EM can run on bare metal for best performance or under an operating system with special arrangements (e.g. one thread per core with thread affinity).

The concept and this API are made to be easy to implement for multiple general purpose or networking oriented multicore packet processing systems on chip, which typically also contain accelerators for packet processing needs. Efficient integration with modern HW accelerators has been a major driver in EM concept.

One general principle of this API is that all calls are multicore safe, i.e. no data structure gets broken, if calls are simultaneously made by multiple cores, but unless explicitly documented per API call, the application also needs to take the parallel world into consideration. For example if one core asks for a queue mode and another one changes the mode at the same time, the returned mode may be invalid (valid data, but the old or the new!). Thus modifications should be done under atomic context (if load balancing is used) or otherwise synchronized by the application. One simple way of achieving this is to use one EO with an atomic queue to do all the management functionality. That guarantees synchronized operations (but also serializes them limiting performance)

EM_64_BIT or EM_32_BIT (needs to be defined at makefile) defines whether (most of) the types used in the API are 32 or 64 bits wide. NOTE, that this is a major decision, since it may limit value passing between different systems using the defined types directly. Using 64-bits may allow more efficient underlying implementation, as more data can be coded in 64-bit identifiers for instance.

Some principles

Open Event Machine optional fork-join helper.

An event group can be used to trigger join of parallel operations. The number of parallel operations need to be known by the event group creator, but the separate events handlers don't need to know anything about the other related events.

1. a group is created with em_event_group_create()

2. the number of parallel events is set with em_event_group_apply()

3. the parallel events are sent normally, but using em_send_group() instead of em_send()

4. as the receive function of the last event is completed, the given notification event(s) are sent automatically and can trigger the next operation

5. the sequence continues from step 2. for new set of events (if the group is reused)

So here the original initiator only needs to know how the task is split into parallel events, the event handlers and the one continuing the work (join) are not involved (assuming the task itself can be separately processed)

Note, that this only works with events targeted to an EO, i.e. SW events

32 bit version

This is documentation represent the 32 bit version of Event Machine API. Define EM_64_BIT or EM_32_BIT to select between 64 and 32 bit versions.