Data Processors

Data processors transform an Eventstream and return a new one. They are available as methods on the Eventstream object. In the examples throughout this section, stream refers to an Eventstream instance.

Processors can be chained:

stream = (
    rete.datasets.load_ecom()
    .add_start_end_events()
    .filter_events(drop={"event": ["checkout_bug"]})
    .rename_events({"wishlist_add": "add_to_wishlist"})
)

Each processor returns a new Eventstream, so the original is never modified.

Event names are validated

Processors check every event name you pass against the events actually present in the eventstream, and raise a PreprocessingConfigError listing the available names when one doesn't match:

stream.filter_events(drop={"event": ["bot_ping"]})
# PreprocessingConfigError: [filter_events] Value(s) ['bot_ping'] not found in
# column 'event'. Available values: ['account_page', 'add_to_cart', 'cart', ...]

This is deliberate. A typo in an event name would otherwise turn into a filter that quietly keeps everything, or a rename that quietly does nothing — and you would find out several steps later, from a chart that looks plausible and is wrong. Use stream.get_event_counts() (or describe()) to get the exact spelling of every event before writing a pipeline.

Overview

Adding events

• add_events — insert synthetic events derived from existing events or a SQL query.
• add_start_end_events — prepend a path_start and append a path_end synthetic event to each path.

Renaming, editing & merging events

• rename_events — rename events using a mapping dict.
• edit_events — rename and/or delete events in a single operation.
• collapse_events — merge consecutive or grouped events into a single representative event.
• urls_to_events — turn a raw URL column into structured event names using a URL path tree.

Filtering, truncating & sampling

• filter_events — keep only rows that match a column filter, a Python predicate, or a SQL query.
• drop_events — remove events from the eventstream by name.
• filter_paths — keep only paths that satisfy a metric condition.
• truncate_paths — trim each path to the window between two anchor events.
• sample_paths — randomly sample paths (and all their events).

Sessions & lifecycle

• split_sessions — split each path into sub-sessions and add session ID and index columns.
• to_daily_states — convert the eventstream into daily lifecycle-state events (new, current, at-risk, dormant, ...).

Segments & clustering

• add_segment — add a new categorical segment column to the eventstream.
• drop_segment — remove a segment column from the eventstream.
• add_clusters — cluster paths using ML and add a new segment column with integer cluster labels.
• rename_segment_levels — rename levels within an existing segment column.