rtom #3 – Approach to Data

This is a follow-on from “RealTime Offer Management” (RTOM) post published on 19th November 2016 which can be found here.

The incentive for building the RTOM application was in looking at how to implement an Event Enabled Enterprise (E3) in a manner that was flexible, rule driven and easy to implement.  There are two what I will call “traditional” ways to deliver such a solution.  The first is “services” driven which requires all events to deliver “event triggers” when executed and requires an ESB/SOA driven approach, the second is a “data driven” approach where all data is gathered in one place (data warehouse, big data) and evaluated for required conditions.  Both of these approaches require significant degrees of architecture & development work plus the implementation of coded rules or a generic rule engine which then feeds back into a campaign or offer management system.

Within the world of Data we traditionally look at entity models, objects, attributes using a “real world” view but then end up in most (not all) cases storing the data in rigid tabular or document structures and try to mimic real world scenarios using complex code and queries to interrogate the data.  The NoSQL evolution has helped to soften this approach to a degree but the way we think about data and how to store it is still very “tabular” and “relational” in nature.  This in turn creates its own issues as the receipt of data needs to be processed stored, accumulated with scheduled processes running on a near continual basis seeking changes over a complete dataset to try and mimic realtime processing for triggered events.


In addition to this it is usually near impossible to provide a solution to users which allows them to implement rules referring to objects/data with real-world context.  You almost always end up referring to columns in tables and building additional data-marts, playing with the data in excel just to answer sometimes pretty simple questions and thus lose the capability to “offer in realtime”.

For RTOM the approach to data has been to look at a model where the data is a cloud of easily referenceable;

  • Events
  • Data Attributes
  • Meta Data
  • Higher Level “Data Nodes”

attached to a patron.


NOTE – For the remainder of this post I will use the term Patron to refer to Patrons, Guests, Customers of your business.

Events Are basically any transaction or data set generated by an operational application when the patron interacts with your property/business.  These events can be active or  passive.  Active events are transaction such as check-in, pay a bill, card-in, turn on slots, hand of cards, etc – essentially anything that is deliberately initiated by the patron.  Passive events are activities captured by virtue of the Patron’s interactional context with your business / property, these are mostly although not wholly related to location and environment aware data streams from a mobile device or interactions such as parking a vehicle, etc.

Data Attributes are the individual data items generated that describe the business context of an event.  (e.g. arrival-date, room-rate, F&B total, win/loss amount, etc)

Meta Data Is the descriptive data around the context of an Event, type of Data Node and various filter, computational & offer rules to be evaluated on creation or change of an Event/Data-Node.

Data Nodes are higher level data elements sourced from Event Creation, Event Data Attributes and/or other Data Nodes.  (e.g. a switch to indicate “on-property”, “gaming-in-progress”, accumulators to store count, sum, etc  persistant instances for Events/Data-Attributes at various levels of granularity.

The RTOM data design is based on an event driven model which uses “Data Change” agents to recognise events and send JSON formatted messages to the RTOM.  The data received by RTOM is considered “operational” and RTOM is not intended to be a Data Warehouse or mimic Big Data.  It is however an Analysis Engine that uses data changes to trigger rule processing for data nodes (switches, accumulators, etc) and for evaluating offer conditions based on events, data nodes, patron attributes, etc.

Data is within RTOM classified in two different layers;

The basic classification is foundational and applies to the Data Attributes of an Event;

  • Alphanumeric
  • Date
  • Time
  • Date-Time
  • Integer
  • Decimal
  • Money (Allows for automatic handling of exchange rates for monetary amounts)
  • Virtual (Allows for summary of raw data attributes)

The secondary layer of data classification is behavioural and applies to the Data-Nodes;

  • Static (new values always overwrite the old value)
  • Evolve (history of old values is maintained)
  • Switch (yes/no boolean based on rulesets)
  • Collection (list of values)
  • Sum (total of integer, decimal or monetary data-attributes)
  • Count (count of events)
  • Minimum (minimum data-attribute value received for data-node)
  • Maximum (maximum data-attribute value received for data-node)
  • Average (average data-attribute value received for data-node)
  • Points (special data-node type used for patron point management)

RTOM deals with four main classes of data around which all of its rules and processing have been defined.  These are;

  • Events
  • Data-Nodes
  • Patron Preferences and Attributes
  • Offers

Events are received in a JSON format which has the following layout;

{ meta: { // Standard Information about the event
  data: { // The payload area
  analysis: { // Lower level detail about the payload

RTOM enhances the meta data elements to attach additional criteria for evaluation and selection rules.  The Event “data” and “analysis” structures are stored in original JSON format with indexes composed of the meta data items.

By treating Events as JSON documents the RTOM application can store and refer to any data attributes required to be processed for offer evaluation.

A Data-Node is essentially a key-value paring with enhanced indexing and context rules.  A data node will hold a single value and be referenced by meta-data for  evaluation and selection rules.  Data nodes are stored in a traditional relational DB table structure.

Using an extended key-value construct allows for the RTOM application to be easily configured to create, store and process any data-node that can be derived from Events and their data-attributes.

Patron Preferences and Attributes are a very simple key-value pairing allowing referenceable information about the Patron to be addressed in the rules.  This information is also stored in a traditional relational DB table structure.

Offers consists of two parts, the Offer Definition data and the Offers Made data.  For flexibility and processing efficiencies the Offer Definitions are stored in a combination of relational DB table structures and JSON documents.  Offers Made data is “operational” in nature and more rigid in format so is stored in relational DB table structures.

Using the combination of the Events stored in JSON with Data-Nodes, Patron Pref’s and Offers Made stored in relational table structures and using a dot “.” notation it was possible to allow a rules language to be defined to access data with the following object.data references

  • e (current event in scope)
  • <event_name>
  • e.~meta_data_element / <event_name>.~meta_data_element
  • e.data_attribute / <event_name>.data_attribute
  • n (current data node in scope)
  • <data node> value
  • n.~meta_data_element / <data_node>.~meta_data_element
  • p.item for patron preferences and attributes
  • o (current offer in scope)
  • <offer_name>
  • o.~meta_data_element / <offer_name>.~meta_data_element

For for example:

A simple rule rewarding people who are gold or platinum members when checking in;

if check_in snd p.loyalty_tier = ["gold", "platinum"]

A simple rule looking for someone who is staying at a hotel every 5th time since the start of the year.  Assume a data-node accumulator has been configured to store stays;

if property_stays_year adjust-by-fulfill = 5

The adjust-by-fulfill "qualifier" adjusts the stored value by 
the value it was set to when the last offer was made

A simple rule looking to reward patrons who play between 9am and 5pm on Tuesday and Wednesdays and execute at least 100 plays.  Assume a data-node counter has been configured to store plays by day;

if card_in.~dow = [#tue, #wed] 
   and check_in.~time >< ["09:00", "17:00"] 
   and day_plays => 100

~dow is a "day of week" meta data attribute

A more complex rule to make an offer based on someone

  1. Visiting a minimum of five fine-dining restaurants
  2. At least twice per venue
  3. Spending a minimum of $200 at each sitting
  4. Within the last 10 months

the rule would be;

if count 
   { f&b_bill_pay // Event is payment of bill
       where e.~date => #str_month(-10) // Last 10 months
             e.~business = @fine_dining // Fine Dining business list
             e.total_bill => 200 // Bill is 200 or more
       group-by [e.~business] 
       group-drop count < 2 // Only keep groups of 2 or more entries
   => 5 // 5 or more restaurants

@fine_dining refers to a Quick-List element which can be
configured to hold any list of information items
#str_month(x)is a keyword producing a start month date X months ago

So in summary, by focusing purely on the domain model being addressed and also taking consideration of the usage of the data with a flexible user orientated rules syntax structure the data model for RTOM has been evolved to be;

  • A combination of JSON, enhanced key-value pair and relational DB models
  • Fully capable  of storing any data provided by the operational systems
  • Fully capable of storing any data derived from the events/data-attribute
  • Is Patron Centric
  • Is driven by meta-data defining actions to be executed based on creation/update of events and data-nodes

In the next article I will delve more deeply into the rules language, how it has evolved and some compromises I needed to make on the starting vision.

Complete list of articles in series;

About Brian Maguire

Working in IT for 30+ years. Recent position was Global Enterprise Architect for Las Vegas Sands Corporation. Currently immersed in the startup scene in Budapest.
This entry was posted in Architecture, Gaming, Hospitality, IT Strategy, Product Development, Realtime Offers, Software Development, Uncategorized. Bookmark the permalink.

2 Responses to rtom #3 – Approach to Data

  1. Larry O'Brien says:

    Hi Brian, this should be in a college text book on Software development – thanks for sharing with the world.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s