This is a follow-on from “RealTime Offer Management” (RTOM) post published on 19th November 2016 which can be found here.
Before diving into the details of what I ended up building, I want to document the overall “grand plan” and then focus on individual elements as these posts progress. The approach I took to the RTOM Application was based on the set of Architecture Principles and Functional Design Guidelines outlined below.
- Cloud based
- Suitable for a SaaS pricing model
- The database should be “Client Data Agnostic”
- Definition of Events that drive the application must be 100% configurable
- The application should be “Data Reactive”
- Mobile first for patron/client offer engagement
- Integration with source and target systems in real-time
- Offer evaluation, presentation and redemption should be “realtime”
Client Data Agnostic refers to a principle where the information stored by a user of RTOM has no impact on the structure of the RTOM database and the contents of the data while available to the rules and analysis engines for assessment are essentially “not of structural interest” to the RTOM application.
Data Reactive is a term I use to refer to a design principle where an update of an element of data causes the application to react and “do something”. Essentially the data structure has meta-data attached to each data element which describes and initiates the relevant processing should an item of this data-element type be created or updated
FUNCTIONAL DESIGN GUIDELINES
- Ease of Integration
- The “Rules Language” should be close to natural language expressions
- The rules language should allow references to events, data, and meta-data
- Offers should be presented in a “Closed Loop” to track uptake and prevent fraud
- Offers should be managed via a mobile platform
- Possible to attach realtime “budget management” and tollgates to both offers and patrons/clients (comping limits, etc)
- Realtime redemption management
- Offer Redemption Throttles limiting number of redemptions allowed by time periods
- Offer lifetime, availability and expiry controls
- Offers can be customised to patron/client profiles and preferences
- Offer values/discounts can be data/event driven and derived
- Offers can be “fixed rewards”, “choice of reward options” or “rewards based on result of a game on the patrons mobile device”
Rules Language refers to the scripting language the RTOM uses to allow its users to define conditions for filtered data accumulators (nodes & switches), qualifying offers, targeting reward sets to specific profiles, defining reward discount %’s & amounts, etc. It is essentially the glue that binds the processing together.
Closed Loop processing basically means that all actions taken against offers feed back to the RTOM for evaluation, verification and redemption / rejection.
Conceptually the RTOM Application looks as follows;
So far I have managed to keep to the architecture principles and function design guidelines without having to compromise to any great extent. The two areas that have caused headaches and much rework have been the “rules language” and the “database design”.
Coming up with a rules language is far more complex and challenging than I had originally anticipated. I originally wanted to get to a point where the language would be close to natural language expressions rather than computer focused “if x = y then z” syntax. This is easier to envisage than actually implement. In essence I ended up building a “Domain Specific Language” or DSL. I am currently on the third iteration and think I have it cracked (famous last words as they say). The key lessons I learned to get to this point are;
- Raw English (and I presume German, French, Hungarian, etc) is a crazy language to try and articulate processing requirements in. Regardless of how “non-programatically” you wish to make an approach we always resort to structured sentences and conditions when describing evaluating a situation.
- Work within the confines of the Domain in which the new language is going to be used. In this respect the RTOM domain has a fixed set of object types (Event, Data Element, Offer, Patron, etc) and a defined set of meta-data elements about each type.
- Define a set of verb based functions that allow greater flexibility. For example use keywords like “match, sum, count, average” to hide complexity and aid readability
- Do not over-design that solution
- Build a structure that is extensible and flexible (within reason) before trying to make it pretty
- Do not design first, and test later. I wrote hundreds of offer related conditions and expressions before and during the design process to visually and grammatically test out expression sequences. In essence I wrote the examples and then built the code to fit them, rather than trying to build a fully generic piece of code the then seeing if the test worked – Test Driven Development principles really worked well here
- Do not be afraid to start with a clean sheet rather than trying to rescue a lost-cause
I ended up writing a syntax validator, parser and processor. While they are at their third iteration at present I know there is still room for improvement and will probably evolve them to v4 and v5 within six months.
I will discuss the approach to the data concepts & database approach and how they evolved in the next post.