DomainModel.stream()

Welcome to the Furets !

Sept 2007 - Broker Desktop Application

Sept 2008 - Broker Website Widget

Jun 2009 - Broker Directory : Assuremieux.com

Jan 2010 - Insurance Aggregator

Apr 2012 - New brand : LesFurets.com

Sep 2012 - First TV advertising

Dec 2015 - Market leader

• 1 website, 5 Insurance Products : Car, Health, Home, Bike, Loan
• 1 codebase, 450k lines of code, 60k unit tests, 150 selenium tests
• 22 Developers, 2 DevOps, 3 Architects
• 19 production servers including Load balancers, Frontend, Backend, Databases, BI
• 1 release per day
• 7 years of code history
  
• 2.5M quotes/year, 31% of market share

Domain Model, what do you mean ?

Moving the architecture

Architecture Buzz

• Copy on write
• Immutable data
• NoSQL
• Clustered Data Base
• Data streaming
• Micro batching
• Lambda architecture
• Data on demand

Lambda architecture - Nathan Marz

What you have ...

What you want …

You are in trouble ...

Front app evolutions

• Your domain model is cloned using two different languages.
  
• Governance of the domain model and change requests are difficult to implement.
  
• Development owners of the server and front application are different.
  
• UI Spaghetti code.

Front storage evolutions

• Mapping entities in Cassandra is painful.
• Don’t migrate your Entity/Relation model using cut & paste!
• You should probably flat your domain model to fit a single row.
• Infinite storage.

BackOffice evolutions

• A lot of rework to implement.
• Spark and ML-lib love data vectors.
• Spark read raw data, aggregate, write them in microbatches.
• No more (very long) nightly batches.
• Full history is available for custom computations.

What we do now ?

Add keys everywhere !

A bit of decoration

Let's start with this model...

... and transform it to a key-value model

User model

User model

User model

Account model

Account model

Account model

Account model

Account model

From domain model to key-value model

• Step 1 : create one Enumeration with all keys
• Step 2 : annotate your domain model
• Step 3 : generate the key value model

Let's see the domain model & the key-value model

Step 1 : one Enumeration with all keys

• one literal for each key
• position (optional) : used when collections are denormalized
• tag (optional) : used to add meta-information

Step 2 : one Annotation to annotate the domain

• one method to retrieve the FieldId
• one method to retrieve the Constraint

Step 3 : annotate your domain model

• attribute, getter or setter can be annotated
• getter and setter must follow Java naming conventions

Step 4 : generate the key value model

... Just one little issue.

Account & User ID

sampleModel.getUser().getId()

Account & User ID

sampleModel.getAccount().getId()

Many key for a single attribute

• Each key must have a unique path to retrieve its value
• How to retrieve the right value for each field?
• getAccount().getId()? getUser().getId()?

Solution : constraints

One for each field, specify which getter to use

Add a constraint for each annotation

• the code generator only use the path with the specified getter
• each field is now mapped correctly

Let the magic begin ...

Code generator under the hood

• Execution is done using a maven plugin
• Annotated properties are processed using Java Reflection
• Generate handwritten style code, easy to read
• Enable JIT optimisations by combining lambdas and enumerations
• Lightweight pattern, designed for performance

Generation output

Generation output

An Enumeration containing metadate : type, generics type

Generation output

a CSV containing each field, path, and value type

Usages are unlimited

You can use Java 8 stream on your Domain Model !

• FieldModel#stream() : a stream of key/value pair (FieldId/Object)
• FieldModel#parallelStream() : parallel stream and better performance
• Easy use a filter/map/reduce operations on a domain model

Could you write a domain model diff tool now ?

Where we use it at LesFurets.com

• Simple UI binding by replacing JSON model by a Dictionary (80% less code)
• Use JSON Dictionary for client/server RPC
• Batch and ETL with an email provider (90% less code)
• Persistence in cassandra (from scratch with 200 lines of code for a whole model)
• Injecting user data in selenium (40% less code)
• Mocking our domain model in thousand of unit tests

... everything in iterative fashion during the past 2 years.

Future and ideas

• Push as OSS our Cassandra serialisation code
• Push some Spark samples
• JEP 276: Dynamic Linking of Language-Defined Object Models

Model-Map available on Github

• https://doov.org
• Framework and examples
• Apache Licence
• Try it and contribute !