How to Create
a Fluent API DSL
with Lambda Builders ?
Nantes Java User Group

Welcome to the Furets!

@ozangunalp – Ozan Gunalp

PhD in Computer Science, JenkinsPipelineUnit author
Software Architect at LesFurets.com

@gdigugli – Gilles Di Guglielmo

Designer of sweet cooked software since 1999
Software Architect at LesFurets.com

1 website, 5 Insurance Products : Car, Health, Home, Bike, Loan
1 codebase, 500k lines of code, 60k unit tests, 150 selenium tests
25+ Developers, 2 DevOps, 4 Architects
30+ production servers including Load balancers, Frontend, Backend, Databases, BI
1 release per day
10+ years of code history
3M quotes/year, 40% of market share, 4M of customers

Introduction

LesFurets service orchestration

Insurers partners

We have 100 live insurers, on 5 products, each with business validation rules, that filter prospects based on their profile

Insurer exclusions

Hierarchy of 492 legacy classes, no governance or auditability

Use case: insurer exclusions

Insurer exclusions based on object model (legacy code)

                
public void check(FieldContext context, FormuleMoto formule, Conducteur conducteur,
                  Vehicule vehicule, Void unused, Besoins besoins,
                  Set<EAbTestingScenario> scenarios)
                  throws ExclusionException {
    if (besoins == null) {
        return;
    }
    if (besoins.getDateDebutContrat() == null) {
        return;
    }
    if (!DateHelper.isAfter(besoins.getDateDebutContrat(),
                    DateHelper.ajouteJoursADate(DateHelper.getToday(), NBR_JOURS),
                    DateHelper.EPrecision.jour)) {
        throw new ExclusionException(DATE_EFFET_PLUS_60_JOURS);
    }
}

Use case: goal

compliance: the rules correspond to the specification documents
auditability: understand a rule without looking at the code
governance: maintenance of the rules catalogue
clarity: productivity for developers

Same rule, more fluent:

                
public ExclusionRule exclusionRule() {
    return DOOV.when(dateContrat.after(todayPlusDays(60)))
               .exclusionRule();
}

Why a fluent API?

Java is verbose, but you can reduce the noise and write code like natural language with a fluent API

                
// SQL query with jOOQ
DSL.using(conn)
        .selectFrom(SALES)
        .where(MODULE.equal(module.getIntValue()))
        .and(NOTIF_DATETIME.greaterOrEqual(from))
        .and(DELETED.equal(SaleState.NO.getIntValue()))
        .stream()
        .map(SalesDAO::toSale)
        .collect(toList());

Meet dOOv!

Domain Object Oriented Validation

dOOv is a fluent API for typesafe domain model validation

dOOv : Under the hood

                
DOOV.when(accountCompany.eq(Company.LES_FURETS)
            .and(accountPhoneNumber.startsWith("+33"))).validate();

Live code

                
public static boolean validateAccount(User user,
                Account account, Configuration config) {
  if (config == null) {
      return false;
  }
  if (user == null || user.getBirthDate() == null) {
      return false;
  }
  if (account == null || account.getCountry() == null ||
      account.getPhoneNumber() == null) {
      return false;
  }
  if (YEARS.between(user.getBirthDate(), LocalDate.now()) >= 18
          && account.getEmail().length() <= config.getMaxEmailSize()
          && account.getCompany() == Company.LES_FURETS
          && account.getPhoneNumber().startsWith("+33")) {
      return true;
  }
  return false;
}

                
public static SampleModel sample() {
  User user = new User();
  user.setId(1);
  user.setFirstName("Foo");
  user.setLastName("BAR");
  // ...
            
  Account account = new Account();
  account.setCompany(Company.LES_FURETS);
  account.setId(9);
  // ...

                
public class Account extends Identity {

  @SamplePath(field = SampleFieldId.LOGIN, readable = "account.login")
  private String login;     
  // ...

  @SamplePath(field = SampleFieldId.COMPANY, readable = "account.company")
  private Company company;
  // ...

                
SampleModelResourceBundle_en_US.properties

account.login = account login
account.password = account password
account.country = account country
account.company = account company
...

SampleModelResourceBundle_fr_FR.properties

account.login = l'identifiant de connection
account.country = le pays
account.company = la soci\u00e9t\u00e9
...

                
> ./gradlew -p sample build

BUILD SUCCESSFUL in 23s
10 actionable tasks: 3 executed, 7 up-to-date

                
public class RulesCodeOneTest {
  SampleModelRule demoRule = DslSampleModel
          .when(DOOV.matchAll(
                  userBirthdate.ageAt(today()).greaterOrEquals(18),
                  accountEmail.length().lesserOrEquals(configurationMaxEmailSize),
                  accountCompany.eq(Company.LES_FURETS),
                  accountPhoneNumber.startsWith("+33")))
          .validate();
}

                
@Test
public void test_account() {
    SampleModel sample = SampleModels.sample();
              
    Result result = demoRule.executeOn(sample);
              
    Assertions.assertThat(result).isTrue();
    System.out.println(demoRule.readable());
    System.out.println(demoRule.markdown(Locale.FRANCE));
}

                
rule when match all [user birthdate age at today >= 18, //
  account email length is <= configuration max email size, //
  account company = LES_FURETS, account phone number starts with '+33'] validate
* règle
  * lorsque
    * correspond à tous
      * la date de naissance âge à la date du jour >= 18
      * l'émail a une longueur <= la taille maximum de l'émail
      * la société = LES_FURETS
      * le numéro de téléphone commence par '+33'
  * valider

                
@Test
public void test_account_failure_cause() {
    SampleModel sample = SampleModels.sample();
    sample.getAccount().setPhoneNumber("+1 12 34 56 78");
              
    Result result = demoRule.executeOn(sample);
              
    Assertions.assertThat(result).isTrue();
}

                
java.lang.AssertionError: Expected result to be true 
  (invalidated nodes: [account phone number starts with '+33'])

                
@Test
public void test_account_failure_cause() {
    SampleModel sample = SampleModels.sample();
    sample.getAccount().setPhoneNumber("+1 12 34 56 78");
              
    Result result = demoRule.executeOn(sample);
              
    Assertions.assertThat(result)
            .isFalse()
            .hasFailureCause("account phone number starts with '+33'");
}

                
@Test
public void test_account_failure_cause_2() {
    SampleModel sample = SampleModels.sample();
    sample.getAccount().setPhoneNumber("+1 12 34 56 78");
    sample.getAccount().setCompany(Company.BLABLACAR);
              
    Result result = demoRule.withShortCircuit(false).executeOn(sample);
              
    Assertions.assertThat(result)
            .isFalse()
            .hasFailureCause("match all [account company = LES_FURETS, " +
                    "account phone number starts with '+33']");
}

A few predicate reductions examples

Multiple 'and' operator

Validate that a profile
has an email with less than 20 characters
has at least 18 years when their country is France
their country is France when their phone number starts with '+33'

                  
  DOOV.when(accountEmail.length().lesserThan(20)
        .and(userBirthdate.ageAt(today()).greaterThan(18)
          .and(accountCountry.eq(Country.FR)))
        .and(accountCountry.eq(Country.FR)
          .and(accountPhoneNumber.startsWith("+33"))))
      .validate();

Multiple 'and' operators

                  
  model.getAccount().setEmail("test@test.org");
  model.getAccount().setCountry(Country.FR);
  model.getUser().setBirthDate(LocalDate.now().minusYears(19));
  
  ValidationRule rule = DOOV
      .when(accountEmail.length().lesserThan(20)  
        .and(userBirthdate.ageAt(today()).greaterThan(18)
            .and(accountCountry.eq(Country.FR)))
        .and(accountCountry.eq(Country.FR)
            .and(accountPhoneNumber.startsWith("+33"))))
      .validate();
  
  Result result = rule.withShortCircuit(false).executeOn(wrapper);
  System.out.println("> " + result.getFailureCause());

                  
  > account phone number starts with '+33'

'count' operator for predicates

Validate that a profile match at least two conditions :

profile has at least 18 years

country is France

phone number starts with '+33'

                  
  DOOV.when(count(
        userBirthdate.ageAt(today()).greaterThan(18),
        accountCountry.eq(Country.FR),
        accountPhoneNumber.startsWith("+33")).greaterThan(1))
      .validate();

'count' operator for predicates

                  
  model.getUser().setBirthDate(LocalDate.now().minusYears(19));
  model.getAccount().setCountry(Country.CAN);
  model.getAccount().setPhoneNumber("1 23 45 67 89");
  
  ValidationRule rule = DOOV
      .when(count(
              userBirthdate.ageAt(today()).greaterThan(18),
              accountCountry.eq(Country.FR),
              accountPhoneNumber.startsWith("+33"))
          .greaterThan(1))
      .validate();
  
  Result result = rule.withShortCircuit(false).executeOn(wrapper);
  System.out.println("> " + result.getFailureCause());

                  
  > account country = FR and account phone number starts with '+33'

Beyond bean validation

Annotation based APIs for validating POJOs

Bean Validation 2.0 : JSR 380
Reference implementation : Hibernate Validator

Beyond Bean Validation: types

Bean Validation rules are not strongly typed since it's annotation based. This code will fail at compile time, but your IDE won't be able to tell you why.

                
public class Account {

    @NotNull @Email
    private Email email;

}

Beyond Bean Validation: complex validation

Because Bean Validation constraints are based on field annotation, cross validation between fields are only available through the extension mechanism.

                
public class Account {

    @Pattern(regexp = "(FR)|(UK)")
    private String country;

    @Pattern(regexp = "???")
    private String phoneNumber;

}

Beyond Bean Validation: natural language

Bean Validation rules are not written with a natural language syntax and does not provide a syntax tree

                
@Size(min = 10, max = 200,
      message = "About Me must be between 10 and 200 characters")
private String aboutMe;

DSL

                
userAboutMe().length().between(10, 200).validate().readable()
> When user about me length is between 10 and 200, validate

Beyond Bean Validation: syntax versus consistency

Bean Validation allows syntax validation

Bean Validation rules are written on the model

Bean Validation uses reflection

dOOv allows consistency checks

dOOv rules are writtent outside of the model, allowing complex checks

dOOv uses code generation

Going further

LesFurets service Mapping

Domain Object Oriented Mapping

Next step is extending the DSL to create a
bean mapping framework

It features the same AST to text and statistics functionalities.

Use Case : Object-to-Object Mapping

source model

                    
class Model {
  User user;
  Account account;
}

class User {
  String firstName;
  String lastName;
  LocalDate birthdate;
}

class Account {
  String email;
  boolean acceptEmail;
  Country country;
}

→

target model

                    
class Employee {
  String fullName;
  String email;
  int age;
  String country;
  String company;
}

Use Case : Object-to-Object Mapping

                      Model model = ...;
Employee employee = new Employee();
// declarative mapping rule
MappingRule rules = mappings(
  when(accountAcceptEmail.isTrue())
      .then(map(accountEmail).to(employeeEmail)),

  map(userFirstName, userLastName)
      .using(biConverter((f, l) -> f + " " + l, "", "combine names"))
      .to(employeeFullname),

  map(userBirthdate.ageAt(today())).to(employeeAge),

  map(accountCountry)
      .using(converter(c -> c.name(), "country name"))
      .to(employeeCountry)
);
// then execute the mapping
rules.executeOn(model, employee)

Use Case : Object-to-Object Mapping

                      Model model = ...;
Employee employee = new Employee();
// declarative mapping rule
MappingRule rules = mappings(
  when(accountAcceptEmail.isTrue())
      .then(map(accountEmail).to(employeeEmail)),

  map(userFirstName, userLastName)
      .using(biConverter(
            (f, l) -> f + " " + l, "", "combine names"))
      .to(employeeFullname),

  map(userBirthdate.ageAt(today())).to(employeeAge),

  map(accountCountry)
      .using(converter(c -> c.name(), "country name"))
      .to(employeeCountry)
);
// then execute the mapping
rules.executeOn(model, employee)

Use Case : Object-to-Object Mapping

                      Model model = ...;
Employee employee = new Employee();
// declarative mapping rule
MappingRule rules = mappings(
  when(accountAcceptEmail.isTrue())
      .then(map(accountEmail).to(employeeEmail)),

  map(userFirstName, userLastName)
      .using(biConverter(
            (f, l) -> f + " " + l, "", "combine names"))
      .to(employeeFullname),

  map(userBirthdate.ageAt(today())).to(employeeAge),

  map(accountCountry)
      .using(converter(c -> c.name(), "country name"))
      .to(employeeCountry)
);
// then execute the mapping
rules.executeOn(model, employee)

Use Case : Object Manipulation

                
MappingRule rules = mappings(
  when(employeeAge.lesserThan(18))
      .then(mapNull(employeeFullname)),
  map(employeeEmail)
      .using(converter(s -> sha256.hashString(s), "sha256 string"))
      .to(employeeEmail)
)
// then execute the mapping
.executeOn(employee, employee)

We survived the GDPR!

Conclusion

Goal

We migrated our 492 business rules to dOOv,
we now have compliance, auditability, governance, clarity
(and more!)

dOOv 2.6.3 is out!

Gradle code generation support
Better failure causes
New operators for some types
Improved documentation (javadoc and wiki)
dOOm mapping support
Rule templates strongly typed parameters for predicate and mapping rules
HTML rendering including mapping rules and templates

Enjoy dOOv.org

http://www.dOOv.org

https://github.com/doov-org/doov (framework and examples)
https://repo1.maven.org/maven2/io/doov (maven central: 2.6.3)
https://github.com/doov-org/doov-docs (slides)
Apache Licence
Try and contribute!

Appendices

DSL

"A domain-specific language (DSL) is a computer language specialized to a particular application domain. This is in contrast to a general-purpose language (GPL), which is broadly applicable across domains"

dOOv ecosystem

How to write a validation rule?

key model

                    
// Root class of model
class Model {
  User user;
}

// Add key named EMAIL
enum ModelFieldId {
  EMAIL;
}

// Annotate email field
class User {
  @Path(field = EMAIL
        readable = ...)
  String email;
}

code generate

                    
// dOOv typed field class
class DslModel {
  StringFieldInfo userEmail;
}

write rules

                    
// Create rules by using
// generated fields
// in DslModel
DslModel
  .when(userEmail.eq(...))
  .validate()
  // Optionaly add rules
  // to a registry
  .registerOn(DEFAULT);

How to validate a model

get model

                    
// Get model from somewhere
// or instanciate it
User user = new User();
user.setEmail("e@mail.com");

Model model = new Model();
model.setUser(user);

execute

                    
// Use executeOn method
DslModel.when(email.matches(...))
        .validate()
        .executeOn(model);

// Or use the registry
DEFAULT.stream()
  .map(rule -> rule.executeOn(model));

Why a fluent API?

Java is verbose, but you can reduce the noise and write code like natural language with a fluent API

                
// JUnit API
assertEquals(9, fellowshipOfTheRing.size());
assertTrue(fellowshipOfTheRing.contains(frodo, sam));
assertFalse(fellowshipOfTheRing.contains(sauron));

// AssertJ API (fluent)
assertThat(fellowshipOfTheRing).hasSize(9)
                               .contains(frodo, sam)
                               .doesNotContain(sauron);

Fluent API

New elements in Java 8 makes it easier to write a fluent API

                
// java.util.function (io.doov.core.dsl.impl.LogicalBinaryCondition)
left.predicate().and(right.predicate()).test(model, context)

// java.util.stream (io.doov.core.dsl.impl.LogicalNaryCondition)
steps.stream().anyMatch(s -> s.predicate().test(model, context))

// lambda and method reference (io.doov.core.dsl.impl.NumericCondition)
predicate(greaterThanMetadata(field, value),
          (model, context) -> Optional.ofNullable(value),
          (l, r) -> greaterThanFunction().apply(l, r));

Fluent API

Many popular libraries propose fluent APIs like
jOOQ, AssertJ, Apache Spark, etc.

                
Dataset<Row> averagePrice = prices
        .filter(value.<String>getAs("insurer")
                     .equals("COOL insurer"))
        .groupBy("product")
        .agg(avg("price").as("average"))
        .orderBy(desc("average"));

Syntax tree

Why a syntax tree?

Makes readable text generation possible:
we can output a multi-language rules catalog
in multiple formats (text, markdown, HTML, etc.)

Execution statistics

We make daily statistics that helps us shape the business,
by removing or tweaking rules as needed

Combined 'and' & 'or' operators

Validate that a profile
has at least 18 years when their country is France
and their phone number starts with '+33'
has at least 21 years when their country is Canadian
and their phone number starts with '+1'

                      
    DOOV.when(userBirthdate.ageAt(today()).greaterThan(18)
            .and(accountCountry.eq(Country.FR)
            .and(accountPhoneNumber.startsWith("+33")))
            .or(userBirthdate.ageAt(today()).greaterThan(21)
                  .and(accountCountry.eq(Country.CAN)
                  .and(accountPhoneNumber.startsWith("+1")))))
        .validate();

Combined 'and' & 'or' operators

                    
    model.getUser().setBirthDate(LocalDate.now().minusYears(22));
    model.getAccount().setCountry(Country.FR);
    
    ValidationRule rule = DOOV
        .when(userBirthdate.ageAt(today()).greaterThan(18)
            .and(accountCountry.eq(Country.FR)
            .and(accountPhoneNumber.startsWith("+33")))
            .or(userBirthdate.ageAt(today()).greaterThan(21)
                  .and(accountCountry.eq(Country.CAN)
                  .and(accountPhoneNumber.startsWith("+1")))))
        .validate();
    
    Result result = rule.withShortCircuit(false).executeOn(wrapper);
    System.out.println("> " + result.getFailureCause());

                    
    > account phone number starts with '+33'
        or (account country = CAN and
            account phone number starts with '+1')

Combined 'matchAll' & 'matchAny' operators

                      
    DOOV.when(matchAny(
            matchAll(userBirthdate.ageAt(today()).greaterThan(18),
                     accountCountry.eq(Country.FR),
                     accountPhoneNumber.startsWith("+33")),
            matchAll(userBirthdate.ageAt(today()).greaterThan(21),
                     accountCountry.eq(Country.CAN),
                     accountPhoneNumber.startsWith("+1"))))
         .validate();

Combined 'matchAll' & 'matchAny' operators

                    
    model.getUser().setBirthDate(LocalDate.now().minusYears(22));
    model.getAccount().setCountry(Country.FR);
    
    ValidationRule rule = DOOV
        .when(matchAny(
            matchAll(userBirthdate.ageAt(today()).greaterThan(18),
                     accountCountry.eq(Country.FR),
                     accountPhoneNumber.startsWith("+33")),
            matchAll(userBirthdate.ageAt(today()).greaterThan(21),
                     accountCountry.eq(Country.CAN),
                     accountPhoneNumber.startsWith("+1"))))
        .validate();
    
    Result result = rule.withShortCircuit(false).executeOn(wrapper);
    System.out.println("> " + result.getFailureCause());

                    
    > match any [account phone number starts with '+33',
                 match all [account country = CAN,
                            account phone number starts with '+1']]

'anyMatch' operator for enumerated values

Validate that a profile country is Canadian or French

                    
    DOOV.when(accountCountry.anyMatch(Country.CAN, Country.FR)).validate();

'anyMatch' operator for enumerated values

                    
    model.getAccount().setCountry(Country.UK);
    
    ValidationRule rule = DOOV
        .when(accountCountry.anyMatch(Country.CAN, Country.FR))
        .validate();
    
    Result result = rule.withShortCircuit(false).executeOn(wrapper);
    System.out.println("> " + result.getFailureCause());

                    
    > account country != UK

Appendices: rules design

Going further : Implementing rules

              
enum Company {
    BLABLACAR, CANAL_PLUS, DAILYMOTION,
    LES_FURETS, MEETIC, OODRIVE,
}

Validate that the company of an account should NOT be
Dailymotion or Blablacar

              
DOOV.when(accountCompany.noneMatch(DAILYMOTION, BLABLACAR)).validate();

Implementing boolean logic

              
¬ DAILYMOTION ∧ ¬ BLABLACAR
// is equivalent to
¬ ( DAILYMOTION ∨ BLABLACAR )
// is equivalent to
LES_FURETS ∨ CANAL_PLUS ∨ MEETIC ∨ OODRIVE

De Morgan's laws
https://en.wikipedia.org/wiki/De_Morgan's_laws
https://en.wikipedia.org/wiki/Conjunctive_normal_form

Failure cause with noneMatch

                
model.getAccount().setCompany(DAILYMOTION);

ValidationRule rule = DOOV
      .when(accountCompany.noneMatch(DAILYMOTION, BLABLACAR))
      .validate();
Result result = rule.withShortCircuit(false).executeOn(model);
System.out.println("> " + result.getFailureCause());

                  
  > account company match none  : DAILYMOTION, BLABLACAR

Failure cause with notEq + and

                
DOOV.when(accountCompany.notEq(DAILYMOTION)
        .and(accountCompany.notEq(BLABLACAR))).validate();

                    
  > account company != DAILYMOTION

Conjuctive Normal Form (CNF)

Failure cause with not + or

                  
DOOV.when(accountCompany.eq(DAILYMOTION)
        .or(accountCompany.eq(BLABLACAR)).not()).validate();

                      
    > not (account company = DAILYMOTION or account company = BLABLACAR)

Failure cause with anyMatch

                    
DOOV.when(accountCompany.anyMatch(LES_FURETS, CANAL_PLUS, MEETIC, OODRIVE))
        .validate();

                
> account company != DAILYMOTION

Failure cause with eq + or

                      
DOOV.when(accountCompany.eq(LES_FURETS).or(accountCompany.eq(CANAL_PLUS)
        .or(accountCompany.eq(MEETIC).or(accountCompany.eq(OODRIVE)))))
                .validate();

                  
  > account company = LES_FURETS or (account company = CANAL_PLUS 
         or (account company = MEETIC or account company = OODRIVE))

Speed is not enough

                  
Benchmark                                     Mode  Cnt     Score      Error   Units
noneMatch                                     thrpt   20  8775.818 ± 148.951  ops/ms
notEq + and  [failure cause OK]               thrpt   20  5022.391 ± 147.550  ops/ms
not + or                                      thrpt   20  3022.433 ± 586.881  ops/ms
anyMatch     [failure cause OK]               thrpt   20  6002.415 ±  94.531  ops/ms
eq + or                                       thrpt   20  1855.837 ±  50.183  ops/ms

                    
Benchmark                                     Mode  Cnt     Score      Error   Units
noneMatch    with short circuit               thrpt   20  6839.429 ± 262.318  ops/ms
notEq + and  with short circuit               thrpt   20  7397.586 ± 252.090  ops/ms
not + or     with short circuit               thrpt   20  5084.227 ± 450.013  ops/ms
anyMatch     with short circuit               thrpt   20  6275.185 ±  56.600  ops/ms
eq + or      with short circuit               thrpt   20  1820.198 ±  60.300  ops/ms

Writing rules in CNF provides the best performance and failure causes

How to Create a Fluent API DSL with Lambda Builders ? Nantes Java User Group

Welcome to the Furets!

@ozangunalp – Ozan Gunalp

@gdigugli – Gilles Di Guglielmo

Introduction

LesFurets service orchestration

LesFurets service orchestration

Insurers partners

Insurer exclusions

Use case: insurer exclusions

Use case: goal

Why a fluent API?

Meet dOOv!

Live code

A few predicate reductions examples

Multiple 'and' operator

Multiple 'and' operators

'count' operator for predicates

'count' operator for predicates

Beyond bean validation

Beyond Bean Validation: types

Beyond Bean Validation: complex validation

Beyond Bean Validation: natural language

DSL

Beyond Bean Validation: syntax versus consistency

Going further

LesFurets service Mapping

Domain Object Oriented Mapping

Use Case : Object-to-Object Mapping

Use Case : Object-to-Object Mapping

Use Case : Object-to-Object Mapping

Use Case : Object-to-Object Mapping

Use Case : Object Manipulation

Conclusion

Goal

dOOv 2.6.3 is out!

Enjoy dOOv.org

Appendices

DSL

dOOv ecosystem

How to write a validation rule?

How to validate a model

Why a fluent API?

Fluent API

Fluent API

Syntax tree

Why a syntax tree?

Execution statistics

Combined 'and' & 'or' operators

Combined 'and' & 'or' operators

Combined 'matchAll' & 'matchAny' operators

Combined 'matchAll' & 'matchAny' operators

'anyMatch' operator for enumerated values

'anyMatch' operator for enumerated values

Appendices: rules design

Going further : Implementing rules

Implementing boolean logic

Speed is not enough

How to Create
a Fluent API DSL
with Lambda Builders ?
Nantes Java User Group