Goto Cassandra in 60 minutes
   [at] LesFurets

Welcome to the Furets!

@gdigugli – Gilles Di Guglielmo

  • Designer of sweet cooked software since 1999
  • Software Architect at LesFurets.com

Insurance comparison (and more !)

  • 1 website :
  • 5 Insurance Products : Car, Health, Home, Bike, Loan
  • Banks, energy, loans, Internet Box
  • 1 codebase, 450k lines of code, 60k unit tests, 150 selenium tests
  • 23 Developers, 2 DevOps, 4 Architects
  • 20+ production servers including Load balancers, Frontend, Backend, Databases, BI
  • 1 release per day
  • 8 years of code history
  • 3M quotes/year, 40% of market share, 4M of customers

Problems to solve !

Limited time availability of online data

  • In SQL database as XML blobs
  • Question Sets only available for 7 days (then on NAS as XML files)

Past architecture

Growing fast !

Prevent disaster on olap database

Tool selection criteria

  • Scale in volume and speed at low cost
  • No master slave architecture
  • Hard schema constraints

Inspiration : Lambda architecture - Nathan Marz

Our target architecture

Data models governance

Keying a model

Cassandra in a Nutshell

  • Created at Facebook ...
  • ... and now used at Netflix, Apple and Instagram !
  • A mix of the data model of Google BigTable
  • ... and the distributing model of Amazon Dynamo DB
  • hard schema constraints...
  • ... and tunable consistency

Cassandra distribution

Line Structure

Cassandra distribution

Consistent Hashing of the Partition Key

Cassandra distribution

Query Path

Cassandra distribution

Tunable consistency (1/2)

Cassandra distribution

Tunable consistency (2/2)

Moving the architecture

Current state

On Demand Machine Learning

Moving the infrastructure

Cassandra cluster current setup

Cassandra cluster next setup

Back to the code

                
public static SampleModel sample() {
  User user = new User();
  user.setId(1);
  user.setFirstName("Foo");
  user.setLastName("BAR");
  // ...
            
  Account account = new Account();
  account.setCompany(Company.LES_FURETS);
  account.setId(9);
  // ...
                
              
                
SampleModel model = new SampleModel();
model.setAccount(new Account());
model.getAccount().setEmail("data-team@lesfurets.com");
System.out.println(model.getAccount().getEmail());
          
FieldModel fieldModel = new SampleModelWrapper(model);
System.out.println(fieldModel. get(EMAIL));
          
fieldModel.set(EMAIL, "gdu@lesfurets.com");
System.out.println(fieldModel. get(EMAIL));
                
              
                
FieldModel model = SampleModels.wrapper();
Map map = model.stream()
        .collect(toMap(Entry::getKey, Entry::getValue));
System.out.println(map);
          
SampleModelWrapper newModel = map.entrySet().stream()
        .collect(SampleModelWrapper.toFieldModel());
newModel.stream().forEach(System.out::println);
System.out.println(newModel.getModel().getAccount().getEmail());
                
              
                
FieldModel model = SampleModels.wrapper();

Map map = model.stream().collect(toMap(Entry::getKey, Entry::getValue));
SampleModelWrapper newModel = map.entrySet().stream()
        .filter(e -> e.getKey().hasTag(SampleTag.ACCOUNT))
        .collect(SampleModelWrapper.toFieldModel());
          
newModel.stream().forEach(System.out::println);
                
              
                
FieldModel model = SampleModels.wrapper();

CreateTable createTable = SchemaBuilder.createTable("sample", "model")
      .withPartitionKey("snapshot_id", DataTypes.TIMEUUID)
      .withClusteringColumn(LOGIN.name(), DataTypes.TEXT);
          
for (FieldInfo info : model.getFieldInfos()) {
      createTable = createTable.withColumn(info.id().code(), cqlType(info));
}
          
CreateTableWithOptions createTableWithOptions = createTable
      .withClusteringOrder(LOGIN.name(), ClusteringOrder.DESC);
                
              
                
CqlSession session = CqlSession.builder().build();

MutableCodecRegistry registry = session.getContext().getCodecRegistry();
registry.register(new EnumNameCodec<>(Country.class));
registry.register(new EnumNameCodec<>(EmailType.class));
...

try (CqlSession session = session()) {
  session.execute(createTableWithOptions);
}
                
              
              
cqlsh:sample> DESCRIBE model

CREATE TABLE sample.model (
    snapshot_id timeuuid,
    login text,
    birthdate date,
    ...
    country text,
    creation_date date,
    email text,
    ...
    PRIMARY KEY (snapshot_id, login)
) WITH CLUSTERING ORDER BY (login DESC)
              
            
              
FieldModel model = SampleModels.wrapper();

Map values = model.stream().collect(Collectors.toMap(
    e -> e.getKey().code(),
    e -> QueryBuilder.literal(e.getValue(), codecRegistry)));
                
Insert insertRequest = QueryBuilder.insertInto("sample", "model")
    .value("snapshot_id", QueryBuilder.literal(Uuids.timeBased()))
    .values(values);
              
            
              
cqlsh:sample> SELECT * FROM model;

snapshot_id                          | login  | account_id | address       | birthdate  |
-------------------------------------+--------+------------+---------------+------------|
d102c880-7736-11e8-ac36-478151704385 | foobar |          9 | 41 rue Thiers | 1980-08-01 |
                 
(1 rows)
              
            
                
public class Account extends Identity {

  @SamplePath(field = SampleFieldId.LOGIN, readable = "account.login")
  private String login;     
  // ...

  @SamplePath(field = SampleFieldId.COMPANY)
  private Company company;
  // ...
                
              
                
> ./gradlew -p sample build

BUILD SUCCESSFUL in 23s
10 actionable tasks: 3 executed, 7 up-to-date
                
              
                
FieldModel model = SampleModels.wrapper();

try (CqlSession session = CqlSession.builder().build()) {
    model.getFieldInfos().stream()
        .filter(f -> session.getMetadata()
              .getKeyspace("sample").get()
              .getTable("model").get()
              .getColumn(f.id().code()).isPresent())
        .forEach(f -> session.execute(SchemaBuilder.alterTable("sample", "model")
              .addColumn(f.id().code(), cqlType(f)).build()));
}
                
              

Enjoy dOOv.org

http://www.dOOv.org

Thank You!