bigdata[at]lesfurets.com

https://github.com/lesfurets/

Welcome to the Furets !

@gdigugli – Gilles Di Guglielmo

  • Designer of sweet cooked software since 1999
  • Software Architect at LesFurets.com


@troisdiz - Denis Rampnoux

  • In the software business since 2001
  • Team Data Techlead at LesFurets.com

Sept 2007 - Broker Desktop Application

Jun 2009 - Broker Directory : Assuremieux.com

Jan 2010 - Insurance Aggregator

Apr 2012 - New brand : LesFurets.com

Sep 2012 - First TV advertising

Sep 2015 - Toys Launch

Sep 2016 - Market Leader

  • 1 website, 5 Insurance Products : Car, Health, Home, Bike, Loan
  • 1 codebase, 450k lines of code, 60k unit tests, 150 selenium tests
  • 20 Developers, 2 DevOps, 4 Architects
  • 20 production servers including Load balancers, Frontend, Backend, Databases, BI
  • 1 release per day
  • 9 years of code history
  • 2.5M quotes/year, 31% of market share

Growing fast !

Prevent disaster

Risk and Capabilities

Business Objectives

Moving the architecture

Lambda architecture - Nathan Marz

Existing architectures

Our target architecture

The path

Adding visualization and BI

Moving the infrastrucure

Hosting at OVH

System to replace

Previous MySQL setup

New Maria DB - Galera Cluster

Cassandra cluster setup

ETL mandatory evolutions

[Old] ETL

[Old] ETL : round #1

[Old] ETL : round #1

[Old] ETL : round #2

[Old] ETL : round #2

[Old] ETL : round #3

[Old] ETL : round #3

[New] ETL

[New] ETL : round #1

[New] ETL : round #1

[New] ETL : round #2

[New] ETL : round #2

[New] ETL : round #3

[New] ETL : round #3

Summary

  • Introducing timestamp is mandatory to achieve the C* full migration
  • Auto IDs could be corrupted in recovery & replication cases
  • Timestamp are more reliable to synchronize asynchronously two databases
  • Timestamp provide millisecond precision instead of minutes for REF_TIMES_ID
  • SQL queries using REF_TIMES_ID are really painful to read & write

Enjoy LesFurets.com

  • https://www.lesfurets.com
  • Compare and buy !
  • https://github.com/lesfurets
  • Try and contribute !

Thank You!