Data Mesh: Beyond the Data Lake
The data world has moved through Data Warehouses in to Data Lakes (which became Data Swamps), all without achieving the success of architectures like microservices. Data Mesh is a new paradigm that marries reuse patterns in microservices to persistent data problems.
While modern architectures like microservices have take the world by storm, the data world seems left behind. The world gradually migrated away from data warehouses, with too much structure, to data lakes, which had too little structure, and a host of other headaches. Applying some of the same reuse patterns underlying service mesh, Data Mesh is a new paradigm for handling data in highly distributed environments, addressing nagging issues like data quality, applicability, PII, and so on, while still enjoining the advantages of distributed teams and event-driven architectures. This talk highlights the innovations ThoughtWorks is pursuing for this modern perspective on data.
About Neal Ford
Neal is Director, Software Architect, and Meme Wrangler at ThoughtWorks, a global IT consultancy with an exclusive focus on end-to-end software development and delivery.
Before joining ThoughtWorks, Neal was the Chief Technology Officer at The DSW Group, Ltd., a nationally recognized training and development firm. Neal has a degree in Computer Science from Georgia State University specializing in languages and compilers and a minor in mathematics specializing in statistical analysis.
He is also the designer and developer of applications, instructional materials, magazine articles, video presentations, and author of 6 books, including the most recent The Productive Programmer. His language proficiencies include Java, C#/.NET, Ruby, Groovy, functional languages, Scheme, Object Pascal, C++, and C. His primary consulting focus is the design and construction of large-scale enterprise applications. Neal has taught on-site classes nationally and internationally to all phases of the military and to many Fortune 500 companies. He is also an internationally acclaimed speaker, having spoken at over 100 developer conferences worldwide, delivering more than 600 talks. If you have an insatiable curiosity about Neal, visit his web site at http://www.nealford.com. He welcomes feedback and can be reached at nford@thoughtworks.com.