Want to learn more about Databricks? We can help. We've written some handy guides and listed ressources we've found helpful when we started our Databricks journey. We keep all these resources up to date so we hope you find these resources handy.
Data Engineering
This post explains how to create and write to Delta Lake tables using Rust on AWS S3 while addressing the challenge of eventual consistency. It demonstrates what can happen if multiple concurrent writers are not synchronised when writing data to a Delta table and how this challenge can be overcome. By the end of this post, you will also gain a better understanding of how the Delta Transaction Protocol works and how cloud storage services differ.
March 22, 2023
20 mins
Data Engineering
Most data engineers correlate delta lake format with Spark and Databricks. That's not true. Delta can be used by so many other tools and most cloud providers have added delta support to their analytics tools. In this post we will see how to use delta lake from a Rust client.
January 16, 2023
15 mins
Data Engineering
Do you use Databricks? Do you think it is just a hosted optimised flavour of Apache Spark? In that case you are missing many features and ideas that can take your productivity to the next level. This post has tips and tricks to take you to the next level on Databricks.
December 6, 2022
15 mins
General
There is tonnes out of Information out there about Databricks and it can be a bit overwhelming. Where does somebody even start? We have done you a favor and curated a list of learning materials we found useful when we started our Databricks journey and we share with new employees.
July 29, 2022
5 mins
Data Engineering
This post explains how to create and write to Delta Lake tables using Rust on AWS S3 while addressing the challenge of eventual consistency. It demonstrates what can happen if multiple concurrent writers are not synchronised when writing data to a Delta table and how this challenge can be overcome. By the end of this post, you will also gain a better understanding of how the Delta Transaction Protocol works and how cloud storage services differ.
March 22, 2023
20 mins
Data Engineering
Most data engineers correlate delta lake format with Spark and Databricks. That's not true. Delta can be used by so many other tools and most cloud providers have added delta support to their analytics tools. In this post we will see how to use delta lake from a Rust client.
January 16, 2023
15 mins
Data Engineering
Do you use Databricks? Do you think it is just a hosted optimised flavour of Apache Spark? In that case you are missing many features and ideas that can take your productivity to the next level. This post has tips and tricks to take you to the next level on Databricks.
December 6, 2022
15 mins
General
There is tonnes out of Information out there about Databricks and it can be a bit overwhelming. Where does somebody even start? We have done you a favor and curated a list of learning materials we found useful when we started our Databricks journey and we share with new employees.
July 29, 2022
5 mins
