Please turn JavaScript on
header-image

New board topics in Databricks Community

Is this your feed? Claim it!

Publisher:  Unclaimed!
Message frequency:  49 / week

Message History

With the first day of December comes the first window of our Databricks Advent Calendar. It’s a perfect time to look back at this year’s biggest achievements and surprises — and to dream about the new “presents” the platform may bring us next year.

2

Feature serving can terrify some, but when combined with Lakebase, it lets you create a ...

Read full story

This has happened alot in the previous weeks although both azure and Databricks showed no issues at the time the error was recieved by both Databricks python SDK and Java SDK, now I started creating a retry mechnaism to retry those errors selectively.now, shouldn't such errors be handled directly through the SDKs code?

Read full story

I am using the platform DBUtils.fs.mv() on databricks clusters, and facing issues with move operation slowness.I move files in UC Volumes or ADLS storage abfss links, which work but is so slow.
I mean it takes hours to transfer files that used to take minutes when done on HDFS.
What is the best solution for this ? and what could've caused it?

Read full story

Hi everyone,

I’m researching the data‑engineering challenges behind modern streaming apps.For example, platforms like Kisskh — which manage thousands of daily active users and large volumes of video metadata — often struggle with performance and data‑scalability issues.

Here are some real technical problems that such platforms typically face:

• Sudden traffi...

Read full story

Hi everyone,I’m working on building and optimizing data pipelines in Databricks, especially for large-scale workloads, and I want to learn from others who have hands-on experience with performance tuning, architecture decisions, and best practices.

I’d appreciate insights on the following:

  • Best practices for designing scalable pipelines in Databricks
  • ...

Read full story