Has anyone successfully run Docling on Databricks?

0
0
Asked By MellowOrbit42 On

My team has started using Docling for document parsing in Databricks. The results are promising, but we occasionally run into GIL-related runtime failures and instability. Has anyone found a reliable way to deploy or scale Docling with Databricks Agent Bricks, or are there better deployment patterns or alternatives we should consider?

4 Answers

Answered By QuietMaple88 On

You may also want to benchmark alternatives instead of assuming Docling is the best fit. In our testing, Kreuzberg, now called Xberg, was significantly faster and more reliable for our documents, with better task-level results.

Answered By SilverNook24 On

The GIL failures seem to be a tradeoff for Docling’s parsing capabilities. If you stay with it, isolate parsing into separate processes or services and keep each batch relatively small rather than running everything in one long-lived Databricks process.

Answered By CopperLynx7 On

We ran into similar problems. Processing smaller batches and spreading the workload across more workers helped keep the GIL from locking up the runtime. It still needs some monitoring, but the approach is workable for moderate workloads.

Answered By VividHarbor31 On

We ended up running Docling through a serving layer rather than directly inside the notebook or job process. That made the setup more scalable, reliable, and cost-effective, especially when several parsing requests were running at once.

Related Questions

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.