---

📅 Today’s Picks

Faster Table Joins with Polars Multi-Threading

Problem

pandas processes joins on a single CPU core, leaving other cores idle during large table operations.

Solution

Polars distributes join operations across all available CPU cores, achieving significantly faster joins than pandas on large datasets.

What makes Polars fast:

• Processes rows in parallel batches
• Uses all available CPU cores
• Zero configuration required

---

🔄 Worth Revisiting

Faster Polars Queries with Programmatic Expressions

Problem

When you want to use for loops to apply similar transformations, each Polars with_columns() call processes sequentially.

This prevents the optimizer from seeing the full computation plan.

Solution

Instead, generate all Polars expressions programmatically before applying them together.

This enables Polars to:

• See the complete computation plan upfront
• Optimize across all expressions simultaneously
• Parallelize operations across CPU cores

---

☕️ Weekly Finds

Mole [Python Utils] – Deep clean and optimize your Mac with a simple command-line tool.

marker [LLM] – Convert PDF, DOCX, PPTX, and other documents to markdown with high speed and accuracy.

pathway [Data Engineer] – Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

Looking for a specific tool? Explore 70+ Python tools →

---