Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Filter by Categories
About Article
Analyze Data
Archive
Best Practices
Better Outputs
Blog
Code Optimization
Code Quality
Command Line
Course
Daily tips
Dashboard
Data Analysis & Manipulation
Data Engineer
Data Visualization
DataFrame
Delta Lake
DevOps
DuckDB
Environment Management
Feature Engineer
Git
Jupyter Notebook
LLM
LLM Tools
Machine Learning
Machine Learning & AI
Machine Learning Tools
Manage Data
MLOps
Natural Language Processing
Newsletter Archive
NumPy
Pandas
Polars
PySpark
Python Helpers
Python Tips
Python Utilities
Scrape Data
SQL
Testing
Time Series
Tools
Visualization
Visualization & Reporting
Workflow & Automation
Workflow Automation

Polars 1.41 – Speed Up Wide Parquet Scans Without Code Changes

Polars 1.41 – Speed Up Wide Parquet Scans Without Code Changes

Grab your coffee. Here are this week’s highlights.


📅 Today’s Picks

LiteLLM – One interface for OpenAI, Anthropic, and Gemini

Code example: LiteLLM - One interface for OpenAI, Anthropic, and Gemini

Problem

Frameworks like LangChain and LlamaIndex are useful when you need chains, agents, retrieval, memory, or orchestration.

But if you only need to call different providers through the same interface, you may not want to restructure your app around a full LLM framework.

Solution

LiteLLM solves this as a lightweight interface layer.

You can keep one completion() call across 100+ providers, then switch models by changing only the model string.


Polars 1.41 – Speed Up Wide Parquet Scans Without Code Changes

Code example: Polars 1.41 - Speed Up Wide Parquet Scans Without Code Changes

Problem

scan_parquet is Polars’ lazy way to read a Parquet file, allowing it to optimize the query before loading data.

Before the query runs, Polars still needs to decode the Parquet footer to understand the schema, row groups, and column statistics. For very wide files, that setup step can take noticeable time.

Solution

Polars 1.41 makes scan_parquet faster by replacing the old metadata decoder with one built specifically for Parquet files.

In the Polars 1.41 release benchmark, footer decoding was up to 3.29x faster on a 10,000-column file.


Stay Current with CodeCut

Actionable Python tips, curated for busy data pros. Skim in under 2 minutes, three times a week.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top

Work with Khuyen Tran

Work with Khuyen Tran