Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Filter by Categories
About Article
Analyze Data
Archive
Best Practices
Better Outputs
Blog
Code Optimization
Code Quality
Command Line
Course
Daily tips
Dashboard
Data Analysis & Manipulation
Data Engineer
Data Visualization
DataFrame
Delta Lake
DevOps
DuckDB
Environment Management
Feature Engineer
Git
Jupyter Notebook
LLM
LLM Tools
Machine Learning
Machine Learning & AI
Machine Learning Tools
Manage Data
MLOps
Natural Language Processing
Newsletter Archive
NumPy
Pandas
Polars
PySpark
Python Helpers
Python Tips
Python Utilities
Scrape Data
SQL
Testing
Time Series
Tools
Visualization
Visualization & Reporting
Workflow & Automation
Workflow Automation

Newsletter #239: Delta Lake: Insert + Update in One Operation

Newsletter #239: Delta Lake: Insert + Update in One Operation


๐Ÿ“… Today’s Picks

Delta Lake: Insert + Update in One Operation

Code example: Delta Lake: Insert + Update in One Operation

Problem

In pandas, implementing upserts means running 3 separate operations: filter existing records, update matches, and append new ones.

Each step requires a full data scan, increasing both code complexity and execution time.

Solution

Delta Lake’s MERGE replaces this 3-step process with a single transaction that updates existing records and inserts new ones.

How it works:

  • Compares source data with existing table records
  • Updates matching records with new values
  • Inserts records that don’t exist yet
  • Executes all changes together with automatic rollback if any step fails

โ˜•๏ธ Weekly Finds

Boruta-Shap [ML] – A tree-based feature selection tool combining the Boruta algorithm with SHAP values to identify the most important features for machine learning models.

a2a-python [LLM] – Official Python SDK for building agentic applications as A2A Servers following the Agent2Agent Protocol, with async support and optional integrations.

respx [Python Utils] – A Python library for mocking HTTPX and HTTP Core with request pattern matching and customizable response side effects for testing purposes.

Looking for a specific tool? Explore 70+ Python tools โ†’

Stay Current with CodeCut

Actionable Python tips, curated for busy data pros. Skim in under 2 minutes, three times a week.

Leave a Comment

Your email address will not be published. Required fields are marked *

0
    0
    Your Cart
    Your cart is empty
    Scroll to Top

    Work with Khuyen Tran

    Work with Khuyen Tran