Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Filter by Categories
About Article
Analyze Data
Archive
Best Practices
Better Outputs
Blog
Code Optimization
Code Quality
Command Line
Daily tips
Dashboard
Data Analysis & Manipulation
Data Engineer
Data Visualization
DataFrame
Delta Lake
DevOps
DuckDB
Environment Management
Feature Engineer
Git
Jupyter Notebook
LLM
LLM
Machine Learning
Machine Learning
Machine Learning & AI
Manage Data
MLOps
Natural Language Processing
NumPy
Pandas
Polars
PySpark
Python Tips
Python Utilities
Python Utilities
Scrape Data
SQL
Testing
Time Series
Tools
Visualization
Visualization & Reporting
Workflow & Automation
Workflow Automation

Environment Management

ItsDangerous: Safely Pass Trusted Data to Untrusted Environments and Back

When passing data between different web requests, there is a risk of malicious code injection.

To ensure the safety of passing data to untrusted environments, use ItsDangerous. ItsDangerous adds a unique signature to the data to verify that the data has not been tampered with during transmission.

ItsDangerous: Safely Pass Trusted Data to Untrusted Environments and Back Read More »

Poetry: Python Tool for Dependency Management and Packaging

Have you ever updated a dependency of your project to a new version, and your code suddenly broke? That could be due to the incompatibility of the current dependencies and the new dependency.

Wouldn’t it be nice if you can check the compatibility between dependencies before installing new ones? That is when Poetry comes in handy.

In the code above, since pandas==1.4.1 requires numpy>=1.18.5, numpy<1.18 is not installed.

Another cool thing about Poetry is that when you remove a dependency, it also removes sub-dependencies that are no longer needed in your project.

Link to Poetry.

My article about Poetry.
Favorite

Poetry: Python Tool for Dependency Management and Packaging Read More »

Compare Dependencies of Two Requirements Files

It can be cumbersome to compare the dependencies between two requirements files. Especially when there are a lot of dependencies in each file. To automate the comparison, use compare-requirements.

The code above shows the comparison of 2 requirements files using compare-requirements.

Link to compare-requirements.

My previous tips on tools to manage your environment.
Favorite

Compare Dependencies of Two Requirements Files Read More »

Generate requirements.txt File for Jupyter Notebooks Based on Imports

pip freeze saves all packages in the environment, including ones that you don’t use in your current project. To generate a requirements.txt based on imports in your Jupyter Notebooks, use pipreqsnb. 

For example, to save all packages in your current project to a requirements.txt file, run:

<meta http-equiv="content-type" content="text/html; charset=utf-8">$ pipreqsnb . 

Link to pipreqsnb.

To generate requirements.txt for Python scripts, use pipreqs instead.

For a deeper dive into managing and tracking changes in notebooks, check out one of my articles on Jupyter Notebook. These 3 essential tools for version-controlling Jupyter notebooks.

Favorite

Generate requirements.txt File for Jupyter Notebooks Based on Imports Read More »

pydeps: Python Module Dependency Visualization

If you want to generate the graph showing the dependencies of your Python modules, try pydeps.

For example, to generate the dependency graph for files in the folder top_github_scraper, I type:

<meta http-equiv="content-type" content="text/html; charset=utf-8">$ pydeps top_github_scraper

The image above is the output of the command.

Link to pydeps.
Favorite

pydeps: Python Module Dependency Visualization Read More »

pip-autoremove: Remove a Package and Its Unused Dependencies

When using pip uninstall, you only remove a specific package. Wouldn’t it be nice if you can install that package and its unused dependencies?

That is when pip-autoremove comes in handy.

In the code above, using pip-autoremove removes pandas-profiling and its unused dependencies.

Link to pip-autoremove.

Link to previous tips on some useful tools I found.
Favorite

pip-autoremove: Remove a Package and Its Unused Dependencies Read More »

0
    0
    Your Cart
    Your cart is empty
    Scroll to Top

    Work with Khuyen Tran

    Work with Khuyen Tran