Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Filter by Categories
About Article
Analyze Data
Archive
Best Practices
Better Outputs
Blog
Code Optimization
Code Quality
Command Line
Daily tips
Dashboard
Data Analysis & Manipulation
Data Engineer
Data Visualization
DataFrame
Delta Lake
DevOps
DuckDB
Environment Management
Feature Engineer
Git
Jupyter Notebook
LLM
LLM
Machine Learning
Machine Learning
Machine Learning & AI
Manage Data
MLOps
Natural Language Processing
NumPy
Pandas
Polars
PySpark
Python Tips
Python Utilities
Python Utilities
Scrape Data
SQL
Testing
Time Series
Tools
Visualization
Visualization & Reporting
Workflow & Automation
Workflow Automation

Workflow Automation

Simplifying Browser Automation with Helium

Before: Verbose Selenium Code

Traditional Selenium browser automation scripts can be verbose and difficult to maintain, requiring explicit element locators and waits.

Here’s an example of a Selenium script that logs into GitHub, navigates to a repository, and stars and unstars it:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# Start Chrome

with webdriver.Chrome() as driver:
# Navigate to GitHub login page
driver.get('https://github.com/login')

# Login
username_field = driver.find_element(By.ID, 'login_field')
password_field = driver.find_element(By.ID, 'password')

username_field.send_keys('1mh')
password_field.send_keys('1Secretpw')

login_button = driver.find_element(By.NAME, 'commit')
login_button.click()

# Navigate to repository
driver.get('https://github.com/mherrmann/helium')

# Wait for and click Star button
wait = WebDriverWait(driver, 10)
star_button = wait.until(EC.element_to_be_clickable((By.XPATH, "//button[contains(text(), 'Star')]")))
star_button.click()

# Wait for and click Unstar button
unstar_button = wait.until(EC.element_to_be_clickable((By.XPATH, "//button[contains(text(), 'Unstar')]")))
unstar_button.click()

After: Simplified Helium Code

Helium provides several benefits over traditional Selenium scripts:

Helium’s APIs simplify code, making it concise and readable by eliminating explicit element locators and waits.

Helium scripts are more maintainable, as you can focus on logic without worrying about implementation details.

Helium’s intuitive API enables faster development, allowing you to write browser automation scripts quickly and save time.

With Helium, the same script becomes much simpler and more intuitive:

from helium import *

# Start Chrome and navigate to GitHub login page
start_chrome('github.com/login')

# Enter username and password
write('1mh', into='Username')
write('1Secretpw', into='Password')

# Click the Sign in button
click('Sign in')

# Navigate to the Helium repository
go_to('github.com/mherrmann/helium')

# Star and then unstar the repository
click(Button('Star'))
click(Button('Unstar'))

# Close the browser
kill_browser()

 Link to Helium.
Favorite

Simplifying Browser Automation with Helium Read More »

Adding Sound Notifications to Your Python Code with Chime

Have you ever wanted your computer to make a sound when your Python code reaches a certain state? Maybe you want to be notified when a long-running task is complete, or when an error occurs. With the chime library, you can easily add sound notifications to your Python code.

Basic Usage

Here’s an example of how you can use chime to play different sounds:

import chime

chime.success()
chime.warning()
chime.error()
chime.info()

Try running this code and listen to the different sounds that play.

Using Chime for Error Notifications

One useful application of chime is to make a sound when an error occurs in your code. Here’s an example:

a = 0
try:
b = 2/a
except ZeroDivisionError:
print("You can't divide a number by 0!")
chime.error()

In this example, when the code tries to divide by zero, it catches the ZeroDivisionError exception and plays an error sound using chime.error().

Link to chime.
Favorite

Adding Sound Notifications to Your Python Code with Chime Read More »

Building a Conversational Interface for Elasticsearch Data with Kestra and OpenAI

To create natural language interactions with Elasticsearch data, use Kestra with the OpenAI plugin. This combination transforms structured data into an intuitive, conversational interface.

Key advantages:

Custom data processing and AI prompts for your specific use case

Easy configuration and deployment with Kestra’s YAML-based workflow

Here’s a code example demonstrating this workflow:

id: movie_recommendation_system
namespace: entertainment.movies

inputs:
– id: user_preference
type: STRING
defaults: I like action movies with a bit of comedy

tasks:
– id: search_movies
type: io.kestra.plugin.elasticsearch.Search
connection:
hosts:
– http://localhost:9200/
indexes:
– movies_database
request:
size: 5
query:
bool:
must:
multi_match:
query: "{{ inputs.user_preference }}"
fields: ["title", "description", "genre"]
type: best_fields

– id: format_movie_results
type: io.kestra.plugin.core.debug.Return
format: >
{% for movie in outputs.search_movies.rows %}
Title: {{ movie.title }}
Genre: {{ movie.genre }}
Description: {{ movie.description }}
{% endfor %}

– id: generate_recommendations
type: io.kestra.plugin.openai.ChatCompletion
apiKey: sk-proj-your-OpenAI-API-KEY
model: gpt-4
maxTokens: 500
prompt: |
You're a movie recommendation assistant.
Based on the USER PREFERENCE and the MOVIE RESULTS, suggest 3 movies from the list.
Explain why each movie matches the user's preference.
USER PREFERENCE: {{ inputs.user_preference }}
MOVIE RESULTS: {{ outputs.format_movie_results.value }}

– id: display_recommendations
type: io.kestra.plugin.core.log.Log
message: "{{ outputs.generate_recommendations.choices | jq('.[].message.content') | first }}"

This example shows:

search_movies task uses Elasticsearch Search to query the index based on user preference.

format_movie_results task formats the search results.

generate_recommendations task uses OpenAI ChatCompletion to analyze results and user preference, creating personalized movie recommendations with explanations.

Favorite

Building a Conversational Interface for Elasticsearch Data with Kestra and OpenAI Read More »

Exploring Google Trends with Pytrends API

Google Trends provides valuable insights into search patterns and public interest over time. Pytrends, an unofficial API for Google Trends, offers a simple way to access and analyze this data programmatically.

To track a keyword’s trend on Google Search, you can use pytrends. Here’s an example that shows the interest in “data science” from 2019 to 2024:

from pytrends.request import TrendReq

pytrends = TrendReq(hl="en-US", tz=360)
pytrends.build_payload(kw_list=["data science"])

df = pytrends.interest_over_time()
df["data science"].plot(figsize=(20, 7))

Pytrends allows you to easily access various types of trend data, including interest by region, related topics, and queries, making it a powerful tool for researchers, marketers, and data analysts.

Link to pytrends.
Favorite

Exploring Google Trends with Pytrends API Read More »

Pipe: A Elegant Alternative to Nested map and filter Calls in Python

Pipe is a Python library that enables infix notation (pipes), offering a cleaner alternative to nested function calls. Here are some of the most useful methods from the Pipe library:

select and where (aliases for map and filter):

Python’s built-in map and filter functions are powerful tools for working with iterables, allowing for efficient data transformation and filtering. However, when used together, they can lead to code that’s difficult to read due to nested function calls. For example:

nums = [1, 2, 3, 4, 5, 6]

list(
filter(lambda x: x % 2 == 0,
map(lambda x: x ** 2, nums)
)
)

[4, 16, 36]

Pipe allows for a more intuitive and readable way of chaining operations:

from pipe import select, where

list(
nums
| select(lambda x: x ** 2)
| where(lambda x: x % 2 == 0)
)

[4, 16, 36]

In this version, the operations are read from left to right, mirroring the order in which they’re applied. The select method corresponds to map, while where corresponds to filter. This syntax not only improves readability but also makes it easier to add, remove, or reorder operations in your data processing pipeline.

traverse:

The traverse method recursively unfolds nested iterables, which is useful for flattening deeply nested lists:

from pipe import traverse

from pipe import traverse

nested = [[1, 2, [3]], [4, 5]]
flattened = list(nested | traverse)
print(flattened)

[1, 2, 3, 4, 5]

chain:

The chain method combines multiple iterables:

from pipe import chain

result = list([[1, 2], [3, 4], [5]] | chain)
print(result)

[1, 2, 3, 4, 5]

take and skip:

These methods allow you to select or skip a specific number of elements from an iterable:

from pipe import take, skip
from itertools import count

first_five = list(count() | take(5))
print(first_five)

[0, 1, 2, 3, 4]

skip_first_two = list([1, 2, 3, 4, 5] | skip(2))
print(skip_first_two)

[3, 4, 5]

Link to Pipe.
Favorite

Pipe: A Elegant Alternative to Nested map and filter Calls in Python Read More »

Effortless AI Chatbot Integration with Chatbase

Chatbots can significantly lower operational costs by automating routine tasks and inquiries. However, many existing chatbot solutions require extensive technical knowledge and are time-consuming to implement.

Chatbase simplifies the process, enabling you to quickly deploy a functioning chatbot.

Key features:

✅ Just upload your data, train the AI, and your chatbot is ready!

✅ Tailor your chatbot’s appearance to align with your website.

✅ Easily connects with platforms like Slack, WhatsApp, and Zapier.

Link to Chatbase.
Favorite

Effortless AI Chatbot Integration with Chatbase Read More »

Jurigged: Seamless Live Code Updates Without Restarts

Traditionally, modifying code requires stopping the running application, making changes, saving, and restarting – wiping out the application’s current state and data in memory.

Jurigged solves this by enabling live code updates while the application runs, allowing immediate changes without restarts or loss of state.

Link to Jurigged.
Favorite

Jurigged: Seamless Live Code Updates Without Restarts Read More »

Streamline Code Reviews with CodeRabbit

Code reviewing is crucial, but it can be tedious and error-prone when done manually. CodeRabbit, an AI-powered code review assistant, can speed up the process and minimize the chance of missing bugs.

Key Features:

Quick and detailed pull request summaries

Line-by-line analysis to spot issues and improvement opportunities

Quickly address problems with a single click

Real-time support for questions, code generation, and fine-tuning.

Link to CodeRabbit.

View the PR.
Favorite

Streamline Code Reviews with CodeRabbit Read More »

Kestra: Unleash the Power of Event-Driven Workflow Automation

Kestra has just introduced a new feature called Realtime Triggers that enables triggering workflows in response to events with millisecond latency.

For instance, with Realtime Triggers, a financial services company can call an ML inference endpoint to detect unusual transactions in real-time.

This rapid responsiveness is a new paradigm in orchestration that I haven’t seen yet in any other orchestration tool so far.

Link to Kestra.
Favorite

Kestra: Unleash the Power of Event-Driven Workflow Automation Read More »

Magika: Detect File Content Types with Deep Learning

Detecting file types helps identify malicious files disguised with false extensions, such as a .jpg that is actually malware.

Magika, Google’s AI-powered file type detection tool, uses deep learning for precise detection. In the following code, files have misleading extensions, but Magika still accurately detects their correct types.

Link to Magika.

Try out this code.
Favorite

Magika: Detect File Content Types with Deep Learning Read More »

0
    0
    Your Cart
    Your cart is empty
    Scroll to Top

    Work with Khuyen Tran

    Work with Khuyen Tran