Motivation

Testing Python code that interacts with external services, such as databases, can be challenging. The behavior of these services might vary over time due to data changes, configuration issues, or network problems. This introduces inconsistencies and makes tests unreliable, affecting overall test readability and efficiency.

For example, consider the following test that retrieves a user from a PostgreSQL database:

# %%writefile test_postgres.py
import psycopg2


def fetch_user_from_db(user_id):
    conn = psycopg2.connect(
        dbname="testdb", user="testuser", password="password", host="localhost"
    )
    cur = conn.cursor()
    cur.execute("SELECT * FROM users WHERE id = %s;", (user_id,))
    result = cur.fetchone()
    cur.close()
    conn.close()
    return result


# Example test without mocking
def test_fetch_user_from_db():
    user = fetch_user_from_db(1)
    assert user == (1, "John Doe")

Running this test without mocking can result in errors if the database is unavailable, credentials are incorrect, or the target data is missing. For example:

E       psycopg2.OperationalError: connection to server at "localhost" (::1), port 5432 failed: could not initiate GSSAPI security context:  The operation or option is not available: Credential for asked mech-type mech not found in the credential handle
E       connection to server at "localhost" (::1), port 5432 failed: FATAL:  role "testuser" does not exist

Such failures are unrelated to the logic being tested and make debugging difficult. This is where pytest mocking comes into play, offering a solution for more reliable and isolated unit testing in Python.

Introduction to pytest-mock

pytest mock is a pytest plugin that provides a mocker fixture. The pytest_mock library enhances the capabilities of unittest.mock, making it easier to use in pytest-based test suites.

To install pytest-mock, run:

!pip install pytest pytest-mock

In this Python mock tutorial, we will cover how pytest-mock can simplify mocking external services, such as database connections, and improve overall test scenarios. We’ll explore various pytest-mock examples to demonstrate its versatility in Python testing.

Mocking External Services with pytest-mock

Using pytest-mock, we can mock database connections and queries to make tests predictable and independent of external factors. Let’s rewrite the test to mock the PostgreSQL database connection using the pytest mocker fixture:

# %%writefile test_postgres.py
import psycopg2


def fetch_user_from_db(user_id):
    conn = psycopg2.connect(
        dbname="testdb", user="testuser", password="password", host="localhost"
    )
    cur = conn.cursor()
    cur.execute("SELECT * FROM users WHERE id = %s;", (user_id,))
    result = cur.fetchone()
    cur.close()
    conn.close()
    return result


def test_fetch_user_from_db_with_mock(mocker):
    # Mock psycopg2.connect using pytest mocker.patch
    mock_conn = mocker.patch("psycopg2.connect", autospec=True)
    mock_cursor = mock_conn.return_value.cursor.return_value
    mock_cursor.fetchone.return_value = (1, "John Doe")

    # Call the function
    user = fetch_user_from_db(1)

    # Assertions
    mock_conn.assert_called_once()
    mock_cursor.execute.assert_called_once_with(
        "SELECT * FROM users WHERE id = %s;", (1,)
    )
    assert user == (1, "John Doe")

Here’s how it works:

1. mocker.patch: Replaces psycopg2.connect with a mock object, preventing any real database interaction. This is an example of python mocker patch in action, demonstrating how to use pytest patch object for mocking.
2. mock_cursor.fetchone.return_value: Defines a simulated return value for the database query, showcasing how to set up mock objects and their attributes.
3. Assertions: Verifies that the mocked connection and cursor are used correctly and the query is executed with the expected parameters. These assert statements are crucial for effective unit testing and test efficiency.

Running this test produces a controlled and deterministic output:

• The test passes without requiring a real PostgreSQL database.
• There is no dependency on external configurations or data, ensuring test isolation.

Use Cases for pytest-mock

The pytest-mock library is versatile and can be used in various scenarios to mock external systems in Python data science projects:

Mocking External APIs

For applications that interact with web APIs, pytest-mock can mock requests to ensure consistent responses:

import requests


def fetch_weather(city):
    response = requests.get(f"http://api.weather.com/{city}")
    return response.json()


def test_fetch_weather(mocker):
    mock_get = mocker.patch("requests.get")
    mock_get.return_value.json.return_value = {"city": "Paris", "temperature": 25}

    weather = fetch_weather("Paris")

    mock_get.assert_called_once_with("http://api.weather.com/Paris")
    assert weather == {"city": "Paris", "temperature": 25}

Mocking File Operations

When working with file-based data, pytest-mock can simulate file read/write operations for testing:

def read_data(file_path):
    with open(file_path, "r") as file:
        return file.read()


def test_read_data(mocker):
    mock_open = mocker.patch("builtins.open", mocker.mock_open(read_data="test_data"))

    data = read_data("dummy.txt")

    mock_open.assert_called_once_with("dummy.txt", "r")
    assert data == "test_data"

Conclusion

pytest-mock is a powerful tool for simplifying and standardizing tests that involve external dependencies in Python. By mocking these dependencies, you can ensure that your tests are reliable, fast, and easy to debug. Whether you’re working with databases, APIs, or file systems, pytest-mock provides the utilities to make testing seamless and predictable.

Link to pytest-mock.