Testing

It can be frustrating to wait for a lot of tests to run before knowing the status of the tests. If you want to instantly see the failures and errors with a progress bar, use pytest-sugar.

pytest-sugar is a plugin for pytest. The code above shows how the outputs will look like when running pytest.

Link to pytest-sugar.

Link to previous tips on testing in Python.
Favorite

When testing the outputs of your functions, it can be frustrated to see your tests fail because of something you don’t care too much about such as:

order of items in a list

different ways to specify the same thing such as abbreviation

exact value up to the last decimal point, etc

Is there a way that you can exclude certain parts of the object from the comparison? That is when DeepDiff comes in handy.

Link to DeepDiff.

Link to my previous tips on testing.
Favorite

If you want to benchmark your code while testing with pytest, try pytest-benchmark. The code above shows what pytest-benchmark does.
Link to pytest-benchmark.
Link to the source code. Favorite

When working on a data science project, don’t assume that the output will be what you expect. Test your assumption. The easiest way to do that in Python is pytest.
For example, in the code above, I use pytest to test my assumption about the output of the text. When running pytest file_name.py, pytest will run the functions that start with test_.
By running the test, I know where can my function goes wrong.
Check out other examples in my article about pytest.Favorite

When testing your function using Pytest, you might want to print only failed tests instead of both failed and successful tests. If so, add –lf to pytest. “lf” stands for “list fails”.
If you are not aware of Pytest, it is the framework that makes it easy to write small tests in Python. You can find more about Pytest from my previous article.Favorite

Sometimes, you might want to use the same data to test different functions. If you are using pytest, use pytest fixtures to provide data to different test functions instead.
In the code above, I used the decorator @pytest.fixture to create the data I want to use many times.
Have this data as the input for any test function that uses this data!Favorite

It is a good practice to test your functions to make sure they work as expected, but sometimes you need to test 100 times until you found the rare cases when the test fails. That is when pytest-repeat comes in handy.
Yesterday, I tested with 100 instances and found 2 failed tests! I’m glad that I didn’t settle down for 10 passed tests. You can find the doc for pytest-repeat hereFavorite

Your code might work with data at a certain state but might not work if your data changes. How do you detect the change in data before running into errors or encounter the drop in your model performance? That is when Schema comes in handy.
Below is the code snippet I used to validate the datatypes and the range of the values in a column. If this is what you are looking for, check out my article on how to use Schema here.Favorite