If you want to assign values to multiple new columns, instead of assigning them separately, you can do everything in one line of code with df.assign.
In the code above, I first created col3 then use col3 to create col4. Everything is in one line of code.
Favorite
Assign Values to Multiple New Columns Read More »
If you want to reduce the memory of your pandas DataFrame, start with changing the data type of a column. If your categorical variable has low cardinality, change the data type to category like below.
The memory is now is reduced to almost a fifth of what it was!
Favorite
Reduce pd.DataFrame’s Memory Read More »
Outliers are unusual values in your dataset, and they can distort statistical analyses. If you want to trim values that the outliers, one of the methods is to use df.clip.
Above is how to use the 0.5-quantile as the lower threshold and .95-quantile as the upper threshold
Favorite
If you want to get all data starting in a particular year and exclude the previous years, simply use df.loc['year':] like above. This works when the index of your pd.Dataframe is DatetimeIndex.
Favorite
Pandas DataFrame Get Data in a Year Range Read More »