Automate your Weekly Insights with Python
After some years in the data analytics field, the most important thing is not just to visualize the results and the data.
In the past few years, I have created more than a thousand Dashboards, most of them in Power BI, some of them were having a huge impact in each organization and some others only for one or two people. But even the report with high impact and more views it was and it is until today a dashboard where someone still needs to login somewhere find the report find some specifics KPIs and after that begin digging to see and find what really changed from the previous update or in the current month’s/week’s/day’s data.
What makes the difference?
The difference is that with a few lines of code you can just automate your report, send it via email if you want and apply machine learning (time series forecasting) to your data. Personalize the specific insights based on your data, on your department and your company’s targets.
What can you find below?
Below you can read:
- Creation of Dummy Dataset
- (Simple) Time Series Forecasting with Prophet
- YTD,MTD,MoM,WoW Key Performance Indicators (KPIs)
- Outliers in the latest month
- Compile all the above together
- A look at the Results
- How to run the code
- Some more ideas to implement
Of course, all of the above have been made with Python
The Dummy Dataset
For the given use case there are many datasets in the internet but in order to avoid any commercial issue the dataset has been created from zero with random data. In the create_df functions given start date, end date and number of rows for each product the code is creating random revenues and amount values. Off course all these with the power of Pandas and Numpy.
The “product ”categories are DSLR,Laptop,Smartphones,TVs,Gaming and for each of these product’s some manufacturers.
- DSLR: Nikon,Canon,GoPro,Sony
- Smartphones: Apple, Samsung, Xiaomi, Sony, OnePlus, Huawei, TCL, Vivo, Poco, Motorola, Google
- TV:Samsung, LG, Sony, Xiaomi, FU, Toshiba, Panasonic
- Laptop: Lenovo, Dell, HP, Apple, Huawei, Acer, Other
- Gaming: Sony, Microsoft, Nintendo, Steam
Now for each product the code creates the given number of rows and fill this dataframe with random choice of each product’s manufacturers and a random value for revenue and amount. There are some rules for the revenue and for the amount in order to not having a normalized data in the dataset.
Time Series Forecasting with Prophet
For the forecast calculation it has been used facebook’s Prophet, the use in this case is not to calculate the most accurate results or the most accurate machine learning model but just to give a simple forecast. In older personal use cases of prophet, this package is having good accuracy and is simple to use it in any case, in the code it just takes 6 lines of code to forecast. Keep in mind that you can save the accuracy results of the package and visualize them at the end(!).
YTD, MTD, MoM, WoW
These four KPIs are the “must” for time series monitoring in sales, product and many other departments in any organization.
- YTD: Year to Date is the comparison from the first day of the year until today with the exact same period last year.
- MTD: Month to Date is almost the same comparison as YTD but from the first day of the month.
- MoM: Month on Month is the comparison of the current month forecasted versus the last month.
- WoW: Week on Week is the comparison of the current week forecasted versus the last week.
*as described above the forecast is to use only for MoM and WoW. There is another metric of comparing current Month Forecasted vs the same month in the past year.
Outliers in the last month
Another metric that many people in the business need to watch is the new High Scores or the new Low Scores. It is so simple as you read it, if a product has a new all-time high or low score in the last month, then this is an insight where it would be nice to share in your company, in any other case just ignore and make your business as usual.
Compile all the above together
In the image above there is a high level of how the flow/code works, if you need to find more you can look in the repository that you can find in the references.
Results
For each given list of category, the code calculates the four KPIs that we have mentioned before for each product and manufacturer. The output for each one is the same as in the picture above. Please keep in your mind that in any case of comparing results, time periods are nice to highlight the Highs and Lows. Like the picture above where with red colour is the lowest and with green colour the highest. Some more results you can find below.
While for the new highs and low the output is really simple ( Imagine that if your business is growing up really quick the results in this one are going to be many!)
Give a Try
In the References there is the code. You can run it by using your data and load them in the function of automatedInsights. Change only with your dataframe the last 20 rows of the code and give a run.
- df: should be the dataframe
- categories: the columns from the dataframe
- dates: the datetime column
- value: the column which contains the data that you need to compare.
For any questions, please leave a comment.
More Ideas
Below are some ideas that based on the above you can implement in order to gain the maximum values from your data!
- Compare the Trend. MoM this Week vs the Last Week
- Compare Week on Week trends with Delta
- Create from the code a pdf Report and send it via email
- Schedule it with Cron Job in an ubuntu or with windows Task Scheduler
If you have loved this article just follow my profile and like the story! There will be more articles like this!
