Visualising data, is an important way to convey a message. However, I am ever more seeing animated graphs built from, occasionally, large, complex datasets. Sometimes the patterns they produce are really impressive and cannot be shown just as a static image or by simply looking at the data itself.
I became inspired (just a little bit) and decided to learn and create my own. This is not a comprehensive detailed guide but it will certainly give you the knowledge to make your own way through this data world.
First, I needed a dataset to analyse. Weilded with python, I started brainstorming… I’ve always been fascinated by RNG (random number generators) so picking a lottery was an obvious choice for making a small dataset from; it is largely available from multiple websites. I picked a site and set to work.
After downloading the HTML pages, I built a tool in python using a module called BeautifulSoup. The module basically takes inputs in the form of an HTML or XML DOM and parses it into objects. This allows you to navigate the page effeciently, to change or extract certain elements or the values within the element. After some tinkering, I was able to reduce a long HTML page down to the data I wanted to, with a few lines of code.
I’m going to be honest here. This part took most of my time up; BS documentation was a nightmare to navigate and understand…. The initial tutorial was helpful at first but towards the end I felt like many fundamentals have been left out. There are good tutorials out there and after reading a few, you get a sense of how dynamic BS can be. One trick I used was to assign a variable the results of a BS into a new list and parse the list back into a new BS object. ( To do this method, you’ll need to use encode() )
Another tip whilst working on data, I found an invaluable way to sort it by date. If (like this example) the data you’re working with has date/time values, it might be necessary and important to sort the data via date. I know this might not be the most efficiant way, but but quite effective nonetheless. One by one mask the date/time string with strptime() and change to Unix epoch. You can now easily sort the data. You can now loop and convert the epoch back to your desired date/time layout using strftime()
Sending the data from memory to a CSV or TSV spreadsheet will seal the deal and your ready to start making graphs!
To generate graphs, I used a python module called MATPLOTLIB. There are many tutorials out there. Get a good idea about how you would like to frame your data and what you will like to show. When you know that, play around with MATPLOTLIB until you have it plotting test data correctly and how you want.
Be sure to find out how to save as an image and close the method properly before generating the next graph slide. For a video file, you’ll probably want to save the first image as 0000.PNG and the next as 0001.PNG etc. as to easily encode into a video (and to keep them in order)
After generating your graphs, you can now easily encode the files into a video to create a graph which seems to move.
My first attempt and final product can be seen here. Have fun experimenting!