Recently I have become incredibly interested in the science behind data mining and the implications it has on our everyday lives. Every piece of data on the internet, in databases, and anywhere in public can be ‘mined’ to discover hidden patterns, useful observations, and invaluable insights. The range of applications for data mining are extensive, and it’s proliferation has had a profound influence on the world and in the way that modern businesses operate.
Despite all the hype around big data and data mining, I wanted to take a closer look and see if I could use it to learn something interesting about the coverage from various media organisations in New Zealand. I mined the Twitter feeds of Newshub, One News and Stuff News to see what they’re covering most, and most importantly, how their coverage differs.
The method:
- I created a Twitter app that I could use to download a copy of each news organisations feed over a time period of several months
- I used the R statistics package and various extensions to mine and visualise the data
The results:
Keep in mind, these results are from three media organisations over the same period of time. To help visualise what keywords they focused on I’ve created word clouds, with the keyword size increasing based on how often the word is mentioned.
Newshub:
One News:
Stuff.co.nz:
Without even looking at the individual news stories these word clouds can already give you a picture of the type of articles that have been presented recently. ‘Police’ and ‘Auckland’ consistently rank as two of the most discussed topics, with other keywords more randomly spread between the organisations.
But although this is interesting there is still more to see. I wanted to know how these stories are presented, and if there was any differences worth noting. With Auckland consistently being a hot topic I decided to drill deeper and see how the media portray Auckland in the news.
I ran an analysis that looked for the most common words used alongside stories about Auckland, and the results are quite different:
Top 10 keywords used alongside stories about Auckland:
Newshub:
Taxis, Youth, City, Brawl, Carjacking, CBD, Congestion, Dolphin, Drivers, Licensing
One News:
Brutal, South, Meth, Carjacking, Offenders, Corrections, Die, Cordon, Housing, Hospitalised
Stuff.co.nz
Housewives, Power, Outage, Properties, Winds, Mayoralty, Attackers, Motorway, Closures, Brawl
So what?
The keywords being used to describe Auckland differ quite wildly between the three outlets I looked into. It is interesting to note that it would seem that One News has paired Auckland with a large amount of negative phrases while Newshub and Stuff.co.nz have coverage of a range of keywords (both positive and negative). This was especially interesting as all of the exact same events have occurred, yet the way in which they are covered (or not covered) is completely different.
Putting aside the news organisations though, the real world implications of this type of data mining are huge. When was the last time you signed into your favourite online store, swiped a loyalty card or filled out a survey? Companies can use the exact same type of data mining I’ve just used to identify what you like the most and what combinations of products you respond to. Even if you don’t realise it, every action you take forms part of a pattern, and with the right tools companies can not only understand your pattern but predict it too.
—
Do you follow any of the news outlets I analysed? Are you surprised by the results? I’m interested to hear what you think about this topic and what the implications of data mining mean for you, your business or your community.
About the author:
Angus Allan
Managing Director of Hello Digital
Angus is an experienced and passionate digital entrepreneur who has worked with clients around the world on a wide range of E-commerce and retail projects. Angus is excited about technology, change, and transformation, and has spoken about these topics, and more, at various events, workshops, and classes in New Zealand. As Hello Digital's Managing Director he oversees the day-to-day operations of the company and is the primary point of contact for our consulting clients and international partners.