The Matrix...now that's Big Data ;-)

The Matrix...now that's Big Data ;-)

I remember when I first came across the power of Big Data back in 2009 when I was starting my own ski guiding company (alongside being an IT recruiter) and I was preparing some facebook advertising to target backcountry skiers & snowboarders for my tours to a backcountry ski haven called Gulmarg nestled in the Himalayas of Kashmir, India.

I was choosing my target audience - the geographic location I wanted my audience to be from the UK, NZ, Aus, Europe etc, the age demographic 18-65 year olds and interests - 'skiing', 'snowboarding', 'telemarking', 'mountaineering' 'backcountry skiing' etc and then it spat out that the advert would serve to 268,000 people across the globe on Facebook....I was blown away....little old me running a small, very specialised backcountry ski company, which has a very specific niche of customer and I had 13 paying clients for my first season.

 

Snowboarding Powder in Gulmarg, Kashmir photo by Tim Silverwood

Snowboarding Powder in Gulmarg, Kashmir photo by Tim Silverwood

 

Is this BIG Data you say?

15-20 years ago you could not have imagined that kind of reach to a global audience.....Facebook went on to start their Applied Machine Learning team in September 2015 see: http://fortune.com/facebook-machine-learning/

 

It's a small example of how things have changed.  As humans, through social media we've never had so much detailed reachable personal profile information which is able to be accessed on the internet.  Let alone information that can be targeted by algorithms.  For many industries this has meant companies can segment this data and target niches for marketing serving up very accurate and relevant advertising. Of course there are many more uses as this new frontier of Big data starts to really kick in....

 

Big Data Meet up

I recently attended the Christchurch Big Data meet up listening to a very interesting talk about the increasing large amount of raw data per day a software company specialising in the CPM advertising community was gaining & how they went about scaling their environment with Big data tools.

 

As data started to grow......

IMG_6469.JPG

 

Apache Spark introduced......

 

What they learnt....

Come along to some great Big Data talks in Christchurch which you can find on meetup.com

 

So what skills & experience do you need to become a Big Data Developer / Data Scientist?

I bumped into one of the experienced Big Data Architects I know (and have placed) and asked him....

What are some good Big Data online courses / forums to follow?

This is an excellent course I found on Coursera: https://www.coursera.org/learn/machine-learning/#

Also there is a nice place which you will find the best blogs of the week here https://www.hadoopweekly.com

Regarding to core skills / experience what would be useful to transition into Data Science?

I would say knowing how to use (Apache) Spark and machine learning algorithms are the key.

 

Some further Big Data / Data Science courses on the internet:

Coursera: https://www.coursera.org/specializations/big-data#courses

Udemy: https://www.udemy.com/taming-big-data-with-apache-spark-hands-on/

 

So what do employers look for in a Big Data / Data Science candidate?

 

I've been working on Big Data for the past few years and the key things we've found employers look for are:

 

***Degree in Computer Science, Engineering or Maths, MSc or PhD in machine learning / algorithms

***Strong Coding skills in Java (fits with the Hadoop eco system), Python, R or C++

***Algorithms / Machine learning

***Experience in Big Data platforms like Hadoop / Apache Spark, tools like Apache Zeppelin, Cassandra, MapReduce, Redis, MongoDB, Kafka, Impala

***A passion for data but also a strong problem-solving & an inquisitive mind to get under the hood and work out why

 

How do I become a Data Scientist?

 

The other way to get into Big Data from a graduate perspective is to:

1) Study Data Science at University (or degrees / courses related to Big Data like Computer Science & Maths/Stats), you could also check out the Post Doc at Canterbury Uni please see: http://www.canterbury.ac.nz/future-students/qualifications-and-courses/masters-degrees/master-of-applied-data-science/

2) Join as a graduate coder within a company that specialises / works within the Big Data domain & thus look to progress into this area internally, once you have the product knowledge & your commercial coding skills are up to scratch

 

Current Big Data / Data Scientist roles that Sunstone is currently recruiting for on behalf of our clients:

Junior Data Scientist

Data Scientist

 

I'm founder & Principal Consultant of Sunstone, an IT Recruitment company specialising in software, web & mobile jobs based in Christchurch, New Zealand. You can contact me on: paul@sunstonetalent.com

Comment