Join our upcoming online event - "CDO TechVent: Data Observability- Thursday, August 25, 11:30 a.m. Eastern Time.

Options for Building a Predictive Analytics Program

The good news and the bad news is that that amount of business data now collected is gigantic! Experfy (a marketplace for Predictive Analytics Professionals (PAPs)) reports that Sprint handles some 250,000 phone calls every minute from its customers’ mobile phones and that these data producing objects (phones) are now the dominant intelligent lifeform on the planet! That is to say, there are now more mobile phones in the world than humans. Add to that the social media data and marketing data that can now be collected on the internet and we are collecting hundreds of times more data on our customers and our businesses than we did even ten years ago.  The good news is that the more data you have the better the predictive models and other analytics will be (more data generally lowers the error rates). The bad news is that the shear effort required in moving that data around, storing it and analyzing it can seem daunting.

 What you need are some good people. Especially some good people who are well versed in the field of predictive analytics.

It’s Hard to Find Good People

Using predictive analytics in your company has many advantages like being able to predict customer behavior. But to do predictive analytics you most often need to collect a team of highly skilled people who are currently in great demand. These predictive analytics professionals (to use one of many different terms that might describe them) require special care and feeding and are difficult to find in the job market. So you might consider whether you could outsource these folks or find some other way to acquire this talent.

The reason that the people who can make sense of all this big data are in such high demand is that the amount of data is doubling every two years but the number of people skilled in analyzing that data is not.

McKinsey and Abi Research report that there will be a shortage of up to 190,000 data scientists by 2018. And a predictive analytics professional (PAP) is a more highly trained version of a data scientist. They are even more rare and our universities aren’t making enough of them to keep up with the demand. In fact, a report from Burtch Works shows that 57% of the world’s supply of predictive analytics professionals at the lowest end of the compensation scale are working in the United States but they are not U.S. Citizens. In fact Experfy reports that 86% of the new analytics jobs will be created in the US, UK and Japan while 74% of the analytics talent will come from elsewhere (e.g. India, China and Brazil). This is a mismatch of supply and demand where the need for PAPs is in the United States but they are physically located in other countries.

You Need Critical Mass to Build an In-House Team

Given this challenge of location and scarce human resources, the simple solution is to leave the PAPs where they live and just outsource your work to them. This is a pretty smart strategy as you can get started much more quickly, be more nimble and it may also be a long term strategy if you don’t have enough need to build a large PA team in house.

PAPs are concentrated in just a few industries – more than half of the need for them is in just two industries: 28% of PAPs are in financial services and another 25% of them are in marketing or advertising. If you are not in one of those two industries it can be challenging to hire (and retain) these predictive analytics professionals because you may not provide the critical mass or expertise to be able to attract them or manage them. For example, professionals want to be surrounded by like professionals that have similar core competencies so that they can learn skills and advance their own careers.  So it may be difficult to attract a PAP to your internal group if it is not large enough.

It is also hard to manage these super-scientists if you don’t have a deep understanding of what they are doing and how they do it. To have the critical mass necessary to attract these PAPs, your company doesn’t have to have predictive analytics as its top core competency (as, for instance, you might argue that Google does) but it is critical that there be enough warm bodies. A team of five is way better than one or two. But even five doesn’t provide much of a path for career advancement within the company. So before you think about a long term plan about creating a predictive analytics (PA) group consider whether that group will be large enough to sustain itself – or if you are willing to incur the higher costs of over-paying for top tier PAPs and also incur the cost of higher than normal turnover.

To Outsource or Not to Outsource

Given the challenges of finding top Predictive Analytics Professionals you should be considering outsourcing or using consultants rather than employees to start off with.  If so, you might find that you have these questions about whether to outsource or keep things in-house: 

  1. “How do I know when I should bring my predictive analytics in-house versus outsource?”
  2. “When am I out of my depth with self-service tools and need to call in an expert?”
  3. “How do I find and vet qualified predictive analytics professionals whether for in-house or as consultants?”
  4. “What are the top benefits and risks of each approach?”
  5. “What is actually different about a self-service predictive analytics offering?  Could I use one?”

I’ll go over the detailed answers to these questions in future articles but suffice it to say that if you are posing these questions then the best way to start is to find a top-tier consultant and begin with an outsourced solution.

Outsourcing Is Not Your Only Long Term Option

After you get going with a consultant and an initial outsourced solution you can then decide what to do next. And you have quite a few choices for how to build your team. I’ll list them here moving from those that require the most effort but deliver the most control, to the least effort and the least control and flexibility:

  1. You can hire your own predictive analytics team in-house.
  2. You can train existing employees.
  3. You can hire consultants who work for you at your company.
  4. You can hire consultants who are outsourced and offshore and never come to your actual place of business.
  5. You can utilize specialized outsourced systems that have crowds of highly qualified professionals
  6. You can do it yourself by utilizing self-service predictive analytics solutions.
  7. You can do it yourself with automated predictive analytics solutions.

The right long-term solution really depends on you and your specific business needs and resources. But even if you are sure you want to eventually build a large group internally you should probably start with consultants. You can learn a lot from them and you can grow from them into a powerful in-house team over time.

Stephen J. Smith

Stephen Smith is a well-respected expert in the fields of data science, predictive analytics and their application in the education, pharmaceutical, healthcare, telecom and finance...

More About Stephen J. Smith

Books by Our Experts