The purpose of this article is to elaborate and visualize (surprise!) my comment I left over on Reddit: Professional data scientists: did you overcome the feeling of never knowing enough? If so, how?
I think this concept can be applied to any field - not just data science.
My personal advice that has worked for me to quell any “insecurities” is frame your mindset in terms of business value vs technical knowledge.
Providing business value and getting things into production or persuading others to act on your analyses should be your top priority. This should always be your North Star. How to get the ship moving in at least the right direction.
Technical knowledge comes from a mix of experience, experimentation, personal development and constant iteration. Find a problem, suggest a solution, deploy, look at the data then iterate or pivot. Be comfortable with the fact that you will never know everything. Most companies won’t need to be on the bleeding edge of data science.
Business value sounds nebulous but I see it as all activities that directly influence the trajectory of an organization. This could either be a new feature release, getting multiple stakeholder buy-in to align on the product roadmap for the next few years, or even working on changing the internal culture.
Technical knowledge, on the other hand, are the tools of the trade for a specific domain, company or position. For example, a plumber is expected to know how to use a cup plunger, flange plunger or drain snake. A software engineer would need to know how to write to different coding languages and technologies that exist out there.
We can plot the relationship between these two variables with business value on the x-axis and technical knowledge sitting on the y-axis.
Indeed, someone further along the right side of the business value continuum contributes relatively more than someone to the left. On the technical knowledge axis, ascending vertically indicates an increasing competency in the field.
To illustrate this we can use an outlandish example of a newborn baby - zero business value and technical knowledge so they would be placed on the bottom left side.
Taking our plumber example from earlier, an apprentice might offer minimal value but they are still learning the ropes so he or she would place slightly higher than our newborn baby.
For the master plumber, with years of experience they’d be able to diagnose the problem and determine which tool would be the best for unclogging a specific toilet issue: plunger or snake?
To reinforce this concept we can split up the grid and identify four distinct quadrants:
So, which one is more important? That depends on which side of the equation you sit on.
Organizations
It comes down to the immediate needs of the hiring manager.
For instance, a company looking for an enterprise sales representative might need someone to close leads as fast as possible so they would put more weight on candidates with higher business proficiency.
Conversely, a web analytics firm is likely to focus on the technical proficiencies of a potential hire if they need to automate their internal processes.
In both cases they will aim for subject matter experts but when it comes to a “one or the other” hiring decision they will fallback to what they need the most.
Individual Employees
The truth of the matter is most companies hire people to add value to the business rather than having them work on projects purely for academic pursuits.
Thus, it doesn’t matter where someone initially sits on this matrix but after securing the job the goal is to progress further to the right hand side of the business value continuum.
Providing business value and getting things into production or persuading others to act on your analyses should be your top priority. This should always be your North Star. How to get the ship moving in at least the right direction.
With time and experience most folks will see themselves gradually shift upward into the subject matter experts territory - this is where you eventually want to be.
Imposter Syndrome
I want to briefly touch on this topic since a fellow Reddit user left this response:
Among many other compelling reasons, the framework in which you bifurcate biz value vs tech knowledge was oddly comforting.
It is understandable why we have this growing phenomena because it seems like every other day there is a newer, better or faster way of doing something.
Unfortunately, this means we ( You ) put focus and compare our relative standing versus others ( P# ) only on the technical expertise spectrum:
Looking at this from a single dimension can be detrimental because it feels as though you’re lagging behind a rapidly changing landscape.
This is where the framework comes in: definitely learn about the new stuff so you don’t get complacent but have an unrelenting drive to produce business value.
For all we know the placement of social influencers might even look something like this:
If that doesn’t work then remind yourself where the money comes from to pay your bills. ;)