5 big data careers that just keep growing

“Big data” is currently one of the most sought-after areas of tech expertise. In fact, the US Bureau of Labor Statistics has concluded that throughout the 2020s, data roles - which currently have a median wage of over $100,000 - will be one of the top 10 fastest growing careers by 2030. With this growth, a career in big data is certainly an attractive prospect. However, at times, its various job titles can seem confusing. Here, we examine a range of big data careers and give an insight into the scope of each role.

Data analyst

A data analyst studies and analyzes general data and has several duties: mining, data collecting, scrubbing, processing, analyzing, preparing reports and visuals. It’s a role that requires a background in statistics, data collection, exploratory data analysis, and programming. However, data analysts do not necessarily work with big data as such, but rather handle data without taking it to macro levels.

A data analyst can understand, manage, and resolve any company matter that requires data being handled, or the creation of a dashboard with integrated metrics. However, a data analyst wouldn’t be the person to ask to investigate new algorithms or identify complex information requiring major expertise.

Let’s look at a fruit shop as an example: here, a data analyst can process the last six months of sales data for the shop and attempt to design the best sales strategy regarding stock, purchase volumes, days of the week, etc. Their usual work tools range from Excel or SPSS (the classic statistics program) to Microsoft Access, SAS and SAS Miner, SQL databases, and visuals in Tableau. The most common programming languages tend to be Python and R.

Data scientist

A data scientist can be seen as a natural progression for an experienced data analyst who can use raw data, even from multiple independent sources, and focus on predicting by using data. This person can design new algorithms and process big data.

If we look at the fruit shop example again, a data scientist would be able to collect data relating to fruit sales from the past 10 years, global fruit production, and a country’s economic outlook. From there, the data scientist can determine how to optimize fruit selling, what problems there currently are (even though these may not be apparent), and what consumer trends will be seen in the future. As a result, a data scientist can advise on the best way to manage the commercial offer, maximize profit, and be one step ahead of the game, for example selling dragon fruit before it becomes an Instagram trend, or suggesting switching suppliers based on what the data says, even if you hadn’t considered that need until now.

Data scientists are more common in big companies and will rarely be seen in small businesses. They have to work with machine learning and deep learning tools and libraries to build predictive models. They also work with SQL, R, and Python, as well as SAS Enterprise Miner and Hadoop among others.

Data architect

A data architect is fundamentally in charge of gathering raw data from different sources, whether this is in-house data (company data from a CRM, a web analytics dashboard, a sales dashboard, etc.) or external data (stock prices within a sector, industry sales reports, etc.).

Their role is also to design infrastructure that consolidates all this in a database as well as decide what data sources should be taken into account. This senior role makes decisions that are implemented by others (data engineers in particular). In the example of the fruit shop, a data architect is the person deciding what metrics relating to the fruit shop should be considered as well as any other external element (annual production for our country and other countries, per capita consumption for each type of fruit, CPI and wage trends, etc.) so that it can all be put into databases that other colleagues can then analyze.

A data architect’s background draws on various disciplines: they won’t just know about statistics, databases, and managing, they’ll also know about marketing and economics. They’re the ones examining the business with a broader perspective – as part of an ecosystem, where other stakeholders come into play.

Data engineer

Data engineers are also more experienced and more advanced roles than data analysts. Their job is much more technical and requires, in addition to the skills expected of a data analyst, the ability to create and integrate an API (integrations between different software that are subject to a series of rules and limitations). They should also have extensive knowledge of SQL databases, machine learning, and advanced programming.

A data engineer will develop solutions and infrastructure often designed by data architects – which is why you’ll often see software engineers in this position. They need to be experts in Hadoop, scripting, data architecture, SQL/NoSQL, and programming. In our fruit business, they’d be the person building the dashboard or the apps that the data architect requires so other data experts can analyze the information provided.

Chief Data Officer

The Chief Data Officer, or CDO, is the company’s leading figure when it comes to data. They’re in charge of developing the strategy for its use on behalf of all the other related roles. Their work takes place higher up the chain of command, making decisions and adopting a role more akin to a manager than someone working on the ground like most of the previous roles mentioned. A CDO thinks in terms of technology, business, and security, given that they understand data to be an asset for the company. They also establish data handling policies, meaning they are forced to keep a close eye on national and international data regulations and standards.

A Chief Data Officer needs to have extensive experience in data-related roles, meaning they need to have a background in mathematics, statistics, and programming, and even have an understanding of economics and marketing to have a greater overview of the role of data in the company.

In smaller companies, this position won’t exist, or at best, their role will be undertaken by the data analyst, depending on the knowledge in this field held by their manager, the business director, or the chief technology officer. As a company grows, they will take on a data officer who has a proven track record within the company, ample experience in making decisions and communicating with the other teams, and will ultimately be one of the highest-paid employees in the company.

If you’ve studied mathematics, statistics, and programming, you have the perfect academic background for working in one of these roles. It’s normal to start in a role such as a data analyst, analyzing past data and drawing conclusions based on what you find. Over time and with greater experience, you’ll soon be able to choose how to branch out. You may lean more towards a role that’s capable of making predictions such as a data scientist, one that’s responsible for choosing from multiple data sources such as a data architect, a more executive role that develops infrastructure such as a data engineer, or towards a role that’s further down the line and more ambitious such as a data officer.

Photo: Welcome to the Jungle

Translated by Jamie Broadway

Follow Welcome to the Jungle on Facebook on LinkedIn and on Instagram and subscribe to our newsletter to get our latest articles every day!