Explore Data Science Academy says there is a growing need for data engineering skills due to recent investments by Microsoft and AWS in on-premises data centers and the tripling of demand for public cloud services between 2017 and 2022 .
As a result, the academy has added data engineering to its online training offerings. Payscale estimates that the average salary for a data engineer in South Africa is Rand 441,913, rising to around Rand 750,000 for a senior position.
The 12-month course, which begins Monday, January 18, will provide candidates with the specific skills needed to become data engineers – a key role in a company’s successful digital transformation.
“The use of large amounts of data by data scientists now requires people with specialized skills to organize the data and ensure its quality, security and availability for the organization. They are data engineers, ”said Shaun Dippnall, CEO of Explore.
According to Dippnall, besides skills, there is a dearth of online training courses that offer both content and practical application of the data engineering skills required in the workplace. This is especially true of the South African market.
Course participants will learn:
- SQL basics;
- Python programming;
- Data modeling; and
- Cloud computing.
To these will be added specific skills in data architecture such as:
- Big Data Concepts;
- Data warehousing and lakes; and
- Data automation.
“In today’s digital world, data engineering has become a specific job category as the amount of data produced by companies engaged in digital transformation grows exponentially,” said Dippnall.
Two distinct roles
“Businesses today are producing new types of data in large quantities. While data scientists have traditionally been expected to build the infrastructure and data pipelines necessary to do their jobs, the volume and speed of data production has resulted in the roles of scientist and data engineer become separate and distinct, ”he said.
“It is now recognized that businesses need both data scientists and data engineers in an advanced analytics team. Naturally, collaboration between the two roles is frequent, but their skills and tools remain different. “
Essentially, data engineers design and build pipelines that transform and transport data into a format that, by the time it reaches data scientists, is usable.
These pipelines take data from many sources and collect it in a single warehouse that uniformly presents the data as a single source.
Data center stimulation
The importance of data engineering is underscored by the fact that two of the world’s largest cloud computing hyper scalers – Microsoft and Amazon – have recently established data centers in South Africa.
Microsoft launched two Azure cloud data centers in South Africa in 2019 to meet the growing demand for hyperscale cloud infrastructure and services in the region.
While Amazon Web Services (AWS) launched AWS Africa (Cape Town) Region – the first company in Africa – in April of last year.
According to AWS, the AWS Africa (Cape Town) Region will drive innovation across the country.
“South African organizations, from start-ups to businesses and the public sector, now have infrastructure in their countries to take advantage of advanced technologies such as analytics, artificial intelligence (AI), databases and the Internet of Things (IoT), ”he said.
Additionally, the addition of the AWS Africa (Cape Town) Region will enable enterprises to provide lower latency to end users in Sub-Saharan Africa.
Explosive growth expected
“These critical developments from two of the world’s largest companies underscore how important data warehousing is in the modern digital environment,” said Dippnall.
The Explore Data Engineering course requires applicants to have basic analytical training and will involve 10 hours of study per week.
“At Explore, we have always been aware of the need to create jobs for young talented and hardworking South Africans,” said Dippnall.
Spending on public cloud services in South Africa will almost triple in five years from 2017 to 2022, according to a study by International Data Corporation.
“Crucially for us, this study estimates that the exponential adoption of cloud services will generate 112,000 net new jobs in South Africa by the end of 2022,” he said.
Read: Teraco starts working on Africa’s largest data center