Senior Big Data Engineer

Job Description

Senior Big Data Engineer:
On behalf of our Financial Services client, Procom is searching for a Senior Big Data Engineer for a 7 month contract role. This position is a hybrid position with 2 days onsite at our client’s Mississauga office.

Senior Big Data Engineer - Job Description:
Our Financial Services client is seeking a Senior Big Data Engineer to design, build, and optimize large-scale data pipelines and distributed data systems that power critical business intelligence across the organization. Based in Mississauga and operating in a hybrid model, this role involves working within a high-performing engineering team to enhance the reliability and performance of  data infrastructure. 

Senior Big Data Engineer - Responsibilities:
• Build and maintain scalable data pipelines using PySpark within a Big Data environment to process and transform large volumes of structured and unstructured data.
• Design and develop solutions across the Hadoop ecosystem, including Hive, HDFS, Sqoop, Spark, Impala, and Scala.
• Develop and manage real-time and batch data workflows using streaming data platforms.
• Write complex SQL queries to extract, validate, and analyze data across distributed systems.
• Design and implement data models and data architecture patterns aligned with data warehouse principles.
• Automate pipeline scheduling and orchestration using shell scripting and Autosys.
• Independently identify, assess, and resolve technical risks and data issues in a timely manner.

Senior Big Data Engineer - Mandatory Skills:
• Hands-on expertise in PySpark and Big Data processing.
• Practical knowledge of the Hadoop ecosystem, including Hive, HDFS, Sqoop, Spark, Impala, and Scala.
• Proficiency in complex SQL query development.
• Solid understanding of distributed systems architecture.
• Demonstrated knowledge of data modeling and data design.
• Competence in shell scripting and job scheduling using Autosys.
• Strong analytical and problem-solving ability.

Senior Big Data Engineer – Nice-to-Have Skills:
• Experience with DataBricks.
• Experience in BI Platforms, preferably Tableau.

Senior Big Data Engineer – Assignment Length:
This is a 7 month contract position.

Senior Big Data Engineer - Start Date:
ASAP.

Senior Big Data Engineer - Assignment Location:
Mississauga, Ontario, Canada. This is a hybrid position with 3 days in office.

This employer uses both human and technology-assisted tools to support candidate screening and assessment. Final hiring decisions are made by people.