Full Job DescriptionRoles and Responsibilities
Dear,
Greeting for I Square Soft,
Hiring for “Big Data ETL Engineer” – for Bangalore location,
Experience : 3 – 9Years
Notice Period : Immediate – 30 days (serving NP Only)
Skills : ETL + Sprak
JD Is below :
We are looking for a Big Data ETL Engineer that will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across our clients.
Responsibilities
Develop and maintain data pipelines implementing ETL process, monitoring performance and advising any necessary infrastructure changes.
Design and develop ELT pipelines using Big Data tech stack Sqoop, Spark, Hive, Python.
Translate complex technical and functional requirements into detailed design.
Should be able to perform optimization (performance tuning, query optimization) of spark applications on Hadoop cluster.
Investigate and analyze alternative solutions to data storing, processing etc. to ensure most streamlined approaches are implemented.
Serve as a mentor to junior staff by conducting technical training sessions and reviewing project outputs.
Skills and Qualifications
Strong understanding of data warehousing and data modeling techniques.
3+ years of hands-on programming experience in Python and Spark/Pyspark
Must have practical experience in developing spark applications and complex data pipelines using spark RDD, DataFrames, Spark SQL
Practical hands-on experience on Big Data & Hadoop – Hive, HDFS, Oozie, and MapReduce
Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
Experience with Spark, SQL, and Linux.
OOPS concepts of Python
Experience with various messaging systems, such as Kafka or RabbitMQ.
Have experience writing automated unit test for Spark ETL based data pipelines
Role:Software Developer
Salary: Not Disclosed by Recruiter
Industry:IT Services & Consulting
Functional Area:IT Software – Client/Server Programming
Role Category:Programming & Design
Employment Type:Full Time, Permanent
Key Skills
Big DataSprakETL
Education-
UG:Any Graduate
PG:Any Postgraduate