Data Engineer

Location Connecticut
Discipline: Backend
Job type: Permanent
Salary: £140,000 - $200,000 DOE
Contact name: Jessica Ifill

Contact email:
Job ref: 32081
Published: 4 months ago
Startdate: 26/05/2021

Tech-stack: Java/Scala/Python/Spark/Kafka/Hive/Samza


The role offers a unique opportunity to join an exceptional team-building major platforms for their global payment systems.  The company requires data engineers who have been exposed to messy data sets and have been successful in structuring them to be useful for AI products. 


As a data engineer, you will be responsible for writing frameworks building for real-time and batch pipelines to ingest and transform events. You will support by optimizing feature pipelines for fast execution and work alongside software engineers to build event-driven microservices. You’ll have the freedom to explore and experiment with new frameworks to learn and grow. 


What you’ll do: 

  • Working directly with Machine Learning Engineers and Platform Engineering team to create reusable experimental and production data pipelines. 
  • Keeping data whole, safe, and flowing with expertise on high volume data ingest and streaming platforms ( i.e. Spark Streaming, Kafka, etc) 
  • Understand, tune and perfect to processing engines (Spark, Hive, Samza) for daily use 
  • Explore as many new technologies as possible for data, processing, storage
  • Develop and contribute to open source 


Who you are: 

  • Proficient in Java/Scala/Spark/python
  • Degree in Computer Science, Engineering or a related field 
  • Experience in building serious data pipelines
  • Previous experience in Spark or Kafka 
  • Eager to learn and grow 


What’s in it for you: 

  • Competitive salary 
  • Remote working 
  • The company has been awarded ‘Great place to work’ 
  • Flexible hours around core working hours 


If you feel like this is something you would like to be involved in and make an impact on, feel free to get in contact now! |