Big Data Engineer - Inside IR35

  • Category
    IT and Telecoms - Analyst Prog/Dev
  • Location
    City Of London, Greater London
  • Salary
  • External Reference
    JN-062021-188379
  • OrganizationType
    IT

Big Data Engineer
6 months contract
London (remote working initially)
Up to £800 per day (Inside IR35)

The role:
* Working closely with a data-centric application, hosting algorithms to detect possible market abuse
* Designing the ETL architecture, as we look to extract it from an existing legacy application. After that, building out additional ETL layers to support the onboarding of additional data sources.
* Working closely with quants/data scientists to ensure that they have the data necessary to add new algorithms, and that the data is of the necessary quality and timeliness to support these.
* Act as the subject matter expert regarding data pipelines to the DevOps focused team and to external stakeholders


Skills and experience:
* You have experience building data pipelines on top of Big Data technologies, ideally using Spark and Python
* You have experience working with message queues (ideally Kafka or Solace), traditional databases (SQL) and NoSQL databases (ideally MongoDB)
* You have experience working in a DevOps culture and willing to drive it. You are comfortable working with CI/CD tools (ideally IBM UrbanCode Deploy, TeamCity or Jenkins), monitoring tools and log aggregation tools. Ideally, you would have worked with VMs and/or Docker.



Spring Technology acts as an employment agency for permanent recruitment and an employment business for the supply of temporary workers. The Spring Group UK is an Equal Opportunities Employer.

By applying for this role your details will be submitted to Spring. Our Candidate Privacy Information Statement explaining how we will use your information is available on our website.

To speak to a recruitment expert please contact Joanna Morley