DATA ENGINEER position at Thoughtworks
Are you at your most vibrant when you’ve successfully distilled data into its simplest, most meaningful form? Application is the invited for the DATA ENGINEER at Thoughtworks.ThoughtWorks is a global software consultancy with an aim to create a positive impact on the world through technology.
Data Engineers develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions.
- You will partner with teammates to create complex data processing pipelines in order to solve our clients’ most ambitious challenges
- You will collaborate with Data Scientists in order to design scalable implementations of their models
- You will pair to write clean and iterative code based on TDD
- Leverage various continuous delivery practices to deploy data pipelines
- Advise and educate clients on how to use different distributed storage and computing technologies from the plethora of options available
- Develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions
- Create data models and speak to the tradeoffs of different modeling approaches
- You have a good understanding of data modeling and experience with data engineering tools and platforms such as Kafka, Spark, and Hadoop
- You have built large-scale data pipelines and data-centric applications using any of the distributed storage platforms such as HDFS, S3, NoSQL databases (Hbase, Cassandra, etc.) and any of the distributed processing platforms like Hadoop, Spark, Hive, Oozie, and Airflow in a production setting
- Hands-on experience in MapR, Cloudera, Hortonworks, and/or cloud (AWS EMR, Azure HDInsights, Qubole, etc.) based Hadoop distributions
- You are comfortable taking data-driven approaches and applying data security strategy to solve business problems
- Working with data excites you: you can build and operate data pipelines, and maintain data storage, all within distributed systems
- Strong communication and client-facing skills with the ability to work in a consulting environment