- Type of contract: B2B/FTE
- Working model: fully remote
Our client builds tools for 100+ brands and retailers in the e-commerce space that help them offer free two-day shipping, same-day delivery, and product expansion into new marketplaces – keeping them several steps ahead of the curve in a rapidly changing industry. They are a purpose- and culture-driven organization that prides itself on connectivity, equality, diversity, and inclusion. They are headquartered in Chicago, with offices in New York, Conshohocken, PA (Philly area), and Krakow, Poland, and we operate as a subsidiary of one of the biggest logistics companies.
About the role:
The Data Engineer plays a pivotal role within the company, focused on driving engineering innovation, helping define and build the organization and leading the delivery of key business initiatives. S/he acts as a “universal translator” between IT, business, software engineers and data scientists, collaborating with these multi-disciplinary teams. The Data Engineer will contribute to the adherence of technical standards for data engineering, including the selection and refinements of foundational technical components. S/he will work on those aspects of the platform that govern the ingestion, transformation, and pipelining of data assets, both to end users and into data products and services that may be externally facing. Day-to-day, s/he will be deeply involved in code reviews and large-scale deployments. S/he will also provide mentorship and guidance to junior engineers to support the continued training and up-skilling of the Data Engineering team.
Your Duties & Responsibilities:
- Understanding in depth both the business and technical problems
- Building tools, platforms and pipelines to enable teams to clearly and cleanly analyze data, build models and drive decisions
- Scaling up from “laptop-scale” to “cluster scale” problems, in terms of both infrastructure and problem structure and technique
- Delivering tangible value very rapidly, collaborating with diverse teams of varying backgrounds and disciplines
- Championing the adherence to best practices for future reuse in the form of accessible, reusable patterns, templates, and code bases
- Interacting with senior technologists from the broader enterprise and outside of the company (partner ecosystems and customers) to create synergies and ensure smooth deployments to downstream operational systems
Skill/Knowledge Considered a Plus:
- Technical background in computer science, software engineering, database systems, distributed systems
- Fluency with distributed and cloud environments and a strong understanding of how to balance computational considerations with theoretical properties
- Detailed knowledge of the Microsoft Azure tooling for large-scale data engineering efforts and deployments is highly preferred
- A track record of designing and deploying large-scale technical solutions, which deliver tangible, ongoing value
- Direct experience having built and deployed robust, complex production systems that implement modern, data scientific methods at scale
- Ability to context-switch, to provide support to dispersed teams that may need an “expert hacker” to unblock an especially challenging technical obstacle, and to work through problems as they are still being defined
- Demonstrated ability to deliver technical projects with a team, often working under tight time constraints to deliver value
- An ‘engineering’ mindset, willing to make rapid, pragmatic decisions to improve performance, accelerate progress or magnify impact
- Comfort with working with distributed teams on code-based deliverables, using version control systems and code reviews
- Ability to conduct data analysis, investigation, and lineage studies to document and enhance data quality and access
- Use of agile and DevOps practices for project and software management including continuous integration and continuous delivery
- Demonstrated expertise working with some of the following common languages and tools:
- Spark (Scala and PySpark), HDFS, Kafka and other high-volume data tools
- SQL and NoSQL storage tools, such as MySQL, Postgres, Cassandra, MongoDB and ElasticSearch
- Pandas, Scikit-Learn, Matplotlib, TensorFlow, Jupyter and other Python data tools
Our Requirements:
- Bachelor’s Degree in Information Systems, Computer Science or a quantitative discipline such as Mathematics or Engineering and/or equivalent formal training or work experience.
- Hands-on experience in measurement and analysis, quantitative business problem solving, simulation development and/or predictive analytics.
- Extensive knowledge in data engineering and machine learning frameworks including design, development and implementation of highly complex systems and data pipelines.
- Extensive knowledge in Information Systems including design, development and implementation of large batch or online transaction-based systems.
- Strong understanding of the transportation industry, competitors, and evolving technologies.
- Experience providing leadership in a general planning or consulting setting.
- Experience as a senior member of multi-functional project teams.
- Strong oral and written communication skills.
- A related advanced degree may offset the related experience requirements.
What we offer:
• Great partners to work with – sharp engineering minds working on challenging projects
• Work in an atmosphere of mutual trust and an open, inclusive culture
• Benefits like life insurance, sports card, private healthcare, lunch subsidiary
• Nice equipment: strong Macs, two screens, and additional devices you need
• Development budget (9k PLN per person per year) to further grow your skills