Lead Software Engineer, Data Platforms (Apache Iceberg, Opensearch)

• Your opportunity
  As a Lead Engineer for our Data Platform, you will be responsible for leading the design, implementation, and continuous improvement of our data infrastructure. Your expertise with Apache Iceberg, data lakes, OpenSearch, and distributed query engines like Trino will be instrumental in building a scalable, efficient, and robust platform that supports our analytics and data science initiatives.
•  
• What you'll do
• Lead the architecture and development of our Data Platform, ensuring scalability, performance, and reliability.
• Drive the integration of Apache Iceberg to manage large-scale analytic datasets with high concurrency and versioning capabilities.
• Architect and optimize data lakes for efficient data storage, retrieval, and management.
• Implement and fine-tune OpenSearch for search and analytics use cases across large datasets.
• Deploy and manage distributed query engines like Trino to enable high-performance data analysis across various data sources.
• Collaborate with cross-functional teams to understand data needs and deliver solutions that empower data-driven decision-making.
• Establish best practices for data modeling, ETL processes, and data governance.
• Lead performance tuning, benchmarking, and optimization efforts to meet the demands of complex data workloads.
• Stay abreast of emerging technologies and industry trends in the data platform space, evaluating and recommending new tools and approaches.
• Mentor and guide a team of talented data engineers, fostering a culture of technical excellence and innovation.

This role requires

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• Minimum of 8 years of experience in data engineering, with at least 2 years in a leadership role.
• Proven expertise in managing and architecting data lakes and implementing data lake house patterns.
• Strong experience with Apache Iceberg or similar table formats.
• Proficiency with search and analytics engines like OpenSearch or ElasticSearch.
• Hands-on experience with distributed query engines such as Trino, Presto, or Apache Spark.
• Solid understanding of cloud-based data storage and computing services (AWS, Azure, GCP).
• Experience with data pipeline and workflow management tools 
• Good understanding of distributed databases and query processing.

Bonus points if you have

• Active contributor to open source communities like apache iceberg and OpenSearch
• Prior experience working on distributed database and query processing.
• Experience with containerization and orchestration technologies (Docker, Kubernetes).
• Certifications in data engineering or cloud platforms.

Fostering a diverse, welcoming and inclusive environment is important to us. We work hard to make everyone feel comfortable bringing their best, most authentic selves to work every day. We celebrate our talented Relics’ different backgrounds and abilities, and recognize the different paths they took to reach us – including nontraditional ones. Their experiences and perspectives inspire us to make our products and company the best they can be. We’re looking for people who feel connected to our mission and values, not just candidates who check off all the boxes. 

If you require a reasonable accommodation to complete any part of the application or recruiting process, please reach out to [email protected].

We believe in empowering all Relics to achieve professional and business success through a flexible workforce model. This model allows us to work in a variety of workplaces that best support our success, including fully office-based, fully remote, or hybrid.

Our hiring process

In compliance with applicable law, all persons hired will be required to verify identity and eligibility to work and to complete employment eligibility verification. Note: Our stewardship of the data of thousands of customers’ means that a criminal background check is required to join New Relic.

We will consider qualified applicants with arrest and conviction records based on individual circumstances and in accordance with applicable law including, but not limited to, the San Francisco Fair Chance Ordinance.

Headhunters and recruitment agencies may not submit resumes/CVs through this website or directly to managers. New Relic does not accept unsolicited headhunter and agency resumes, and will not pay fees to any third-party agency or company that does not have a signed agreement with New Relic.

Candidates are evaluated based on qualifications, regardless of race, religion, ethnicity, national origin, sex, sexual orientation, gender expression or identity, age, disability, neurodiversity, veteran or marital status, political viewpoint, or other legally protected characteristics. 

Review our Applicant Privacy Notice at https://newrelic.com/termsandconditions/applicant-privacy-policy