We are seeking a senior software engineer to join our new Data Intake team. Our team owns the tooling that sources and processes the training data that Confidential (Registered Users Only)’s future foundation models are built on. As an engineer on the team, you’ll build out a new service that can obtain diverse datasets from a wide variety of external sources, process them into consistent formats, and store them in a reliable and easily accessible location for our models to leverage. You’ll collaborate closely with teammates across the company in data labeling and machine learning to ensure we deliver the datasets our customers need.
This role offers the chance to be a founding team member and help define both the technical direction and the long-term shape of the product area from the ground up. This role is a great fit for you if you enjoy building and iterating quickly on new internal product offerings, are proficient in designing and scaling robust API integrations and data infrastructure, and have a zeal for developing inspiring, easily maintainable, and reusable code. Join our team and make a significant impact at Confidential (Registered Users Only).
You Will:
- Architect and deliver a high-performance analytics environment that serves thousands of queries daily
- Design and scale robust data infrastructure for powering data retrieval and visualization
- Ensure data systems are reliable, observable, privacy-compliant, and performant
- Bridge the gap between data engineering, analytics, and infrastructure to deliver a suite of best-in class, fully featured, and easy to use data tools.
- Build intelligent agentic automation to simplify and accelerate analytical workflows
- Contribute to internal platforms and tools that abstract complexity, improve developer productivity, and accelerate innovation
- Be a tech lead and mentor junior engineers
- Self-organize and take ownership of projects throughout their full lifecycle
You Have:
- 5+ years of work experience building production-grade, scalable, and reliable data systems; B.S. equivalent in CS or sufficient experience.
- AWS infra understanding (VPC, endpoints, private link, Terraform, Amazon EKS)
- Infra networking (k8s ingress, load balancers, and VPC networking)
- Data infra understanding (Trino/Spark system configuration)
- Strong SQL and distributed engine knowledge
- Python and notebook data stack (e.g. pandas, ibis)
- Excellent cross-functional collaboration skills; enjoy building systems that enable and accelerate team productivity