Lead Data Engineer

Design and deliver robust, production-grade data-intensive applications and services that power business-critical use cases at scale. This role combines strong software engineering fundamentals with deep data engineering expertise and technical leadership. Bachelor's degree in computer science, Computer Engineering, relevant technical field, or equivalent; Master's degree preferred. Roles & Responsibility: - Operates effectively with minimal supervision, demonstrating strong ownership and accountability. - Translates high-level business requirements into well-designed, scalable, and robust systems and applications. - Exhibits exceptional data and software engineering expertise, drives architectural decisions and technology stack selection. - Establishes high-quality engineering standards, and ensures adherence through thorough code reviews and best practices. - Independently leads discussions with stakeholders, gathers and refines requirements. - Design end-to-end systems (ingestion, processing, storage, serving) with minimal supervision. - Build and maintain high-quality data products and pipelines on Spark and Databricks. - Design event-driven architectures using Kafka and related technologies for real-time data processing. - Build backend services and APIs that expose data capabilities reliably and securely. - Build and optimize data models for analytical workloads (dimensional modeling, data vault). - Write clean, testable, and maintainable software in Scala (preferred) or Java. - Drive architecture and design discussions with engineers, architects, and stakeholders. - Coach and mentor junior engineers through code reviews, design sessions, and hands-on pairing. - Improve engineering standards across testing, observability, CI/CD, reliability, and documentation. - Deliver production-quality software with strong automated test coverage and clear quality gates. - Use design reviews and lightweight ADR-style documentation for major technical decisions. - Build observability into systems from day one (logs, metrics, tracing, alerts). - Deliver through CI/CD with safe release and rollback strategies. Skills: - 8+ years of professional software engineering experience, including substantial backend and data platform work. - Strong programming background in Scala or Java (Scala preferred). - Proven experience building distributed data pipelines with Spark and Databricks. - Hands-on experience with Kafka in production systems. - Strong understanding of distributed systems concepts and failure handling. - Solid experience designing and developing APIs (REST and/or event-driven service interfaces). - Strong knowledge of relational databases (PostgreSQL), data modeling, and query optimization. - Demonstrated ability to design high-performance, concurrent systems. - Experience leading technical design and architecture decisions across teams. - Experience mentoring or coaching junior engineers. Good To Have: - Practical experience with Flink and advanced stream processing patterns. - Experience with cloud-native data platforms and infrastructure-as-code. - Experience with building & deploying applications on Kubernetes. - Experience with data governance, security, and compliance in data platforms. - Familiarity with table formats and query engines in modern lakehouse ecosystems. - Experience with performance profiling and JVM tuning. Collaboration Scope: - Partner with product, platform, and domain teams to translate requirements into resilient technical solutions. - Work closely with architects and technical leadership to shape platform direction. - Support incident response and root-cause analysis for critical production issues. - Help define roadmaps that align business priorities with engineering sustainability.