Senior Software Engineer

Software Engineer - Data Infrastructure (Senior / Staff) / Rust, C++, Python, DataFusion, ClickHouse, DataBricks

 

Location: New York, NY

Employment Type: Full-time, In-office

 

About the Role

My Client is seeking a senior-to-staff level Software Engineer to help design and scale the core data infrastructure behind a rapidly growing platform handling high-volume telemetry and analytical workloads.

 

This role focuses on evolving an existing analytics system into a scalable, cost-efficient, multi-tenant data platform capable of supporting thousands of customers. You’ll work on foundational infrastructure problems at the intersection of distributed systems, databases, and performance engineering, leveraging open-source technologies and proven industry patterns.

 

The ideal candidate enjoys shaping architectural direction, incrementally improving systems without big-bang rewrites, and operating close to the data with a strong bias toward practical execution.

 

What You’ll Do

• Design, build, and evolve large-scale analytical data infrastructure supporting high-ingest telemetry workloads
• Incrementally migrate existing systems toward a lakehouse / open table format architecture
• Implement scalable streaming ingestion pipelines and object-storage-backed analytics
• Build and operate horizontally scalable compute layers for batch and analytical workloads
• Tune system-level performance by diving into logs, metrics, query plans, and memory usage
• Collaborate with internal teams and external customers to translate product needs into low-level infrastructure requirements
• Evaluate and integrate open-source technologies, balancing build vs. buy decisions
• Contribute directly through hands-on development with a strong bias toward shipping and iteration

 

What We’re Looking For

• Strong experience building or operating distributed data systems, databases, or query engines
• Background supporting high-scale analytics or observability platforms
• Deep interest in performance, efficiency, and system internals
• Comfort working in fast-moving environments with limited headcount and evolving architecture
• Ability to reason about long-term systems design while delivering short-term impact

 

Preferred

• Experience with technologies such as ClickHouse, DuckDB, Spark, Flink, DataFusion, Databricks, or similar
• Familiarity with columnar formats and lakehouse concepts (Parquet, Iceberg, Delta Lake)
• Experience with object storage (S3, MinIO) and large-scale telemetry pipelines
• Background in open-source software (usage or contribution)
• Experience in Rust, C++, Java, or Python
• Academic or professional focus on databases or distributed systems

 

Skills & Tools

• Languages: Rust, C++, Java, Python
• Data & Analytics: ClickHouse, DuckDB, Spark, Flink, DataFusion
• Storage & Formats: Parquet, Iceberg, Delta Lake, object storage
• Systems: Distributed systems, scalable compute, performance optimization
• Concepts: Columnar storage, OLAP, vectorized execution, query optimization

 

This is a high-impact role with strong ownership, competitive senior-level compensation, and the opportunity to shape the long-term data architecture of a fast-scaling platform.