LanceDB Logo

LanceDB

Senior Software Engineer

Posted 17 Days Ago
Remote
Hiring Remotely in USA
180K-250K Annually
Senior level
Remote
Hiring Remotely in USA
180K-250K Annually
Senior level
Design and maintain distributed Lance dataset operations, develop high-performance indices and data formats, and improve query performance across Spark, Ray, and Trino. Contribute Rust-based table formats and encodings, integrate Lance with major open-source data infrastructure systems, operate internal data processing infrastructure, and promote the project through open-source communities and conferences.
The summary above was generated by AI

About LanceDB

AI advances at the speed of its research, and research moves at the speed of its data. LanceDB is the AI-native Multimodal Lakehouse: one system where a researcher curates petabytes of video, audio, and every signal derived from them with a few lines of Python, and the next training run starts as fast as the next idea. Customers like Runway, Midjourney, and Netflix build the future of AI on LanceDB, from frontier and world models to robots and autonomous vehicles.

About the Role

We're looking for a Senior Software Engineer to help expand the reach of Lance and LanceDB within the broader data infrastructure ecosystem. You'll work at the intersection of high-performance computing, big data, and open-source systems. You will contribute scale and performance improvements, integrations with the wider data and AI ecosystem, simplifying distributed operations, and usability and maintainability enhancements.

What You'll Do

  • Designing and maintaining efficient distributed Lance dataset operations

  • Building efficient indices to enable predicate pushdown and accelerate queries in Spark, Ray, or Trino

  • Working on table formats, data encodings, and various aspects of the Lance format in Rust

  • Driving open-source community efforts to integrate the Lance format with Spark, Hive Metastore, Presto, Trino, Ray, and other data infrastructure systems

  • Operating and improving internal data processing infrastructure

  • Promoting the Lance format in open-source communities and at Big Data conferences

What We're Looking For

  • 10+ years of experience building high-performance databases, big data systems, or large-scale data services

  • Deep understanding of internals of open-source Big Data or AI training systems (e.g., Hadoop, Spark, Flink, Ray, Iceberg, Delta Lake, Hudi, ClickHouse, Trino, Presto, PyTorch, or JAX)

  • Strong experience with high-performance computing in C++, Java, and/or Scala

  • Experience with Rust (or willingness to learn it)

  • Proven ability to move fast, work independently, and collaborate with a high-caliber team

Nice to Have

  • Contributor, committer, or PMC member in Apache or other large open-source projects

  • Experience with Apache Arrow, DataFusion, Parquet, Iceberg, or Delta Lake

  • Track record of driving large features or integrations in distributed systems

  • Strong community presence and passion for open-source collaboration

Similar Jobs

2 Days Ago
Remote
60K-114K Annually
Senior level
60K-114K Annually
Senior level
Payments
Build and operate security platform services in Go and Python on AWS and Kubernetes. Own the NATS event bus, security scanning gates, CI/CD integrations, finding workflows, triage automation, and remediation routing. Develop AI-assisted security workflows, improve scanner accuracy and reliability, participate in threat modeling and secure design reviews, and support on-call operations for owned services.
Top Skills: AWSClaude CodeCodeqlCodexConftestCursorElasticsearchGitGitlab CiGitleaksGoGrypeKubernetesKyvernoMySQLNatsOpaOwasp Top 10PostgresPythonReactReact NativeSastSbomScaSemgrepSigstoreSlsaSnykTrivyTypescript
2 Days Ago
Remote
60K-114K Annually
Senior level
60K-114K Annually
Senior level
Payments
Build and operate security platform services in Go and Python on AWS and Kubernetes. Own the NATS event bus, security scanning gates, CI/CD integrations, finding workflows, agentic triage automation, and scanner tuning. Integrate security findings with tickets and metrics, support threat modeling and secure design reviews, and participate in on-call. The role emphasizes reliable, low-noise security enforcement and developer adoption.
Top Skills: AWSClaude CodeCodeqlCodexConftestCursorElasticsearchGitGitlab CiGitleaksGoGrypeKubernetesKyvernoMySQLNatsOpaPostgresPythonReactReact NativeSastSbomScaSemgrepSigstoreSlsaSnykTrivyTypescript
5 Days Ago
Remote
USA
Senior level
Senior level
Fintech • Information Technology
Build and evolve Alpaca’s developer and agent platform, including APIs, SDKs, CLI tools, MCPs, backend systems, governance, documentation, and security standards. Lead architectural decisions, improve observability, reliability, and performance, contribute to shared tooling, review designs, and mentor engineers. Collaborate with Product, Compliance, Risk, and Platform Engineering teams while supporting developer-facing trading infrastructure and API experiences.
Top Skills: C#GoJavaMcpOauthReactTypescript

What you need to know about the Charlotte Tech Scene

Ranked among the hottest tech cities in 2024 by CompTIA, Charlotte is quickly cementing its place as a major U.S. tech hub. Home to more than 90,000 tech workers, the city’s ecosystem is primed for continued growth, fueled by billions in annual funding from heavyweights like Microsoft and RevTech Labs, which has created thousands of fintech jobs and made the city a go-to for tech pros looking for their next big opportunity.

Key Facts About Charlotte Tech

  • Number of Tech Workers: 90,859; 6.5% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Lowe’s, Bank of America, TIAA, Microsoft, Honeywell
  • Key Industries: Fintech, artificial intelligence, cybersecurity, cloud computing, e-commerce
  • Funding Landscape: $3.1 billion in venture capital funding in 2024 (CED)
  • Notable Investors: Microsoft, Google, Falfurrias Management Partners, RevTech Labs Foundation
  • Research Centers and Universities: University of North Carolina at Charlotte, Northeastern University, North Carolina Research Campus

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account