Model Engineer, Multimodal Understanding

About the role

In this role, you will design and optimize large-scale distributed systems that process multimodal data, including images, audio, and text. You will work on cutting-edge technologies that drive the development of advanced AI capabilities, such as real-time video processing and spatial-temporal compression.

You will be part of a team that values innovation and rapid iteration, collaborating across the stack to improve model performance and user experiences. Success in this position requires a proactive approach and a willingness to tackle complex challenges in a high-intensity environment.

The salary range for this position is $160,000 to $250,000, which is part of a comprehensive compensation package.

Skills that matter here

• Python: This role requires writing production-quality code primarily in Python.
• PyTorch: Experience with PyTorch is essential for building and optimizing ML systems.
• JAX: Proficiency in JAX is preferred for advanced modeling and optimization tasks.
• Rust: Rust is needed for performance-critical components of the systems.
• C++: C++ expertise is valuable for high-performance applications.
• Spark: Hands-on experience with Spark is important for managing large-scale data orchestration.

Who this role suits

• A proactive self-starter who thrives in fast-paced environments.
• Someone with a strong technical background in distributed ML systems.
• A candidate passionate about multimodal AI and user experience.
• An individual comfortable owning end-to-end initiatives and driving improvements.

From the employer

Responsibilities:

• Design, build, and optimize large-scale distributed systems for multimodal pre-training, post-training, inference, data processing, and tokenization at web/petabyte scale.
• Develop high-throughput pipelines for data acquisition, preprocessing, filtering, generation, decoding, loading, crawling, visualization, and management (images, videos, audio + text).
• Advance multimodal capabilities including spatial-temporal compression, cross-modal alignment, world modeling, reasoning, emergent abilities, audio/image/video understanding & generation, real-time video processing, and noisy data handling.
• Drive data quality and studies: curation (human/synthetic), filtering techniques, analysis, and scalable pipelines to support trillion-parameter models.
• Create evaluation frameworks, internal benchmarks, reward models, and metrics that capture real-world usage, failure modes, interactive dynamics, and human-AI synergy.
• Innovate on algorithms, modeling approaches, hardware/software/algorithm co-design, and scaling paradigms for state-of-the-art performance.
• Build research tooling, user-friendly interfaces, prototypes/demos, full-stack applications, and enable rapid iteration based on feedback.
• Work across the stack (pre-training → SFT/RL/post-training) to enable reasoning, tool calling, agentic behaviors, orchestration, and seamless real-time interactions.

Basic Qualifications:

• Experience writing production-quality code in Python, PyTorch, JAX, Rust, or C++.

Preferred Skills and Experience:

• Expert-level proficiency in Python (core language), with strong experience in at least one of: JAX / PyTorch / XLA.
• Proven track record building or optimizing large-scale distributed ML systems (training/inference optimization, GPU utilization, multi-GPU/TPU setups, hardware co-design).
• Deep experience designing and running data pipelines at scale: curation, filtering, generation, quality studies, especially for noisy/real-world multimodal data.
• Strong fundamentals in evaluation design, benchmarks, reward modeling, or RL techniques (particularly for interactive/agentic behaviors).
• Proactive self-starter who thrives in high-intensity environments and is passionate about pushing multimodal AI frontiers.
• Willingness to own end-to-end initiatives and do whatever it takes to deliver breakthrough user experiences.
• Experience leading major improvements in model capabilities through better data, modeling, algorithms, or scaling.
• Familiarity with state-of-the-art in multimodal LLMs, scaling laws, tokenizers, compression techniques, reasoning, or agentic systems.
• Proficiency in Rust and/or C++ for performance-critical components.
• Hands-on work with large-scale orchestration tools such as Spark, Ray, or Kubernetes.
• Background building full-stack tooling: performant interfaces, real-time research demos/apps, or end-to-end product ownership.
• Passion for end-to-end user experience in interactive, real-time multimodal AI systems.

Compensation and Benefits:

• $160,000 - $250,000 USD.
• Base salary is just one part of our total rewards package at SpaceXSI, which also includes equity, comprehensive medical, vision, and dental coverage, access to a 401(k) retirement plan, short & long-term disability insurance, life insurance, and various other discounts and perks.
• SpaceXSI is an equal opportunity employer.