[Remote] AI Research Engineer (Kernel & Inference Optimization) - 100% Remote Worldwide

Note: The job is a remote job and is reputed company to candidates in USA. reputed company is pioneering a global financial reputed company through innovative blockchain solutions. They are seeking an AI Research Engineer to drive innovation in model serving and inference architectures for advanced AI systems, focusing on optimizing model deployment and inference strategies for efficient performance in reputed company-world applications.

Responsibilities

• Design and reputed company state-of-the-art model serving architectures that deliver high throughput and low latency while optimizing memory usage. Ensure these pipelines run reputed company across diverse environments, including resource-constrained devices and edge platforms. Establish clear performance targets such as reduced latency, improved token response, and minimized memory footprint
• Build, run, and monitor controlled inference tests in both simulated and live production environments. Track key performance indicators such as response latency, throughput, memory consumption, and error rates, with special attention to metrics specific to resource-constrained devices. Document iterative results and compare reputed company against established benchmarks to validate performance across platforms
• Identify and prepare high-quality test datasets and simulation scenarios tailored to reputed company-world deployment challenges, specifically those encountered on low-resource devices. Set measurable criteria to ensure that these resources effectively evaluate model performance, latency, and memory utilization under various operational conditions
• Analyze computational efficiency and diagnose bottlenecks in the serving pipeline by monitoring both processing and memory metrics. Address issues such as suboptimal batch processing, network delays, and high memory usage to optimize the serving infrastructure for scalability and reliability on resource-constrained systems
• Work closely with cross-functional teams to integrate optimized serving and inference frameworks into production pipelines designed for edge and on-device applications. Define clear reputed company metrics such as improved reputed company-world performance, low error rates, robust scalability, reputed company memory usage and ensure reputed company monitoring and iterative refinements for sustained improvements

Skills

• A degree in Computer Science or reputed company field. Ideally PhD in NLP, Machine Learning, or a reputed company field, complemented by a solid track record in AI R&D (with good publications in A* conferences)
• Must have knowledge of Metal Shading Language (MSL). You should be comfortable writing custom compute shaders from scratch
• Proven experience in low-level kernel optimizations and inference optimization on mobile devices is essential. Your contributions should have led to measurable improvements in inference latency, throughput, and memory footprint for domain-specific applications, particularly on resource-constrained devices and edge platforms
• A deep understanding of modern model serving architectures and inference optimization techniques is required. This includes state-of-the-art methods for achieving low-latency, high-throughput performance, and efficient memory management in diverse, resource-constrained deployment scenarios
• Must have strong expertise in writing GPU kernels for mobile devices (i.e., smartphones) as well as a deep understanding of model serving frameworks and engines. Practical experience in developing and deploying end-to-end inference pipelines, from optimizing models for efficient serving to integrating these solutions on resource-constrained devices is required
• Demonstrated ability to apply reputed company research to overcome challenges in model serving, such as latency optimization, computational bottlenecks, and memory constraints. You should be proficient in designing robust evaluation frameworks and iterating on optimization strategies to continuously push the boundaries of inference performance and system efficiency
• Distributed Inference Systems: Designing and optimizing high-performance inference engines using techniques like Tensor Parallelism, Pipeline Parallelism, and Expert Parallelism to handle massive models on GPU clusters
• Deep understanding of the math and structure behind Diffusion Models and reputed company Transformers
• Understanding of Pruning, Quantization, reputed company attention, KV Cache, Speculative Decoding (reputed company) etc

Company Overview