Key Responsibilities:
Framework Analysis: Conduct comprehensive evaluations of existing runtime systems and ML runtime frameworks, such as ONNX Runtime, TensorFlow Serving, TorchServe, Apache TVM, VLLM, TRT-LLM, and Glow, to identify their strengths, weaknesses, and applicability to our objectives.
Architecture Design: Develop a robust runtime framework optimized for our servers, ensuring seamless integration with hardware components such as DSPs, video & audio decoders, ARM cores, and DLAs.
Hardware Integration: Collaborate with hardware engineers to adapt and optimize software components, maximizing performance and efficiency across various hardware modules.
Performance Optimization: Implement strategies to enhance system performance, including reducing latency, increasing throughput, and improving resource utilization.
Cross-Functional Collaboration: Work closely with cross-functional teams to align software architecture with business goals and technological advancements.
Requirements:
Educational Background: Bachelor's or Master's degree in Computer Science, Computer Engineering, or a related field.
Experience: Minimum of 5 years in software architecture, with a focus on runtime systems and hardware-software integration.
Technical Proficiency: Extensive experience with runtime systems and ML runtime frameworks, such as ONNX Runtime, TensorFlow Serving, TorchServe, Apache TVM, VLLM, TRT-LLM, and Glow, and a deep understanding of different types of hardware components such as DSPs, hardware accelerators, ARM cores, and DLAs.
Analytical Skills: Strong ability to analyze and compare complex frameworks to inform architectural decisions.
Communication: Excellent verbal and written communication skills, with the ability to convey complex technical concepts to diverse stakeholders.
Preferred Qualifications:
Industry Knowledge: Familiarity with the competitive landscape in runtime frameworks and hardware acceleration technologies.
Project Management: Experience in leading projects from conception through implementation, with a track record of successful delivery.
Continuous Learning: Commitment to staying updated with the latest advancements in runtime systems and hardware integration.
