{"id":2415,"date":"2025-06-20T02:29:52","date_gmt":"2025-06-20T02:29:52","guid":{"rendered":"https:\/\/estreetsecurity.com\/services\/?post_type=jobpost&#038;p=2415"},"modified":"2025-06-20T02:30:23","modified_gmt":"2025-06-20T02:30:23","slug":"principal-software-engineer-model-inference","status":"publish","type":"jobpost","link":"https:\/\/estreetsecurity.com\/services\/jobs\/principal-software-engineer-model-inference\/","title":{"rendered":"Principal Software Engineer &#8211; Model Inference"},"content":{"rendered":"\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Principal Software Engineer &#8211; Model Inference<\/h2>\n\n\n\n<p>Posted: June 19, 2025<\/p>\n\n\n\n<p>Job Type: Permanent<\/p>\n\n\n\n<p>Industry: Computer and Mathematical<\/p>\n\n\n\n<p>Our client, a recognized leader in the technology sector, is actively seeking a highly skilled <strong>Principal Software Engineer<\/strong> to join their dynamic team. As a Principal Software Engineer, you will be an integral part of the Software Engineering department, directly supporting the innovative <strong>OpenShift AI team<\/strong>. The ideal candidate will possess strong communication skills, a deeply collaborative mindset, and an unbridled passion for innovation, ensuring successful alignment within the organization&#8217;s forward-thinking and high-impact environment.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">Location &amp; Compensation:<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Location:<\/strong> Raleigh, NC, or Boston, MA (This is a <strong>hybrid<\/strong> role.)<\/li>\n\n\n\n<li><strong>Salary Range:<\/strong> Competitive<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">What&#8217;s the Job?<\/h3>\n\n\n\n<p>As a Principal Software Engineer focused on Model Inference, you will play a critical role in advancing the capabilities of AI and Machine Learning platforms. Your key responsibilities will include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>High-Performance ML Inference Runtime Development:<\/strong> Leading the design, development, and maintenance of a high-quality, high-performing <strong>ML inference runtime platform<\/strong>. This platform is crucial for enabling multi-modal and distributed model serving at scale.<\/li>\n\n\n\n<li><strong>Open-Source Community Contribution:<\/strong> Directly contributing to significant <strong>upstream inference runtime communities<\/strong>. This includes active participation in projects and libraries such as <strong>vLLM, TGI, PyTorch, OpenVINO<\/strong>, and other relevant open-source initiatives.<\/li>\n\n\n\n<li><strong>CI\/CD Pipeline Maintenance &amp; Optimization:<\/strong> Maintaining and optimizing robust <strong>CI\/CD (Continuous Integration\/Continuous Delivery) build pipelines<\/strong> specifically for container images. This ensures faster, more secure, reliable, and frequent releases of ML inference components.<\/li>\n\n\n\n<li><strong>Stakeholder Coordination &amp; Communication:<\/strong> Effectively coordinating and communicating with various internal and external stakeholders to ensure clear project alignment, transparency, and successful delivery of AI\/ML solutions.<\/li>\n\n\n\n<li><strong>Continuous Learning &amp; AI\/ML Advancement:<\/strong> Applying a strong growth mindset by continuously staying up to date with the latest advancements in the rapidly evolving fields of Artificial Intelligence (AI) and Machine Learning (ML). You&#8217;ll translate these insights into practical applications and improvements.<\/li>\n\n\n\n<li><strong>Problem Solving:<\/strong> Tackling complex technical challenges related to model serving scalability, performance, and efficiency.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">What&#8217;s Needed?<\/h3>\n\n\n\n<p>We&#8217;re looking for a highly experienced and technically profound individual with:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Python &amp; PyTorch Expertise:<\/strong> Extensive hands-on experience with programming in <strong>Python<\/strong> and a deep proficiency in <strong>PyTorch<\/strong>, a foundational framework for deep learning.<\/li>\n\n\n\n<li><strong>Model Optimization Familiarity:<\/strong> Strong familiarity with critical model optimization techniques such as <strong>model parallelization, quantization, and memory optimization<\/strong>. This includes practical experience using relevant libraries like <strong>vLLM, TGI<\/strong>, and other specialized inference libraries.<\/li>\n\n\n\n<li><strong>Python Packaging Experience:<\/strong> Proven experience with <strong>Python packaging<\/strong>, including building and managing <strong>PyPI libraries<\/strong>.<\/li>\n\n\n\n<li><strong>C++ &amp; CUDA (Bonus):<\/strong> Development experience with <strong>C++<\/strong>, especially with the <strong>CUDA APIs<\/strong>, is considered a significant advantage, demonstrating capability in high-performance computing for AI.<\/li>\n\n\n\n<li><strong>Model Inferencing Architectures:<\/strong> A solid understanding of the fundamental principles and architectural patterns behind efficient model inferencing, from single-node to distributed deployments.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">What&#8217;s in it for Me?<\/h3>\n\n\n\n<p>This role offers compelling opportunities for significant professional growth and impact:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Cutting-Edge AI\/ML:<\/strong> The unique opportunity to work daily on cutting-edge <strong>AI and machine learning technologies<\/strong>, pushing the boundaries of what&#8217;s possible in the field.<\/li>\n\n\n\n<li><strong>Collaborative Environment:<\/strong> Join a highly collaborative and inclusive work environment that fosters teamwork, innovation, and mutual support among talented engineers.<\/li>\n\n\n\n<li><strong>Open-Source Contribution:<\/strong> A direct chance to contribute to significant open-source development communities, making a broader impact on the AI\/ML ecosystem.<\/li>\n\n\n\n<li><strong>Diverse Team Engagement:<\/strong> Engage with a diverse team of exceptionally talented engineers, fostering knowledge exchange and continuous learning.<\/li>\n\n\n\n<li><strong>Professional Growth:<\/strong> Access to excellent professional growth and development opportunities, supporting your continuous learning journey and career advancement.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p>If this challenging and rewarding permanent role interests you and you&#8217;d like to learn more, click &#8220;apply now,&#8221; and a recruiter will be in touch to discuss this great opportunity. We look forward to speaking with you!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Principal Software Engineer &#8211; Model Inference Posted: June 19, 2025 Job Type: Permanent Industry: Computer and Mathematical Our client, a recognized leader in the technology sector, is actively seeking a highly skilled Principal Software Engineer to join their dynamic team. As a Principal Software Engineer, you will be an integral part of the Software Engineering [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"menu_order":0,"template":"","jobpost_category":[406,407,605],"jobpost_job_type":[348],"jobpost_location":[1133,1101],"jobpost_tag":[690,1150,1149,1147,249,985,1148,1145,688,832,691,1142,1135,1136,1140,1138,1141,379,1134,1146,245,1143,700,1144,1122,1065,1006,1139,1137],"class_list":["post-2415","jobpost","type-jobpost","status-publish","hentry","jobpost_category-ai","jobpost_category-artificial-intelligence","jobpost_category-engineering","jobpost_job_type-hybrid","jobpost_location-boston-ma","jobpost_location-raleigh-nc","jobpost_tag-ai","jobpost_tag-ai-ml","jobpost_tag-boston-ma","jobpost_tag-c-2","jobpost_tag-ci-cd","jobpost_tag-computer-and-mathematical","jobpost_tag-container-images","jobpost_tag-cuda-apis","jobpost_tag-deep-learning","jobpost_tag-hybrid","jobpost_tag-machine-learning","jobpost_tag-memory-optimization","jobpost_tag-ml-inference","jobpost_tag-model-inference","jobpost_tag-model-parallelization","jobpost_tag-openshift-ai","jobpost_tag-openvino","jobpost_tag-permanent","jobpost_tag-principal-software-engineer","jobpost_tag-pypi","jobpost_tag-python","jobpost_tag-python-packaging","jobpost_tag-pytorch","jobpost_tag-quantization","jobpost_tag-raleigh-nc","jobpost_tag-software-engineer","jobpost_tag-software-engineering","jobpost_tag-tgi","jobpost_tag-vllm"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/estreetsecurity.com\/services\/wp-json\/wp\/v2\/jobpost\/2415","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/estreetsecurity.com\/services\/wp-json\/wp\/v2\/jobpost"}],"about":[{"href":"https:\/\/estreetsecurity.com\/services\/wp-json\/wp\/v2\/types\/jobpost"}],"author":[{"embeddable":true,"href":"https:\/\/estreetsecurity.com\/services\/wp-json\/wp\/v2\/users\/1"}],"wp:attachment":[{"href":"https:\/\/estreetsecurity.com\/services\/wp-json\/wp\/v2\/media?parent=2415"}],"wp:term":[{"taxonomy":"jobpost_category","embeddable":true,"href":"https:\/\/estreetsecurity.com\/services\/wp-json\/wp\/v2\/jobpost_category?post=2415"},{"taxonomy":"jobpost_job_type","embeddable":true,"href":"https:\/\/estreetsecurity.com\/services\/wp-json\/wp\/v2\/jobpost_job_type?post=2415"},{"taxonomy":"jobpost_location","embeddable":true,"href":"https:\/\/estreetsecurity.com\/services\/wp-json\/wp\/v2\/jobpost_location?post=2415"},{"taxonomy":"jobpost_tag","embeddable":true,"href":"https:\/\/estreetsecurity.com\/services\/wp-json\/wp\/v2\/jobpost_tag?post=2415"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}