Senior Machine Learning Engineer, LLMs & Prompt Engineering (Permanent – Remote – Japan)

By estreetsecurity
July 26, 2025July 26, 2025
Comments are off

Remote

Tokyo, Japan

Posted 23 hours ago

An opportunity has come through our network for a Senior Machine Learning Engineer, LLMs & Prompt Engineering at Autify, a global software company specializing in AI-powered solutions that simplify identity verification and data extraction. This permanent position is fully remote within Japan, offering exceptional flexibility for talent residing anywhere in the country. The compensation ranges from ¥8M ~ ¥12M annually, and importantly, no Japanese language proficiency is required. Overseas applicants are welcome, and relocation support to Japan is available.

You will mainly be responsible for developing a new product that leverages cutting-edge machine learning techniques and generative AI, guiding it from a proof of concept into a releasable product. Autify is known for its AI-powered software testing platforms like Autify Nexus and Autify Genesis (which generates test cases and test scripts from product documents), focusing on improving development productivity through test automation and efficiency. This role is a unique chance to contribute to the future of AI in software quality assurance.

About the Position: Shaping AI-Powered Solutions for Software Testing

As a Senior Machine Learning Engineer focusing on LLMs and Prompt Engineering, you’ll be at the forefront of developing innovative AI-powered solutions for the software testing industry. Your expertise will bridge the gap between advanced AI research and practical, impactful products.

Develop New Products Utilizing ML and Generative AI: You will be primarily responsible for developing a new product that meticulously utilizes machine learning techniques and generative AI. This involves taking a product idea from a nascent proof of concept (PoC) phase all the way through to a fully releasable product. Your work will directly shape the future offerings of Autify, leveraging cutting-edge AI for software quality assurance.
Create Effective Prompts for LLM Output Optimization: A crucial responsibility will be to create highly effective prompts to maximize the quality of the output from Large Language Models (LLMs). This requires a deep understanding of prompt engineering principles, iteratively refining prompts based on product specifications to achieve desired results, and ensuring the LLM’s responses are accurate, relevant, and useful for the software testing domain.
Build and Customize Machine Learning Models: You will actively build and customize machine learning models using data relevant to software testing, as needed for specific product functionalities. This involves data collection, preprocessing, model selection, training, and fine-tuning to ensure models perform optimally for tasks such as test case generation, bug detection, or anomaly identification.
Collaborate on Feature Implementation: You will work in close collaboration with other engineers, designers, and product leads to seamlessly implement various features within the new AI product. Throughout this process, you will consistently consider critical aspects such as performance (ensuring speed and efficiency), security (protecting data and systems), and reliability (guaranteeing continuous operation and accurate output). Your teamwork ensures holistic product development.
Create PoC Web/CLI Applications: You will proactively create Proof of Concept (PoC) Web/CLI (Command Line Interface) applications that utilize LLMs or ML models “under the hood” to validate the feasibility of new ideas. These rapid prototypes are essential for quickly testing concepts, gathering early feedback, and demonstrating the potential of AI-driven solutions before full-scale development.
Proactively Identify and Suggest Improvements: You will demonstrate a continuous improvement mindset by proactively identifying functions that can be improved within existing or new products. You will then strategically suggest possible solutions from both technical aspects (e.g., code optimization, new algorithms) and product-related ideas (e.g., new features, enhanced user experience), driving innovation from within.
Participate in On-Call Rotations: You will actively participate in on-call rotations for incident and alert handling. This commitment ensures that critical issues impacting the AI platform or its services are addressed promptly, maintaining system reliability and minimizing downtime for users.

What You Bring: Essential Requirements for AI/ML and Prompt Engineering

To excel as a Senior Machine Learning Engineer focusing on LLMs and Prompt Engineering, you’ll need extensive experience in deep learning, hands-on prompting skills, and proficiency in relevant development frameworks.

Extensive Deep Learning and LLM Experience (2+ years): You must possess over two years of verifiable experience in training and customizing deep learning models, with a specific emphasis on LLMs (Large Language Models). This indicates practical expertise in fine-tuning, adapting, and optimizing large pre-trained models for specific tasks.
Prompting LLMs Experience (1+ year): You have over a year of practical experience in prompting both open-source and closed-source LLMs. This highlights your specialized skill in crafting effective inputs to guide LLM behavior, extract desired outputs, and mitigate potential biases or inaccuracies.
API Development with FastAPI and Python: You have proven experience in developing APIs with FastAPI and Python. This indicates proficiency in building high-performance web APIs using a modern Python framework, crucial for exposing ML model functionalities and integrating with other systems.
Up-to-Date with Latest ML Research: You are up-to-date with the latest Machine Learning Research, especially concerning LLMs and Prompt Engineering. This demonstrates a commitment to continuous learning and staying at the forefront of the rapidly evolving AI field.
Experience in Building Metrics for LLM Evaluation: You have practical experience in building metrics for evaluating various tasks performed by LLMs. This includes defining quantitative measures of model performance, accuracy, relevance, and safety, ensuring that the quality of AI outputs can be objectively assessed and improved.

Nice to Haves: Enhancing Your AI/ML Profile

While not specifically required, the following skills and experiences would be highly beneficial:

Knowledge of NLP and Visual Recognition: Familiarity with natural language processing (NLP) techniques beyond LLMs, and visual recognition (computer vision) knowledge, would be a plus. This indicates a broader understanding of AI domains relevant to software testing.
Proficiency in Analyzing, Hypothesizing, and Testing: You are proficient in the process of analyzing, hypothesizing, and rigorously testing. This scientific approach to problem-solving ensures that solutions are data-driven and validated through experimentation.
Experience in MLOps: Hands-on experience in MLOps (Machine Learning Operations) is desirable. This indicates familiarity with the practices and tools for deploying, monitoring, and managing ML models in production environments.
Modern Development Experience (Agile/Scrum): Experience with modern development methodologies such as Agile or Scrum is a plus, indicating your ability to work collaboratively in iterative development cycles.
E2E Test Automation Experience: Experience in E2E (End-to-End) test automation with tools like Selenium and Appium would be beneficial, as Autify specializes in test automation.