LLM Engineers Job Description Template (Copy-Paste Ready)

Post a JD titled “LLM Engineer” and describe RAG pipeline work in the body, and you will get applications from two different candidate pools stacked on top of each other: engineers who fine-tune open-weight models for a living, and engineers who wire LangChain calls into a product. Neither pool reads the JD the way you meant it, and half your first-round interviews end in fifteen minutes because the candidate built the wrong thing.

The confusion is understandable. “AI engineer” and “LLM engineer” both showed up in job postings within the same eighteen-month window, and most hiring managers never got a clean explanation of where one role ends and the other begins. This guide draws that line explicitly, then hands you a copy-paste JD template built for the model-layer work: fine-tuning, quantization, inference serving, and evaluation, not application-layer integration.

LLM Engineer vs. AI Engineer: Get This Distinction Right First

Before you write a single bullet point, decide which role you actually need. This is the single highest-leverage decision in the entire hiring process, and it is the one most JDs get wrong.

An AI Engineer works at the application layer. They build RAG pipelines, wire LLM APIs into products, manage agentic workflows with LangChain or LlamaIndex, and optimize how an existing model gets called. They rarely touch model weights directly. If your problem is “our product needs to answer questions from our knowledge base,” you want an AI Engineer, and our AI Engineer JD template is the right starting point.

An LLM Engineer works at the model layer. They fine-tune open-weight models with LoRA or QLoRA, quantize models down to a servable footprint with GGUF, AWQ, or GPTQ, stand up and tune inference servers like vLLM or TensorRT-LLM, and build the evaluation harnesses that prove a fine-tuned or quantized model still performs after the changes. If your problem is “the API-hosted models are too expensive at our volume, too slow at our latency target, or don’t know our domain well enough after prompting alone,” you need an LLM Engineer.

The two roles share a vocabulary and almost nothing else in day-to-day work. A candidate who has spent two years calling OpenAI’s API through LangChain will not have a working answer for “walk me through your last QLoRA fine-tune, including rank and alpha selection.” A candidate who has spent two years running vLLM in production will not have a strong opinion on chunking strategy for a vector store, because it was never their job to have one. Name the correct role in the JD title, and say so explicitly in the first paragraph, or you will spend weeks screening the wrong applicants.

What to Include in an LLM Engineer Job Description

Once the role is correctly scoped, an effective LLM engineer JD needs seven components. Skipping any of them is how you end up with a stack of resumes from candidates who fine-tuned a model once in a class project and called it production experience.

Role Summary

State in three or four sentences: which base model family the engineer will work with (Llama, Mistral, Qwen, or a proprietary base), whether the primary mandate is fine-tuning, inference optimization, or both, and what business constraint is driving the hire — cost per token, latency SLA, data privacy, or domain accuracy that prompting alone couldn’t reach. Skip phrases like “work with cutting-edge language models.” An engineer who has actually shipped a quantized model to a GPU cluster will read that phrase as a sign nobody scoped the role.

Reporting Structure and Team Context

Specify who owns the GPU infrastructure decisions, whether this person reports into an ML platform team or directly to engineering leadership, and how the role interacts with data engineering (who owns the fine-tuning dataset) and with any AI/application engineers already on staff. LLM engineering work is infrastructure-adjacent and often solitary at smaller companies — a candidate deciding between two offers will weigh whether they’re the only person who understands the serving stack against whether there’s a real platform team forming around them.

Core Responsibilities

Write responsibilities that map to actual model-layer work, not integration work with different words. Below is a set calibrated for a production LLM engineering role centered on fine-tuning and inference. Adjust to your actual stack and constraints.

• Fine-tune open-weight models (Llama, Mistral, Qwen, or similar) using LoRA or QLoRA on domain-specific datasets, including hyperparameter selection, rank/alpha tuning, and catastrophic forgetting mitigation
• Quantize trained and fine-tuned models to GGUF, AWQ, or GPTQ formats, benchmarking accuracy degradation against latency and memory footprint gains at each quantization level
• Deploy and tune inference servers (vLLM, TGI, or TensorRT-LLM), including continuous batching, KV-cache management, and multi-GPU tensor parallelism configuration
• Build and maintain evaluation harnesses (promptfoo, RAGAS, LangSmith, or Arize Phoenix) to measure model quality before and after fine-tuning or quantization changes ship to production
• Own cost and latency optimization across the serving stack, including batch size tuning, GPU utilization monitoring, and model routing between quantization tiers based on request complexity
• Implement safety and guardrail testing, including jailbreak red-teaming, PII redaction at the inference layer, and refusal-rate regression testing after each fine-tune
• Document model cards, training data provenance, and evaluation results for every fine-tuned checkpoint shipped to production

Required Skills

Required skills for an LLM engineering role should reflect genuine model-layer depth, not general Python familiarity plus a LangChain tutorial. Screen against every item below, and be willing to reject a candidate on a single missing one if the role truly depends on it.

• PyTorch and Hugging Face fluency: comfortable reading and modifying training scripts, not just calling .from_pretrained(); understands attention mechanisms well enough to debug a training run that isn’t converging
• PEFT and fine-tuning experience: has run LoRA or QLoRA fine-tunes end to end, including dataset preparation, rank selection, and evaluation of the resulting checkpoint against a held-out set
• Quantization experience: has quantized at least one model to GGUF, AWQ, or GPTQ and can speak to the accuracy/latency tradeoff at different bit widths from direct experience, not documentation
• Inference serving experience: has deployed a model behind vLLM, TGI, or TensorRT-LLM in a non-toy environment, including GPU memory management and throughput tuning
• Evaluation tooling: has built or operated an evaluation harness (promptfoo, RAGAS, LangSmith, or Arize Phoenix) to catch regressions before they reach production, not manual spot-checking
• Linux and GPU infrastructure basics: comfortable with CUDA driver issues, multi-GPU setups, and container-based deployment (Docker, and typically Kubernetes at scale)
• Git and CI/CD: standard software engineering discipline, including reproducible training runs and versioned model artifacts

Nice-to-Have Skills

Nice-to-have skills should be real differentiators, not a copy of the required list with softer language. An honest nice-to-have section for an LLM engineering role looks something like this:

• Experience with distributed training frameworks (DeepSpeed, FSDP) for fine-tuning runs that exceed single-GPU memory capacity
• Reinforcement learning from human feedback (RLHF) or direct preference optimization (DPO) experience beyond supervised fine-tuning
• Speculative decoding or other inference-acceleration technique implementation experience
• Published or internally documented benchmarking work comparing open-weight model performance against closed-API alternatives for a specific domain task
• Experience building or tuning jailbreak/red-team test suites specifically for fine-tuned or quantized model variants
• Familiarity with model routing architectures that mix quantization tiers by request complexity to control cost
• Contributions to open-source inference or fine-tuning tooling (vLLM, Axolotl, Unsloth, or similar)

Sample Job Description

Below is a complete copy-paste LLM engineer job description template. Replace the bracketed placeholders with your specifics, and keep the title as “LLM Engineer” rather than “AI Engineer” if the actual work is model-layer.

LLM Engineer
[Company Name] | [Location or Remote] | [Full-Time]

About the Role

[Company Name] runs [brief product or infrastructure description]. Our models team owns fine-tuning, quantization, and inference serving for [describe the model use case: a domain-specific assistant, an internal search system, a customer-facing feature, etc.]. We are hiring an LLM engineer to [describe the primary problem: reduce inference cost, fine-tune on proprietary data, stand up a self-hosted serving stack, etc.] using open-weight models.

This is a model-layer role, distinct from our application/AI engineering team, which owns RAG and product integration. You will report to [ML Platform Lead / Engineering Manager / CTO] and work closely with [data engineering, infrastructure/DevOps, or similar].

What You Will Work On

- Fine-tune [Llama / Mistral / Qwen] models using LoRA or QLoRA on [describe dataset: support transcripts, domain documents, proprietary data, etc.]
- Quantize fine-tuned checkpoints to [GGUF / AWQ / GPTQ] and benchmark accuracy against latency and memory gains at each quantization level
- Deploy and tune [vLLM / TGI / TensorRT-LLM] for production inference, including batching and GPU utilization tuning
- Build evaluation pipelines using [promptfoo / RAGAS / LangSmith / Arize Phoenix] to catch quality regressions before deployment
- Implement jailbreak testing and PII redaction at the inference layer for every model version shipped
- [Add any additional responsibilities specific to your use case]

Required Qualifications

- Strong PyTorch and Hugging Face fluency, including reading and modifying training code
- Hands-on LoRA or QLoRA fine-tuning experience with a documented evaluation of the resulting model
- Has quantized at least one model to GGUF, AWQ, or GPTQ format for production or near-production use
- Production or near-production experience with vLLM, TGI, or TensorRT-LLM
- Has built or operated an LLM evaluation harness beyond manual spot-checking
- Comfortable with multi-GPU infrastructure and container-based deployment

Preferred Qualifications

- DeepSpeed or FSDP experience for distributed fine-tuning runs
- RLHF or DPO experience beyond supervised fine-tuning
- Speculative decoding or other inference-acceleration implementation experience
- [Cloud/infra specifics]: on-prem GPU cluster, AWS, or equivalent

Compensation

- [Salary range] + [equity if applicable] + [benefits summary]
- [Remote, hybrid, or on-site]: [location and timezone requirements]

[Company Name] is an equal opportunity employer. [Any additional legal boilerplate.]

Salary and Compensation

Compensation ranges belong in the JD, both because salary range disclosure is now legally required at posting in most US states and because publishing the range filters out mismatched expectations before either side spends time in the process.

For US-market LLM engineers, mid-level roles (3 to 5 years, fine-tuning and quantization experience) are ranging $150,000 to $185,000 in total compensation. Senior engineers (5 to 8 years, inference serving depth and distributed training experience) range $185,000 to $245,000. Staff-level engineers with model architecture and infrastructure ownership sit at $245,000 to $300,000 or above at well-capitalized companies — the model-layer specialization commands a premium over generalist AI engineering roles because the candidate pool is thinner.

If you are hiring nearshore LLM engineers through a staffing partner, comparable ranges run $75,000 to $95,000 for mid-level, $95,000 to $130,000 for senior, and $130,000 to $160,000 for staff-level, all-in through the arrangement. That represents roughly 40 to 55% savings against US equivalents while applying the same fine-tuning and inference-serving screening bar. See the full range breakdown at Nearshore LLM Engineers.

One caution on equity: if it’s meaningful, put it in the JD, because candidates at this level evaluate total compensation and equity that isn’t mentioned doesn’t factor into their decision. If your equity is RSUs at a company with no liquidity path, don’t dress it up as a benefit — experienced candidates will ask about the cap table anyway.

Frequently Asked Questions