About the role
As an MLOps Engineer at TheWebVale, you will manage the operational lifecycle of AI models and inference engines. You will deploy open-weight language models, embedding services, and prediction pipelines on GPU and CPU cloud instances for business client applications.
You will collaborate closely with AI engineers and backend developers in our regular two-to-three week release intervals. You will ensure models are served with low latency, high availability, auto-scaling, and predictable cloud expenditure.
What you will do
- Deploy and scale machine learning and LLM inference engines using vLLM, Triton, or Ollama.
- Build automated CI/CD deployment pipelines for model weights, embeddings, and prompt configs.
- Configure GPU hardware monitoring, memory allocation, and utilization alerts.
- Implement model versioning, rollback protocols, and blue-green deployment strategies.
- Track inference latencies, token consumption rates, and error rates using Prometheus and Grafana.
- Optimize container images to reduce cold-start latency and cloud server costs.
What you will need
- Hold 3–5 years of software engineering or DevOps experience with dedicated MLOps focus.
- Demonstrate strong proficiency with Docker, Linux system administration, and GPU drivers.
- Show hands-on experience serving models with vLLM, Hugging Face TGI, or FastAPI.
- Understand model quantization formats, tensor parallelism, and batching mechanisms.
- Possess experience building automated deployment workflows in GitHub Actions.
- Communicate infrastructure reliability and capacity plans clearly to engineering leads.
Nice to have
- Experience managing GPU clusters on Oracle Cloud (OCI), AWS, or RunPod.
- Familiarity with Kubernetes, Ray Serve, or KServe for distributed model serving.
- Knowledge of model registry systems like MLflow or Weights & Biases.