About the role
As an LLM Application Engineer at TheWebVale, you will build applications powered by large language models for client teams. You will develop AI customer support assistants, contract summarization portals, and intelligent workflow assistants for legal firms, coaching businesses, and software teams.
You will work in close coordination with frontend developers and backend architects throughout our two-to-three week delivery cycles. You will ensure models adhere strictly to business schemas, handle edge cases gracefully, and stream responses with low latency.
What you will do
- Develop production-grade software applications driven by commercial and open-source LLMs.
- Implement function-calling tools that allow models to query client databases and APIs safely.
- Build streaming response handlers with Server-Sent Events for fluid chat interfaces.
- Design Pydantic and Zod validation schemas to enforce strict JSON structure from models.
- Implement token budgeting and conversation truncation algorithms for long chat threads.
- Write comprehensive automated tests to evaluate model reliability across diverse inputs.
What you will need
- Possess 2–4 years of software development experience with focused LLM application work.
- Demonstrate advanced proficiency in Python or TypeScript with asynchronous programming.
- Show extensive hands-on experience with OpenAI, Anthropic, or Gemini API tool-calling.
- Understand prompt formatting, system instructions, few-shot examples, and temperature tuning.
- Exhibit disciplined Git habits, unit testing practices, and clean code documentation.
- Communicate technical constraints and mitigation strategies clearly to client managers.
Nice to have
- Experience with LlamaIndex or LangChain for document orchestration.
- Familiarity with frontend chat interfaces built with Vercel AI SDK.
- Knowledge of local model execution using Ollama or llama.cpp.