Pelham Berkeley Search
https://www.pbsearch.com/wp-content/uploads/2021/10/JB-Square-200x200-1.png
http://www.pbsearch.com
http://www.pbsearch.com
USD
175000
210000
YEAR
true
Job Title: Senior AI Platform Engineer (Kubernetes & LLMOps)
Location: New York, NY
The Mandate We are building the enterprise-grade backbone that powers Generative AI across the firm. We need a Senior Platform Engineer to architect the internal "AI Factory"—the self-service platform that enables our quant, product, and tech teams to deploy GenAI agents and LLMs into production.
This is not a data science role. It is a hard-core engineering role focused on infrastructure, scalability, and developer experience within a highly regulated financial environment. You will bridge the gap between raw Kubernetes infrastructure and high-level AI application logic.
The Tech Stack
What You Will Build
The Candidate Profile
Why This Role? You won't just be building chatbots; you will be defining the engineering standards for how a major financial institution adopts AI for the next decade.
Senior AI Platform Engineer (Kubernetes & LLMOps)
1585 Broadway New York, NY 10036 US
2026-01-22
2026-03-12
Employment Type:
Direct Hire
Industry: Artificial Intelligence (AI) / Machine Learning
Job Number: 3108
Salary Level: 175000-210000
Job Description
Job Title: Senior AI Platform Engineer (Kubernetes & LLMOps)
Location: New York, NY
The Mandate We are building the enterprise-grade backbone that powers Generative AI across the firm. We need a Senior Platform Engineer to architect the internal "AI Factory"—the self-service platform that enables our quant, product, and tech teams to deploy GenAI agents and LLMs into production.
This is not a data science role. It is a hard-core engineering role focused on infrastructure, scalability, and developer experience within a highly regulated financial environment. You will bridge the gap between raw Kubernetes infrastructure and high-level AI application logic.
The Tech Stack
-
Core: Python (FastAPI/Flask), Go.
-
Infra: Kubernetes (OpenShift), Helm, Kustomize, Docker.
-
AI/Data: LangChain, LangGraph, Vector Databases, Kafka, Redis.
-
Ops: GitOps (ArgoCD), Jenkins, Prometheus, Grafana, OpenTelemetry.
What You Will Build
-
The AI Platform: Architect reusable, containerized blueprints for deploying GenAI solutions. You will define how the firm deploys Agents, RAG pipelines, and fine-tuned models on OpenShift.
-
Agentic Infrastructure: Design the orchestration layer for multi-agent workflows (using frameworks like LangGraph) to ensure they can run securely and reliably in production.
-
Developer Experience: Abstract away the complexity of K8s and security compliance. Build the tooling and APIs that allow internal teams to spin up AI environments in minutes, not weeks.
-
Enterprise Hardening: Solve for the specific constraints of NYC finance—implementing rigorous OAuth 2.0 flows, secret management, air-gapped model serving, and immutable audit logs.
-
Observability: Implement deep tracing for LLM performance (token usage, latency, drift) using OpenTelemetry and Cortex.
The Candidate Profile
-
Experience: 10+ years in software engineering with 5+ years dedicated to distributed systems or platform engineering.
-
Industry Context: Experience working in Investment Banking, Fintech, or High-Frequency Trading environments. You understand that "speed" cannot come at the expense of "compliance."
-
Kubernetes Mastery: You aren't just a user; you understand the internals of K8s, operators, and mesh networking.
-
GenAI Reality: 1+ years of hands-on work with LLMs. You have moved beyond "Hello World" tutorials and understand the pain points of state management in LangChain or hosting embeddings at scale.
-
Security First: Deep knowledge of RBAC, OIDC, and secure coding practices required for handling sensitive financial data.
Why This Role? You won't just be building chatbots; you will be defining the engineering standards for how a major financial institution adopts AI for the next decade.
Share This Job:
Related Jobs:
Login to save this search and get notified of similar positions.About New York, NY
Discover exciting job opportunities in the vibrant area around New York, New York! With its iconic skyline, diverse cultural scene, and endless career growth prospects, the Big Apple and its surroundings promise an unparalleled work-life experience. Nestled between Broadway theaters, world-class art galleries like The Metropolitan Museum of Art, and globally renowned cuisine spots in Little Italy and Chinatown, finding your dream job in New York is your gateway to thriving in a city that never sleeps. Embrace the energy of Times Square, catch a game with the Yankees at Yankee Stadium, or unwind in the serenity of Central Park. Explore our job listings today and unlock your potential in the heart of opportunity.