About the Role

We are looking for a highly skilled Lead AI Solutions Engineer to design, build, and optimize a Retrieval-Augmented Generation (RAG) system that underpins our self-serve analytics data applications. In this pivotal position, you will develop a scalable RAG platform, integrate multiple data sources, and create intuitive data interactions to empower teams across the organization.

As part of our Kindred RAG Initiative, you will collaborate with a cross-functional team to implement self-service AI-driven solutions - ranging from NLP Data Analysis and Data Discovery to other “analytics' assistant” we will be deploying to the business to streamlining data access, insights retrieval, and business process automation.

About A&I The Automation and Insights (A&I) department is at the forefront of transforming Kindred into a highly automated, data-driven powerhouse. In this dynamic environment, you'll work alongside experts in data, insights, and cutting-edge AI to build an AI-ready ecosystem that automates data management, insight generation, and machine-to-machine interactions. This isn't just about advancing technology; it's about reshaping our operational DNA to drive innovation, efficiency, and exceptional customer experiences in the new era of analytics and automation.

Key Responsibilities:

RAG System Development:

· Enhance, build new assistants and maintain existing scalable and modular RAG architectures for data retrieval and generation.

· Develop APIs and microservices and dbt to integrate RAG capabilities with existing data sources (mainly user behaviour data, metadata and semantic layer, stored in Redshift, but also other sources e.g. HR systems, legal repositories, SharePoint, etc.).

· Manage and optimize LibreChat UI and Openweb UI Pipelines for chatbot interactions.

Data Engineering & Integration:

· Build efficient data pipelines to support LLM-based querying, semantic search, and metadata retrieval.

· Integrate structured (SQL-based) and unstructured (documents, reports) data sources for real-time and batch processing.

· Maintain and troubleshoot Airflow pipelines for embedding extraction and document processing.

· Ensure data governance, security, and compliance across all applications.

· Manage Vector Database (PGVector), including indexing and similarity search optimizations.

Application Development:

· Develop interactive UI components to enable self-serve data access and visualization using React or other web frameworks.

· Apps examples: generic text to SQL, funnels, user journeys exploration, retention, features active users, attribution, data exploration tool etc.

· Collaborate with data analysts to enable seamless user experiences for natural language queries and structured analytics and data modelling using dbt.

· Implement query translation and enhancement techniques to improve LLM accuracy and retrieval quality.

· Use AI to speed up code development e.g. Cursor

Security & Infrastructure Management:

· Collaborate with Platform Engineering teams to manage apps Kubernetes deployments on Kindred Cloud.

· Ensure security integrations with Azure SSO and SailPoint.

· Work with the Network & Security team to manage configurations for newly created services.

Testing & Deployment:

· Develop unit and integration tests to ensure system reliability and performance.

· Validate applications against real-world data and scenarios to assess LLM accuracy and output quality.

· Deploy applications in Kindred Cloud environments, ensuring scalability and monitoring.

· Use Kubernetes, Docker, Helm and ArgoCD to manage deployments.

Documentation & Knowledge Transfer:

· Deliver technical documentation covering system architecture, APIs, and workflows.

· Conduct training sessions for internal teams to facilitate self-service capabilities and ongoing system enhancements.

Required Skills & Qualifications:

· 5+ years of experience in data engineering, backend development, or AI/ML integration.

· Strong knowledge of Python, FastAPI, Flask, or Node.js for backend API development.

· Experience with LLM-based architectures, retrieval-augmented generation (RAG), and NLP techniques.

· Proficiency in SQL, Redshift, and data warehousing concepts.

· Experience integrating structured and unstructured data sources for AI-driven applications.

· Knowledge of dbt, metadata management, and semantic search.

· Familiarity with React or other frontend frameworks for building intuitive UIs.

· Cloud deployment experience in AWS or Kindred Cloud.

· Experience with Kubernetes, Terraform, and Helm for deployment management.

· Strong problem-solving skills and ability to work in a fast-paced, agile environment.

Nice-to-Have Skills:

· Hands-on experience with Vector Databases (FAISS, Pinecone, Weaviate, PGVector, etc.).

· Experience fine-tuning LLMs for domain-specific applications.

· Knowledge of data privacy, governance, and compliance in AI-driven systems.

· Previous work in self-service analytics or AI-powered business intelligence solutions.

· Experience with Javascript for frontend customization.

· Experience with Airflow for ETL workflow orchestration.

LI-HYBRID #LI-ML1 #LI-MM1