2nd GSI/FAIR AI Workshop

We present a versatile production-grade LLM inference and document intelligence server designed for GSI/FAIR computing infrastructure.

The system employs a modern containerized architecture built on vLLM for GPU-accelerated inference, OpenWebUI for user interaction, and Milvus vector database for Retrieval-Augmented Generation (RAG) capabilities.
The deployment utilizes several specialized vLLM instances running multiple models with support for thinking, tool calling and vision capabilities. A dedicated embedding model for document vectorization and retrieval is available as well.
This multi-model approach, deployed on an 8xH200 GPU server, balances computational cost with task-specific performance requirements.
The RAG implementation enables users to upload and query documentation, code repositories, and technical papers using natural language.
API access allows users to explore and deploy their own solutions while leveraging high performance hardware.
The architecture supports production deployment on a server with enterprise authentication and development configurations for testing on resource-constrained hardware.

Performance characteristics, deployment patterns, and lessons learned from production usage are presented, along with token consumption metrics across different workflows.
This work demonstrates that modern LLM technologies can be successfully integrated into GSI/FAIR computing infrastructure while maintaining security, auditability and performance.