Skip to main content

Welcome to Orchestr8

Orchestr8 is an enterprise-grade AI platform built on Kubernetes, designed to make deploying and managing AI workloads simple, secure, and scalable.

What is Orchestr8?​

Orchestr8 transforms Kubernetes into a comprehensive AI platform by combining:

🤖 AI Runtime Layer​

  • Llama-Stack: Enterprise-ready AI framework for LLM/RAG/Agent workloads
  • Multi-Provider Support: OpenAI, Anthropic, Groq, AWS Bedrock, and local models
  • GPU Orchestration: Intelligent GPU scheduling and resource management
  • Vector Databases: ChromaDB, Qdrant, PGVector for RAG applications

🔒 Enterprise Security & Governance​

  • ArgoCD: GitOps continuous delivery engine
  • Istio: Service mesh for traffic management and mTLS
  • Keycloak: Identity and Access Management (IAM)
  • OAuth2 Proxy: Service authentication layer
  • Network Policies: Zero-trust security by default

📊 Operational Excellence​

  • Prometheus/Grafana: AI-specific monitoring and observability
  • Cert-Manager: Automatic TLS certificate management
  • CloudNative PostgreSQL: High-performance database operator
  • Model Management: Versioning, caching, and lifecycle management

Getting Started​

Prerequisites​

  • uv package manager (automatically bootstraps Python)
  • Kubernetes cluster with GPU nodes (for AI workloads)
  • Additional CLI tools based on your deployment target

See the complete Installation Guide for setup instructions.

Installation​

Install the Orchestr8 CLI using uv:

uv tool install orchestr8-platform

Verify the installation:

o8 --version

Choose Your Platform​

🌩ī¸ Cloud Deployment​

Perfect for production workloads with enterprise features:

đŸ’ģ Local Development​

Ideal for development, testing, and learning:

One-Command Setup​

Once you've chosen your platform:

# For cloud deployment (replace 'aws' with your provider)
o8 setup --provider aws --domain your-domain.com

# For local development
o8 setup --provider local

This command will:

  • ✅ Set up the GitOps platform with ArgoCD
  • ✅ Configure Istio service mesh
  • ✅ Install Keycloak for identity management
  • ✅ Deploy Llama-Stack AI runtime
  • ✅ Set up AI-specific monitoring with Prometheus/Grafana
  • ✅ Configure TLS certificate management
  • ✅ Initialize GPU resource management

Verify Your Installation​

Check that everything is running:

o8 doctor

This command validates:

  • Cluster connectivity and GPU nodes
  • Platform components health
  • AI runtime configuration
  • Model provider access
  • Configuration correctness

What's Next?​

  1. Deploy Your First AI Workload - Create RAG or agent applications
  2. Explore Core Concepts - Understand GitOps, modules, and architecture
  3. Deploy Your First Module - Try example applications
  4. Configure Security - Harden your deployment
  5. Complete Installation Guide - Detailed setup instructions

AI Workloads​

Orchestr8 makes it easy to deploy and manage AI applications:

🤖 Create Your First AI Workload​

# Initialize a RAG application
o8 llama init my-rag-app --template rag --provider openai

# Or create an agentic workflow
o8 llama init my-agent --template agent --provider anthropic

# Deploy to your cluster
cd my-rag-app
o8 llama deploy --environment dev

📊 Monitor AI Applications​

# Check AI workload status
o8 llama status

# View real-time logs
o8 llama logs --follow

# List configured AI providers
o8 llama providers

🔧 Available AI Templates​

  • RAG Applications: Retrieval-Augmented Generation with vector search
  • Agentic Workflows: Multi-step reasoning with tool integration
  • Inference Services: High-performance model serving
  • Custom Workloads: Build your own AI applications

Key Features​

🤖 AI-Native Platform​

  • Multi-Provider AI: OpenAI, Anthropic, Groq, AWS Bedrock, and local models
  • GPU Orchestration: Intelligent scheduling and resource sharing
  • Vector Databases: ChromaDB, Qdrant, PGVector for RAG applications
  • Model Management: Caching, versioning, and lifecycle automation

🔒 Security & Compliance​

  • Zero-Trust Networking: Default-deny network policies with explicit allow rules
  • Pod Security Standards: Restricted security profiles enforced
  • Identity Management: Keycloak OIDC with OAuth2 Proxy integration
  • Compliance Ready: SOC2, GDPR, and HIPAA controls built-in

⚡ Enterprise Operations​

  • GitOps-First: All configuration stored in Git with ArgoCD managing deployments
  • Multi-Environment: Support for local, dev, staging, production environments
  • Module System: Deployable components with namespace isolation
  • AI Monitoring: Specialized metrics for model performance and costs

Need Help?​