ruby_llm

The Ruby AI framework

Build AI features the Ruby way: models, tools, agents, sandboxed code, evaluations, and Rails conventions, designed together, with one consistent API.

Website · Getting Started · What's New in 2.1

Battle tested at - Fully private work AI

Build with the models you want. Move between hosted and local providers without rewriting your application, or connect an OpenAI-compatible endpoint.

Browse models and pricing · Connect a provider

More than a client for model APIs. RubyLLM gives your application the conventions, the runtime, and every piece in between: agents that run code safely, workflows, evaluations, observability, and the data they need, designed to work together.

• Code execution (main): Let the model run Ruby, Python, JavaScript, and shell in a sandbox on your servers with with_code_execution
• Skills (main): Teach the model your workflows with folders of instructions and scripts in the Agent Skills format with with_skills
• MCP: Connect agents to your systems with server classes in app/mcp, per-user inputs, tool filtering, and OAuth with RubyLLM::MCP
• Agents: Reusable assistants with RubyLLM::Agent
• The agentic loop: Drive it yourself with ask_later, step, and complete?
• Evaluations (main): Reusable datasets, model criteria, and Ruby assertions with RubyLLM::Evaluation
• Chat: Conversational AI with RubyLLM.chat
• Vision: Analyze images and videos
• Audio: Transcribe speech with RubyLLM.transcribe and generate it with RubyLLM.speak
• Documents: Ask questions about PDFs, text files, and other supported formats
• OCR: Turn documents into markdown with RubyLLM.ocr
• Image generation: Create images with RubyLLM.paint
• Video generation: Create videos with RubyLLM.animate
• Embeddings: Generate embeddings with RubyLLM.embed
• Reranking: Order retrieval candidates by relevance with RubyLLM.rerank
• Moderation: Content flags, categories, and scores with RubyLLM.moderate
• Judgments: Probabilities, choices, and scores with RubyLLM::Judge and RubyLLM.judge
• Tools: Let AI call your Ruby methods
• Tool approval: Park a run until a human approves with requires_approval
• Provider tools: Web search, code execution, and MCP connectors with with_provider_tools
• Prompt templates: ERB prompts in app/prompts, rendered with RubyLLM.render_prompt
• Workflows: Correlate multi-agent runs in your telemetry with RubyLLM.workflow
• Structured output: Define a Ruby schema and read the result with response.parsed
• Streaming: Real-time responses with blocks
• Rails: Active Record persistence, Active Storage attachments, Hotwire streaming, and generators
• Files: Upload once and reuse across chats with RubyLLM.upload
• Prompt caching: Turn on the provider's cache with with_caching and cache_until_here
• Fallbacks and cancellation: Retry on backup models with with_fallbacks, stop a run with cancel
• Cost tracking: A per-attempt usage ledger behind chat.tokens and chat.cost
• Async: Fiber-based concurrency
• Model registry: Browse capabilities, limits, and pricing across providers
• Extended thinking: Control, view, and persist model deliberation
• Citations: Normalized source citations from documents, search, and grounding
• Batches: Provider-side batch processing with provider-specific discounts via RubyLLM.batch
• Compaction: Let providers condense long conversations with with_compaction
• Token counting: Count a request before you send it with count_tokens
• Providers: OpenAI, Azure, xAI, Anthropic, Gemini, VertexAI, Bedrock, Cohere, DeepSeek, Hetzner, Meta, MiniMax, Mistral, Moonshot AI, Ollama, Ollama Cloud, OpenRouter, Perplexity, GPUStack, ElevenLabs, Deepgram, TypeSafe, Z.ai, and any OpenAI-compatible API

Keep questions and expected answers in a dataset. RubyLLM checks correctness by default. Run the same evaluation from Ruby, RSpec, Minitest, or Rake.

Evaluations are available on main. Follow the development installation guide to try them.

app/agents/support_agent.rb

class SupportAgent < RubyLLM::Agent
  instructions "Accept returns of unopened items within 30 days."
end

app/evals/support_evaluation.rb

class SupportEvaluation < RubyLLM::Evaluation
  def perform(input)
    SupportAgent.new.ask(input)
  end
end

app/evals/support_evaluation.yml

cases:
  - name: within_return_window
    inputs: Can I return an unopened item after 14 days?
    expected_output: Yes, unopened items can be returned within 30 days.
  - name: past_return_window
    inputs: Can I return an unopened item after 45 days?
    expected_output: No, the 30-day return window has passed.

Run the evaluation:

report = SupportEvaluation.run
report.passed?
report.cost.total

The model evaluator checks meaning, so answers can use different wording. Declare your own criteria, add Ruby assertions, or choose a configured evaluator agent or judgment model. Read the evaluation guides.

Save conversations with Active Record and stream replies with Hotwire. The generators give you a working chat UI. Watch the two-minute demo.

# Install Rails Integration
bin/rails generate ruby_llm:install
bin/rails db:migrate
bin/rails ruby_llm:load_models

# Add Chat UI (optional)
bin/rails generate ruby_llm:chat_ui

class Chat < ApplicationRecord
  acts_as_chat
end

chat = Chat.create! model: "gpt-6-luna"
chat.ask "What's in this file?", with: "report.pdf"

Start your Rails server and visit http://localhost:3000/chats to try the chat interface. See the Rails guide for persistence, streaming, and background jobs.

These examples use 2.1.1. Follow Getting Started to install it and configure the providers you want to try. For 1.x, use the 1.x docs.

# Just ask
RubyLLM.chat.ask "What's the best way to learn Ruby?"

# Ask about files with a model that supports their input types
chat = RubyLLM.chat(model: "gemini-3.7-flash")
chat.ask "What's in this image?", with: "ruby_conf.jpg"
chat.ask "What's happening in this video?", with: "video.mp4"
chat.ask "Describe this meeting", with: "meeting.wav"
chat.ask "Summarize this document", with: "contract.pdf"
chat.ask "Explain this code", with: "app.rb"

# Multiple files at once
chat.ask "Analyze these files", with: ["diagram.png", "report.pdf", "notes.txt"]

# Stream responses
chat.ask "Tell me a story about Ruby" do |chunk|
  print chunk.content
end

# Generate images
image = RubyLLM.paint "a sunset over mountains in watercolor style"
image.save "sunset.png"

# Generate videos
video = RubyLLM.animate "a paper boat sailing down a rainy gutter"
video.save "paper_boat.mp4"

# Create embeddings
embedding = RubyLLM.embed "Ruby is elegant and expressive"
embedding.vector

# Rank search results
documents = ["Reset your password in Settings.", "Invoices arrive by email."]
ranked = RubyLLM.rerank("How do I reset my password?", documents, model: "rerank-v3.5")
ranked.results.first.document

# Transcribe audio to text
transcript = RubyLLM.transcribe "meeting.wav"
puts transcript.text

# Turn text into speech
speech = RubyLLM.speak "Hello, welcome to RubyLLM!"
speech.save "welcome.mp3"

# Extract document text as markdown
document = RubyLLM.ocr "contract.pdf"
puts document.markdown

# Check whether a moderation model flags content
RubyLLM.moderate("Some user-generated content").flagged?

# Ask a typed question about your data
class Urgency < RubyLLM::Judge
  probability :urgent, "Does this need attention today?"
end

Urgency.judge("Please refund my duplicate charge today.").urgent.probability

# Let AI use your code
class Weather < RubyLLM::Tool
  description "Get current weather"

  def execute(latitude:, longitude:)
    url = "https://api.open-meteo.com/v1/forecast?latitude=#{latitude}&longitude=#{longitude}&current=temperature_2m,wind_speed_10m"
    JSON.parse(Faraday.get(url).body)
  end
end

chat.with_tools(Weather).ask "What's the weather in Berlin?"

# Define an agent with instructions + tools
class WeatherAssistant < RubyLLM::Agent
  model "gpt-6-luna"
  instructions "Be concise and always use tools for weather."
  tools Weather
end

WeatherAssistant.new.ask "What's the weather in Berlin?"

# Get structured output
class ProductSchema < Schematist::Schema
  string :name
  number :price
  array :features do
    string
  end
end

response = chat.with_schema(ProductSchema).ask "Analyze this product", with: "product.txt"
response.parsed

Install RubyLLM 2.1:

bundle add ruby_llm --version 2.1.1

Configure a provider in your script, or in config/initializers/ruby_llm.rb in Rails:

require 'ruby_llm'

RubyLLM.configure do |config|
  config.openai_api_key = ENV.fetch('OPENAI_API_KEY')
end

Configure the other providers used by the examples as needed: Gemini for files, xAI for video, Mistral for OCR, and Cohere for reranking. Getting Started shows each setup beside its example. Upgrading an existing app? Follow the upgrade guide.

Give your coding assistant the RubyLLM API and documentation that match your application. From your application directory, install the skill packaged with your gem:

gem exec skill add "$(bundle show ruby_llm)" --skill rubyllm --agents coding

--agents coding installs it for the coding assistants your project uses, not into your app's app/skills. See AI Coding Assistants for setup and updates.

Guides · API reference · Models · Upgrading · 1.x docs

See CONTRIBUTING.md.

Released under the MIT License.