BoxLang 🚀 A New JVM Dynamic Language Learn More...

BoxLang AI

⚡︎ BoxLang AI

|:------------------------------------------------------:  |
| ⚡︎ B o x L a n g ⚡︎
| Dynamic : Modular : Productive
|:------------------------------------------------------:  |

Copyright Since 2023 by Ortus Solutions, Corp
ai.boxlang.io | www.boxlang.io
ai.ortussolutions.com | www.ortussolutions.com

👋 Welcome

BoxLang AI Module

Welcome to the BoxLang AI Module 🚀 The official AI library for BoxLang that provides a unified, fluent API to orchestrate multi-model workflows, autonomous agents, RAG pipelines, and AI-powered applications. One API → Unlimited AI Power! ✨

BoxLang AI eliminates vendor lock-in and simplifies AI integration by providing a single, consistent interface across 16+ AI providers. Whether you're using OpenAI, Claude, Gemini, Grok, DeepSeek, MiniMax, Ollama, or Perplexity—your code stays the same. Switch providers, combine models, and orchestrate complex workflows with simple configuration changes. 🔄

✨ Key Features

🔌 16+ AI Providers - Single API for OpenAI, Claude, AWS Bedrock, Gemini, Grok, MiniMax, Ollama, DeepSeek, and more
🤖 AI Agents - Autonomous agents with memory, tools, sub-agents, and multi-step reasoning
🔒 Multi-Tenant Memory - Enterprise-grade isolation with 20+ memory types (standard + vector)
🧬 Vector Memory & RAG - 12 vector databases with semantic search (ChromaDB, Pinecone, PostgreSQL, OpenSearch, etc.)
📚 Document Loaders - 30+ file formats including PDF, Word, CSV, JSON, XML, web scraping, and databases
🛠️ Real-Time Tools - Function calling for APIs, databases, and external system integration
🌊 Streaming Support - Real-time token streaming through pipelines for responsive applications
📦 Structured Output - Type-safe responses using BoxLang classes, structs, or JSON schemas
🔗 AI Pipelines - Composable workflows with models, transformers, and custom logic
🧩 Middleware - Cross-cutting controls for agents/models (logging, retries, guardrails, approval, and replay)
🎓 AI Skills - Reusable, composable knowledge blocks following the Claude Agent Skills open standard for modular agent behavior
📡 MCP Protocol - Build and consume Model Context Protocol servers for distributed AI
💬 Fluent Interface - Chainable, expressive syntax that makes AI integration intuitive
🦙 Local AI - Full Ollama support for privacy, offline use, and zero API costs
⚡ Async Operations - Non-blocking runAsync() on every runnable; aiParallel() for concurrent parallel pipelines
🎯 Event-Driven - 35+ lifecycle events for logging, monitoring, and custom workflows
🏭 Production-Ready - Timeout controls, error handling, rate limiting, and debugging tools
🧪 Testable - Deterministic replay for reliable unit and integration testing

📃 Table of Contents

📄 License
🚀 Getting Started
🤖 Supported Providers
- 📊 Provider Support Matrix
- 🔍 Provider Capability Discovery
📤 Return Formats
🥊 Quick Overview
⚙️ Settings
🛠️ Global Functions (BIFs)
📢 Events
🌐 GitHub Repository and Reporting Issues
🧪 Testing
💖 Ortus Sponsors

📃 License

BoxLang is open source and licensed under the Apache 2 license. 🎉 You can also get a professionally supported version with enterprise features and support via our BoxLang +/++ Plans (www.boxlang.io/plans). 💼

🚀 Getting Started

You can use BoxLang AI in both operating system applications, AWS Lambda, and web applications. For OS applications, you can use the module installer to install the module globally. For AWS Lambda and web applications, you can use the module installer to install it locally in your project or CommandBox as the package manager, which is our preferred method for web applications.

📚 New to AI concepts? Check out our Key Concepts Guide for terminology and fundamentals, or browse our FAQ for quick answers to common questions. We also have a Quick Start Guide and our intense AI BootCamp available to you as well.

OS

You can easily get started with BoxLang AI by using the module installer for building operating system applications:

install-bx-module bx-ai

This will install the latest version of the BoxLang AI module in your BoxLang environment. Once installed, configure your default AI provider and API key in boxlang.json:

{
    "modules": {
        "bxai": {
            "settings": {
                "provider": "openai",
                "apiKey": "${OPENAI_API_KEY}"
            }
        }
    }
}

💡 Tip: Use environment variable placeholders like ${OPENAI_API_KEY} so you never commit secrets to source control. Each provider also auto-detects its own env var (e.g. OPENAI_API_KEY, CLAUDE_API_KEY, GEMINI_API_KEY).

⚙️ All Available Settings

Below is the full reference of every setting you can place under settings in boxlang.json:

{
    "modules": {
        "bxai": {
            "settings": {
                "provider": "openai",
                "apiKey": "${OPENAI_API_KEY}",

                "defaultParams": {
                    "model": "gpt-4o",
                    "temperature": 0.7,
                    "max_tokens": 2000
                },

                "memory": {
                    "provider": "window",
                    "config": {
                        "maxMessages": 20
                    }
                },

                "providers": {
                    "openai": {
                        "params": { "model": "gpt-4o", "temperature": 0.7 },
                        "options": { "timeout": 60 }
                    },
                    "claude": {
                        "params": { "model": "claude-3-5-sonnet-20241022" }
                    },
                    "ollama": {
                        "params": { "model": "qwen2.5:0.5b-instruct" }
                    }
                },

                "timeout": 45,

                "logRequest": false,
                "logRequestToConsole": false,
                "logResponse": false,
                "logResponseToConsole": false,

                "returnFormat": "single",

                "skillsDirectory": "/.ai/skills",
                "autoLoadSkills": true,
                "globalSkills": []
            }
        }
    }
}

Setting	Type	Default	Description
`provider`	`string`	`"openai"`	Default AI provider to use for all requests
`apiKey`	`string`	`""`	Default API key; each provider also reads its own env var (e.g. `OPENAI_API_KEY`)
`defaultParams`	`struct`	`{}`	Default request parameters sent to every provider (e.g. `model`, `temperature`, `max_tokens`)
`memory.provider`	`string`	`"window"`	Default memory type: `window`, `cache`, `file`, `session`, `summary`, `jdbc`, `hybrid`, or any vector provider
`memory.config`	`struct`	`{}`	Provider-specific memory configuration (e.g. `maxMessages`, `cacheName`)
`providers`	`struct`	`{}`	Per-provider overrides — keys are provider names, values have `params` and `options` structs
`timeout`	`numeric`	`45`	Default HTTP request timeout in seconds
`logRequest`	`boolean`	`false`	Log outgoing AI requests to `ai.log`
`logRequestToConsole`	`boolean`	`false`	Print outgoing AI requests to the console (useful for debugging)
`logResponse`	`boolean`	`false`	Log AI responses to `ai.log`
`logResponseToConsole`	`boolean`	`false`	Print AI responses to the console (useful for debugging)
`returnFormat`	`string`	`"single"`	Default response format: `single`, `all`, `raw`, `json`, `xml`, or `structuredOutput`
`skillsDirectory`	`string`	`"/.ai/skills"`	Directory scanned for `SKILL.md` files at startup. Set to `""` to disable auto-discovery
`autoLoadSkills`	`boolean`	`true`	When `true`, skills found in `skillsDirectory` are auto-loaded and injected into every `aiAgent()` as global skills
`globalSkills`	`array`	`[]`	Internal — populated at startup with auto-discovered skills; access via `aiGlobalSkills()`

After that you can leverage the global functions (BIFs) in your BoxLang code. Here is a simple example:

// chat.bxs
answer = aiChat( "How amazing is BoxLang?" )
println( answer )

You can then run your BoxLang script like this:

boxlang chat.bxs

AWS Lambda

In order to build AWS Lambda functions with Boxlang AI for serverless AI agents and applications, you can use the Boxlang AWS Runtime and our AWS Lambda Starter Template. You will use the install-bx-module as well to install the module locally using the --local flag in the resources folder of your project:

cd src/resources
install-bx-module bx-ai --local

Or you can use CommandBox as well and store your dependencies in the box.json descriptor.

box install bx-ai resources/modules/

Web Applications

To use BoxLang AI in your web applications, you can use CommandBox as the package manager to install the module locally in your project. You can do this by running the following command in your project root:

box install bx-ai

Just make sure you have already a server setup with BoxLang. You can check our Getting Started with BoxLang Web Applications guide for more details on how to get started with BoxLang web applications.

🤖 Supported Providers

The following are the AI providers supported by this module. Please note that in order to interact with these providers you will need to have an account with them and an API key. 🔑

☁️ AWS Bedrock - Claude, Titan, Llama, Mistral via AWS
🧠 Claude Anthropic
🧬 Cohere
🔍 DeepSeek
🐳 Docker Model Runner - Local models via Docker Desktop
🎙️ ElevenLabs - Premium text-to-speech and speech-to-text
💎 Gemini
⚡ Grok
🚀 Groq
🤗 HuggingFace
🌀 Mistral
🌟 MiniMax
🦙 Ollama
🟢 OpenAI
🔌 OpenAI-Compatible - Any OpenAI-compatible API
🔀 OpenRouter
🔮 Perplexity
🚢 Voyage AI

📊 Provider Support Matrix

Here is a matrix of the providers and their feature support. Please keep checking as we will be adding more providers and features to this module. 🔄

Provider	Chat & Streaming	Real-time Tools	Embeddings	TTS (Speech)	STT (Transcription)
AWS Bedrock	✅	✅	✅	❌	❌
Claude	✅	✅	❌	❌	❌
Cohere	✅	✅	✅	❌	❌
DeepSeek	✅	✅	✅	❌	❌
Docker Model Runner	✅	✅	✅	❌	❌
ElevenLabs	❌	❌	❌	✅ (Premium)	✅ (Scribe v1)
Gemini	✅	[Coming Soon]	✅	✅	✅
Grok	✅	✅	✅	✅	❌
Groq	✅	✅	✅	❌	✅ (Whisper)
HuggingFace	✅	✅	✅	❌	❌
Mistral	✅	✅	✅	✅ (Voxtral)	✅ (Voxtral)
MiniMax	✅	✅	✅	❌	❌
Ollama	✅	✅	✅	❌	❌
OpenAI	✅	✅	✅	✅	✅ (Whisper)
OpenAI-Compatible	✅	✅	✅	❌	❌
OpenRouter	✅	✅	✅	❌	❌
Perplexity	✅	✅	❌	❌	❌
Voyage	❌	❌	✅ (Specialized)	❌	❌

🔍 Provider Capability Discovery

Every provider exposes a runtime capability API so you can introspect what it supports without consulting documentation — and without risking cryptic errors when you call an unsupported operation. 🛡️

// Get all capabilities a provider supports
var provider = aiService( "openai" );
var caps = provider.getCapabilities();
// → [ "chat", "stream", "embeddings" ]

// Check a specific capability before using it
if ( provider.hasCapability( "embeddings" ) ) {
    var embedding = aiEmbed( "Hello world" );
}

// Voyage is embeddings-only — getCapabilities() reflects this
var voyage = aiService( "voyage" );
voyage.getCapabilities(); // → [ "embeddings" ]
voyage.hasCapability( "chat" ); // → false

The built-in BIFs (aiChat, aiChatStream, aiEmbed) automatically use this system and throw a clear UnsupportedCapability exception when the selected provider does not implement the required capability:

// This will throw UnsupportedCapability — Voyage has no chat capability
aiChat( "Hello?", provider: "voyage" );

// This will throw UnsupportedCapability — Claude has no embeddings capability
aiEmbed( "some text", provider: "claude" );

Capabilities map to the following capability interfaces (in models/providers/capabilities/):

Capability String	Interface	Methods Provided
`chat`, `stream`	`IAiChatService`	`chat()`, `chatStream()`
`embeddings`	`IAiEmbeddingsService`	`embeddings()`
`speech`	`IAiSpeechService`	`speak()`
`transcription`	`IAiTranscriptionService`	`transcribe()`, `translate()`

📤 Return Formats

BoxLang not only makes it extremely easy to interact with multiple AI providers, but it also gives you the flexibility to choose how you want the responses returned to you. You can specify the return format using the responseFormat parameter in your AI calls. Here are the available formats:

Format	Description
`single`	Returns a single message as a string (the content from the first choice). This is the default format for BIFs.
`all`	Returns an array of all choice messages. Each message is a struct with `role` and `content` keys.
`json`	Returns the parsed JSON object from the content string. Automatically parses JSON responses.
`xml`	Returns the parsed XML document from the content string. Automatically parses XML responses.
`raw`	Returns the full raw response from the AI provider. This is useful for debugging or when you need the full response structure with metadata. This is the default for pipelines.
`structuredOutput`	Used internally when `.structuredOutput()` is called. Returns a populated class/struct based on the schema.

🥊 Quick Overview

In the following sections, we provide a quick overview of the main components of BoxLang AI including Chats, Pipelines, Middleware, Agents, Structured Output, Memory Systems, Document Loaders & RAG, and MCP Client/Server. Each section includes quick examples and links to more detailed documentation. For further details, please refer to the official documentation, this is just a high-level overview to get you started quickly. 🚀

💬 Chats

Interact with AI models through simple and powerful chat interfaces 🎯 supporting both one-shot responses and streaming conversations. BoxLang AI provides fluent APIs for building everything from basic Q&A to complex multi-turn dialogues with system prompts, message history, and structured outputs. 💡

🤔 Why Use Chats?

⚡ Simple & Fast - One-line chat interactions with aiChat()
🔄 Streaming Support - Real-time token streaming with aiChatStream()
💾 Memory Integration - Automatic conversation history with memory systems
🎨 Flexible Messages - Support for text, images, files, and structured data
🌊 Fluent API - Chain message builders for readable, maintainable code

💡 Quick Examples

Simple One-Shot Chat:

// Quick question-answer
response = aiChat( "What is BoxLang?" )
println( response )

// With custom model and options
response = aiChat(
    messages: "Explain quantum computing",
    params: { model: "gpt-4", temperature: 0.7, max_tokens: 500 }
)

Multi-Turn Conversation with Memory:

// Create agent with memory
agent = aiAgent(
    name: "Assistant",
    memory: aiMemory( "window", config: { maxMessages: 10 } )
)

// First turn
response = agent.run( "My name is Luis" )

// Second turn - Agent remembers context
response = agent.run( "What's my name?" )
println( response ) // "Your name is Luis"

Streaming Chat:

// Stream tokens as they arrive
aiChatStream(
    messages: "Write a short story about a robot",
    callback: chunk => {
        writeOutput( chunk.choices?.first()?.delta?.content ?: "" )
		bx:flush;
    },
    params: { model: "claude-3-5-sonnet-20241022" }
)

Fluent Message Builder:

// Build complex message chains
messages = aiMessage()
    .system( "You are a helpful coding assistant" )
    .user( "How do I create a REST API in BoxLang?" )
    .image( "diagram.png" )

response = aiChat(
    messages: messages,
    params: { model: "gpt-4o", temperature: 0.7 }
)

📚 Learn More

🚀 Quick Start: Getting Started Guide
📖 Full Guide: Chatting Documentation
🌊 Streaming: Streaming Guide
🎨 Message Formats: Message Builder Guide

🔗 Pipelines

Build composable AI workflows 🎯 using BoxLang AI's powerful runnable pipeline system. Chain models, transformers, tools, and custom logic into reusable, testable components that flow data through processing stages. Perfect for complex AI workflows, data transformations, and multi-step reasoning. 💡

🤔 Why Use Pipelines?

🔄 Composable - Chain any runnable components together with .to()
🧪 Testable - Each pipeline stage is independently testable
♻️ Reusable - Build once, use in multiple workflows
🌊 Streaming - Full streaming support through entire pipeline
🎯 Type-Safe - Input/output contracts ensure data flows correctly

💡 Quick Examples

Simple Transformation Pipeline:

// Create a pipeline with model and transformers
pipeline = aiModel( provider: "openai" )
    .transform( data => data.toUpperCase() )
    .transform( data => data.trim() )

// Run input through the pipeline
result = pipeline.run( "hello world" )
println( result ) // "HELLO WORLD"

Multi-Stage AI Pipeline:

// Define transformation stages as closures
summarizer = ( text ) => {
    return aiChatAsync(
        aiMessage().system( "Summarize in one sentence" ).user( text ),
        { model: "gpt-4o-mini" }
    )
}

translator = ( summary ) => {
    return aiChatAsync(
        aiMessage().system( "Translate to Spanish" ).user( summary ),
        { model: "gpt-4o" }
    )
}

formatter = ( translatedText ) => {
    return {
        summary: translatedText,
        timestamp: now()
    }
}

// Compose pipeline using async futures
result = summarizer( "Long article text here..." )
    .then( summary => translator( summary ) )
    .then( translated => formatter( translated ) )
    .get()

println( result.summary ) // Spanish summary

Streaming Pipeline:

// Stream through entire pipeline
pipeline = aiModel( provider: "claude", params: { model: "claude-3-5-sonnet-20241022" } )
    .transform( chunk => chunk.toUpperCase() )
	.stream(
		onChunk: ( chunk ) => writeOutput( chunk ),
		input: "Tell me a story"
	)

Custom Runnable Class:

// Implement IAiRunnable for custom logic
class implements="IAiRunnable" {

    function run( input, params = {} ) {
        // Custom processing
        return processedData;
    }

    function stream( onChunk, input, params = {} ) {
        // Streaming support
        onChunk( processedChunk );
    }

    function to( nextRunnable ) {
        // Chain to next stage
        return createPipeline( this, nextRunnable );
    }
}

// Use in pipeline
customStage = new CustomRunnable()
pipeline = aiModel( provider: "openai", params: { model: "gpt-4o" } )
	.to( customStage )

Parallel Pipelines:

Run multiple runnables concurrently with the same input and receive a named struct of results. Mirrors LangChain's RunnableParallel — parallelism is a developer/framework concern, not something the LLM decides.

// Fan out to multiple agents/models in parallel
results = aiParallel({
    summary:  summaryAgent,
    analysis: analysisAgent,
    keywords: keywordModel
}).run( "Some long document..." )

// results.summary, results.analysis, results.keywords — all ran concurrently

// Compose in a pipeline — parallel branch then merge
pipeline = aiMessage( "Analyze: ${text}" )
    .to( aiParallel({ researcher: researchAgent, writer: writerAgent }) )
    .transform( r => "Research: #r.researcher#\nDraft: #r.writer#" )

// Or dispatch the same agent for multiple independent inputs asynchronously
futures = [
    researchAgent.runAsync( "Topic A" ),
    researchAgent.runAsync( "Topic B" ),
    researchAgent.runAsync( "Topic C" )
]
results = futures.map( f => f.get() )

📚 Learn More

📖 Full Guide: Runnables & Pipelines
🎯 Overview: Main Components
🔧 Custom Runnables: Building Custom Components
💻 Examples: Check examples/pipelines/ for complete examples

🧩 Middleware

Add cross-cutting behavior around model and agent execution without changing your business logic. Use middleware for observability, reliability, safety, approvals, and deterministic replay in testing. 🎯

🤔 Why Use Middleware?

🔍 Observability - Capture lifecycle logs and execution traces
♻️ Reliability - Retry transient model/tool failures
🛡️ Safety - Block risky tools or argument patterns
👤 Human Approval - Require confirmation for sensitive tool calls
🎬 Replayability - Record and replay runs for regression testing

📦 Core Middleware Included

Middleware	Purpose
`LoggingMiddleware`	Logs agent/model lifecycle activity for observability and troubleshooting.
`RetryMiddleware`	Retries transient LLM/tool failures with configurable backoff.
`MaxToolCallsMiddleware`	Enforces a per-run cap on total tool calls to prevent runaway execution.
`GuardrailMiddleware`	Blocks disallowed tools and rejects risky tool arguments by pattern rules.
`HumanInTheLoopMiddleware`	Requires human approval for selected tool calls (CLI or suspend/resume flow).
`FlightRecorderMiddleware`	Records and replays LLM/tool interactions for deterministic testing and CI.

📌 Registration

Middleware can be attached to agents, models, or both. They are executed in the order registered; after* hooks fire in reverse order (cleanup order).

Via aiAgent() / aiModel() BIF parameter:

import bxModules.bxai.models.middleware.core.LoggingMiddleware;
import bxModules.bxai.models.middleware.core.RetryMiddleware;
import bxModules.bxai.models.middleware.core.GuardrailMiddleware;

agent = aiAgent(
    name: "Safe Assistant",
    middleware: [
        new LoggingMiddleware( logToConsole: true ),
        new RetryMiddleware( maxRetries: 2, initialDelay: 250 ),
        new GuardrailMiddleware( blockedTools: [ "deleteRecord" ] )
    ]
)

Via withMiddleware() on a runnable (fluent API):

agent
    .withMiddleware( new LoggingMiddleware() )
    .withMiddleware( new RetryMiddleware( maxRetries: 3 ) )

// Or pass an array — flattened automatically
agent.withMiddleware( [ mw1, mw2, mw3 ] )

Management methods (available on AiAgent and AiModel):

Method	Returns	Description
`withMiddleware( any middleware )`	`this`	Add one or more middleware (instance, struct, or array)
`clearMiddleware()`	`this`	Remove all registered middleware
`listMiddleware()`	`array`	Return array of `{ name, description }` for all middleware

When an agent runs, its middleware is prepended to any middleware already on the model, so agent-level hooks always fire first.

🪝 Hooks Reference

Middleware exposes two hook styles:

Sequential hooks — called in order; return AiMiddlewareResult. Chain stops if any hook returns a terminal result.

Hook	Fires	Direction
`beforeAgentRun( context )`	Before agent starts	Forward
`afterAgentRun( context )`	After agent completes	Reverse
`beforeLLMCall( context )`	Before each LLM provider call	Forward
`afterLLMCall( context )`	After each LLM provider call	Reverse
`beforeToolCall( context )`	Before each tool is invoked	Forward
`afterToolCall( context )`	After each tool returns	Reverse
`onError( context )`	When any hook throws an exception	—

Wrap hooks — called as nested closures; call handler() to proceed and return a value.

Hook	Purpose
`wrapLLMCall( context, handler )`	Surround each LLM provider call (retry, caching, tracing)
`wrapToolCall( context, handler )`	Surround each tool invocation (retry, mocking, sandboxing)

For wrap hooks, the first registered middleware is the outermost wrapper:

mw1.wrapLLMCall( ctx, () =>
    mw2.wrapLLMCall( ctx, () =>
        actualProviderCall()
    )
)

📬 AiMiddlewareResult

Every sequential hook must return an AiMiddlewareResult. Use the static factory methods:

import bxModules.bxai.models.middleware.AiMiddlewareResult;

// Continue the chain normally
return AiMiddlewareResult.continue()

// Stop the chain immediately (terminal)
return AiMiddlewareResult.cancel( "Too many sensitive operations" )

// Human approved (HITL)
return AiMiddlewareResult.approve()

// Human rejected (terminal)
return AiMiddlewareResult.reject( "Operator rejected this action" )

// Human edited the tool arguments (passes modified args to tool)
return AiMiddlewareResult.edit( { correctedArgs: { query: "safe query" } } )

// Suspend for async human review (terminal — web mode HITL)
return AiMiddlewareResult.suspend( { toolName: "deleteRecord", args: toolArgs } )

Checking results:

Predicate	Meaning
`isContinue()`	Chain proceeds normally
`isCancelled()`	Chain was stopped (terminal)
`isApproved()`	Human approved
`isRejected()`	Human rejected (terminal)
`isEdit()`	Arguments were modified
`isSuspended()`	Waiting for async human input (terminal)
`isTerminal()`	`cancel`, `reject`, or `suspend` — stops the chain

📚 Learn More

📖 Full Guide: Middleware Documentation
💻 Examples: Check examples/agents/ and examples/pipelines/

🗂️ Tool Registry

The AI Tool Registry is a global singleton that stores named ITool instances. Register tools once — at module load, on application start, or anywhere in your code — and then reference them by string name wherever tools are accepted. This decouples tool definitions from call sites and makes it easy to share tools across agents, models, and pipelines. 🎯

🤔 Why Use the Registry?

🔑 By-name references — pass "now@bxai" as a string instead of a live object
📦 Module scoping — namespace tools as toolName@moduleName to avoid collisions
🔍 Lazy resolution — tools are resolved to ITool instances right before each LLM request
🔌 Auto-scanning — annotate methods with @AITool and call scan() to register them all at once
⚡ Built-in tools — now@bxai (current date/time), speak@bxai (text-to-speech), transcribe@bxai (speech-to-text), and translate@bxai (audio-to-English) are registered automatically on module load

💡 Quick Examples

Using the registry:

// Register a tool once (e.g., in Application.bx or a module's onLoad)
aiToolRegistry().register(
    name        : "searchProducts",
    description : "Search the product catalog",
    callback    : ( required string query ) => productService.search( query )
)

// Later, reference by name — no object needed
result = aiChat(
    "Find me wireless headphones under $50",
    { tools: [ "searchProducts" ] }
)

Module-namespaced tools:

// Namespaced registration avoids collisions across modules
aiToolRegistry().register(
    name        : "lookup",
    description : "Look up a customer by ID",
    callback    : id => customerService.find( id ),
    module      : "my-app"
)

// Retrieve by full key or bare name (auto-resolved if unambiguous)
var tool = aiToolRegistry().get( "lookup@my-app" )
var tool = aiToolRegistry().get( "lookup" )       // works if only one "lookup" exists

Scanning a class for @AITool annotations:

// Annotate methods in a class
class WeatherTools {
    @AITool( "Get current weather for a city" )
    public string function getWeather( required string city ) {
        return weatherAPI.fetch( city )
    }

    @AITool( "Get a 7-day forecast for a city" )
    public string function getForecast( required string city ) {
        return weatherAPI.forecast( city )
    }
}

// Register all annotated methods in one call
aiToolRegistry().scan( new WeatherTools(), "my-module" )
// Registered as: getWeather@my-module, getForecast@my-module

Using the built-in now@bxai tool:

// Auto-registered on module load — just reference it by name
result = aiChat(
    "What should I have for dinner tonight?",
    { tools: [ "now@bxai" ] }
)
// AI knows the current date/time without any extra wiring

Using the built-in audio tools (speak@bxai, transcribe@bxai, translate@bxai):

// All three are auto-registered on module load — opt in by name
var agent = aiAgent(
    name         : "VoiceAssistant",
    instructions : "You are a helpful voice assistant. Speak responses aloud.",
    tools        : [ "now@bxai", "speak@bxai", "transcribe@bxai", "translate@bxai" ]
)

// Agent can now convert text to speech, transcribe audio files, or translate audio to English
agent.run( "Say hello to the user and tell them today's date" )
// → AI calls speak@bxai with the greeting text, returns the saved audio file path

// Standalone transcription — agent calls transcribe@bxai automatically
agent.run( "Please transcribe the file at /recordings/meeting.mp3" )

// Translation — agent calls translate@bxai for non-English audio
agent.run( "Translate the Spanish audio at /audio/mensaje.mp3 to English" )

Opt-in httpGet tool (NOT auto-registered):

// Register explicitly when your application needs web access
import bxModules.bxai.models.tools.core.CoreTools;
aiToolRegistry().scan( new CoreTools(), "bxai" )  // registers httpGet@bxai too

Opt-in FileSystemTools (NOT auto-registered — supply allowedPaths for safety):

import bxModules.bxai.models.tools.filesystem.FileSystemTools;

// Restrict the AI to specific directories (strongly recommended)
aiToolRegistry().scanClass(
    new FileSystemTools( allowedPaths: [ "/workspace/data", "/tmp/ai-output" ] ),
    "bxai"
)

// Give a coding agent full filesystem capabilities within a project directory
agent = aiAgent(
    name         : "CodingAssistant",
    instructions : "You are a coding assistant. You can read, write, search, and organize files.",
    tools        : [
        // Read / write
        "readFile@bxai",
        "readMultipleFiles@bxai",
        "writeFile@bxai",
        "appendFile@bxai",
        "editFile@bxai",
        // File management
        "fileMetadata@bxai",
        "pathExists@bxai",
        "deleteFile@bxai",
        "moveFile@bxai",
        "copyFile@bxai",
        // Search & utilities
        "searchFiles@bxai",
        "listAllowedDirectories@bxai",
        // Directories
        "listDirectory@bxai",
        "directoryTree@bxai",
        "createDirectory@bxai",
        "deleteDirectory@bxai",
        // Zip / archive
        "zipFiles@bxai",
        "unzipFile@bxai",
        "checkZipFile@bxai"
    ]
)

agent.run( "Read src/main/bx/App.bx and add a header comment to it" )
agent.run( "List all .bx files under src/ recursively" )
agent.run( "Create a directory reports/ and write a summary.txt file there" )
agent.run( "Search src/ for all files containing the word 'deprecated'" )
agent.run( "Zip the entire src/ directory to /tmp/src-backup.zip" )

🔐 Security note: FileSystemTools validates every path against the allowedPaths list using canonical path resolution, preventing directory-traversal attacks. Leave allowedPaths empty only in fully-trusted environments.

🧑‍💻 Custom Tools via `BaseTool`

For more complex tools ones that need their own state, unit tests, or reusable class structure extend BaseTool directly instead of using a closure. You only need to implement two abstract methods:

Method	Purpose
`doInvoke( required struct args, AiChatRequest chatRequest )`	The tool logic. Return any value serialization is handled automatically.
`generateSchema()`	Return the OpenAI function-calling schema struct. Called by `getSchema()` unless a manual schema override has been set.

invoke() is final on BaseTool it fires the beforeAIToolExecute / afterAIToolExecute events and serializes the result before calling your doInvoke(), so you never need to wire those up yourself.

// MySearchTool.bx
class extends="bxModules.bxai.models.tools.BaseTool" {

    property name="searchClient";

    function init( required any searchClient ) {
        variables.name        = "searchProducts"
        variables.description = "Search the product catalog and return matching items"
        variables.searchClient = arguments.searchClient
        return this
    }

    /**
     * Core tool logic return any type, BaseTool serializes it automatically.
     */
    public any function doInvoke( required struct args, AiChatRequest chatRequest ) {
        return variables.searchClient.search(
            query      : args.query,
            maxResults : args.maxResults ?: 5
        )
    }

    /**
     * OpenAI function-calling schema for this tool.
     */
    public struct function generateSchema() {
        return {
            "type": "function",
            "function": {
                "name"       : variables.name,
                "description": variables.description,
                "parameters" : {
                    "type"      : "object",
                    "properties": {
                        "query"     : { "type": "string",  "description": "Search query text" },
                        "maxResults": { "type": "integer", "description": "Maximum number of results to return" }
                    },
                    "required": [ "query" ]
                }
            }
        }
    }
}

// Register in the global registry
aiToolRegistry().register( new MySearchTool( searchClient ), "my-app" )

// Reference by key name anywhere tools are accepted
result = aiChat( "Find wireless headphones", { tools: [ "searchProducts@my-app" ] } )

Fluent schema helpers (inherited from BaseTool) let you skip writing generateSchema() manually when ClosureTool's auto-introspection isn't available:

tool = new MySearchTool( client )
    .describeFunction( "Search the product catalog" )   // sets description
    .describeQuery( "Search term to look up" )           // describeArg( "query", "..." )
    .describeMaxResults( "Max items to return" )         // describeArg( "maxResults", "..." )

Or supply a fully hand-crafted schema with setSchema( schemaStruct ) when set, it takes precedence over generateSchema().

📚 Learn More

📖 Full Guide: AI Tool Registry Documentation
💻 Examples: Check examples/advanced/ for complete registry examples

🤖 AI Agents

Build autonomous AI agents 🎯 that can use tools, maintain memory, and orchestrate complex workflows. BoxLang AI agents combine LLMs with function calling, memory systems, and orchestration patterns to create intelligent assistants that can interact with external systems and solve complex tasks. 💡

🤔 Why Use Agents?

🛠️ Tool Integration - Agents can execute functions, call APIs, and interact with external systems
🧠 Stateful Intelligence - Built-in memory keeps context across multi-turn interactions
🔄 Self-Orchestration - Agents decide which tools to use and when
🎯 Goal-Oriented - Give high-level instructions, agents figure out the steps
🤝 Human-in-the-Loop - Optional approval workflows for sensitive operations

💡 Quick Examples

Simple Agent with Tools:

// Define tools the agent can use
weatherTool = aiTool(
    name: "get_weather",
    description: "Get current weather for a location",
    callable: ( location ) => {
        return { temp: 72, condition: "sunny", location: location };
    }
)

// Create agent with memory
agent = aiAgent(
    name: "Weather Assistant",
    description: "Helpful weather assistant",
    tools: [ weatherTool ],
    memory: aiMemory( "window" )
)

// Agent decides when to call tools
response = agent.run( "What's the weather in Miami?" )
println( response ) // Agent calls get_weather tool and responds

Autonomous Agent with Multiple Tools:

// Agent with database and email tools
agent = aiAgent(
    name: "Customer Support Agent",
    tools: [
        aiTool( name: "query_orders", description: "Query customer orders", callable: orderQueryFunction ),
        aiTool( name: "send_email", description: "Send email to customer", callable: emailFunction ),
        aiTool( name: "create_ticket", description: "Create support ticket", callable: ticketFunction )
    ],
    memory: aiMemory( "session" ),
    params: { max_iterations: 5 }
)

// Agent orchestrates multiple tool calls
agent.run( "Find order #12345, email the customer with status, and create a ticket if there's an issue" )

Multi-Agent Hierarchy (Sub-Agents):

// Create specialist sub-agents
researchAgent = aiAgent(
    name: "researcher",
    description: "Researches topics in depth",
    instructions: "Provide thorough research summaries"
)

writerAgent = aiAgent(
    name: "writer",
    description: "Writes polished content",
    instructions: "Turn research into engaging articles"
)

// Coordinator automatically registers sub-agents as callable tools
coordinator = aiAgent(
    name: "coordinator",
    description: "Orchestrates research and writing",
    subAgents: [ researchAgent, writerAgent ]
)

// Coordinator decides when to delegate
coordinator.run( "Write an article about BoxLang AI" )

// Inspect the hierarchy
writeln( researchAgent.getAgentPath() )   // /coordinator/researcher
writeln( researchAgent.getAgentDepth() )  // 1
writeln( researchAgent.isRootAgent() )    // false
writeln( coordinator.getRootAgent().getAgentName() ) // coordinator

📚 Learn More

📖 Full Guide: AI Agents Documentation
🎓 Interactive Course: Lesson 6 - Building AI Agents
🔧 Advanced Patterns: Agent Orchestration
💻 Examples: Check examples/agents/ for complete working examples

🎯 AI Skills

Give agents and models reusable, composable knowledge blocks 📚 that can be injected into the system message at runtime. Skills follow the Claude Agent Skills open standard — a description field tells the LLM when to apply the skill, and the body contains the full instructions. 🧩

🤔 Why Use Skills?

📖 Reusable knowledge - Define domain expertise once, share across many agents and models
🗂️ File-based management - Store skills as SKILL.md files in your project, commit alongside code
⚡ Two loading modes - Always-on for universal guidance; lazy-loaded for large skill libraries
🔌 Zero-code discovery - Drop a SKILL.md into .ai/skills/my-skill/ and it's available automatically
🌐 Global skill pool - Register global skills once in module config, automatically available to all agents

📋 Skill File Format

Skills live in named subdirectories under .ai/skills/:

.ai/skills/
    sql-optimizer/
        SKILL.md
    boxlang-expert/
        SKILL.md
    customer-tone/
        SKILL.md

Each SKILL.md file uses optional YAML frontmatter and a Markdown body:

---
description: Optimise SQL queries for maximum performance. Apply when writing or reviewing database queries.
---

## SQL Optimisation Rules

- Always use indexed columns in WHERE clauses
- Prefer JOINs over subqueries for large datasets
- Use EXPLAIN to verify query plans before deploying
- Avoid SELECT * in production queries

Tip: If you omit the frontmatter, the first paragraph of the body is used as the description.

💡 Quick Examples

Inline skill on a model:

// Create an inline skill (no files needed)
sqlSkill = aiSkill(
    name       : "sql-optimizer",
    description: "Apply SQL optimisation rules when writing or reviewing queries",
    content    : "Always use indexed columns. Prefer JOINs over subqueries."
)

// Always-on: injected into every call
model = aiModel( "openai" ).withSkills( [ sqlSkill ] )
response = model.run( "Write a query to get all orders" )

Load skills from the filesystem:

// Load all SKILL.md files from .ai/skills/ (recursive by default)
skills = aiSkill( ".ai/skills" )

// Or load a single skill file
sqlSkill = aiSkill( ".ai/skills/sql-optimizer/SKILL.md" )

// Seed an agent with all discovered skills
agent = aiAgent(
    name           : "data-assistant",
    availableSkills: skills    // Lazy pool — LLM loads on demand
)

Always-on vs lazy skills:

// Always-on: full content injected every call (small, universal skills)
coreSkill = aiSkill( ".ai/skills/writing-style/SKILL.md" )
agent.withSkills( [ coreSkill ] )

// Lazy pool: only a compact index is included; LLM calls loadSkill() as needed
bigLibrary = aiSkill( ".ai/skills" )   // Hundreds of skills
agent.withAvailableSkills( bigLibrary )

// activateSkill() promotes a lazy skill to always-on mid-session
agent.activateSkill( "sql-optimizer" )

Global skills auto-injected into every agent:

// In ModuleConfig.bx settings — all agents get these automatically
settings = {
    globalSkills: aiSkill( expandPath( ".ai/skills" ) )
}

// Or register programmatically via the BIF
globalSkillPool = aiGlobalSkills()   // returns the current global pool

Inspect skill state:

config = agent.getConfig()
writeln( config.activeSkillCount )         // always-on skills count
writeln( config.availableSkillCount )      // lazy skills count

// Render the full skill system-message block for debugging
writeln( agent.buildSkillsContent() )

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased

3.1.0 - 2026-04-16

🥊 New Features

Audio Support — Text-to-Speech, Transcription, and Translation:
- aiSpeak( text, params, options ) BIF: Convert text to speech using any provider that supports TTS. Returns an AiSpeechResponse (with hasAudio(), saveToFile(), getBase64(), getMimeType(), getSize()) or saves directly to a file via options.outputFile.
- aiTranscribe( audio, params, options ) BIF: Transcribe audio (file path, URL, or binary) to text. Returns the transcript string by default or a full AiTranscriptionResponse when options.returnFormat = "response".
- aiTranslate( audio, params, options ) BIF: Translate non-English audio to English text using supported providers.
- IAiSpeechService interface: Implemented by providers that support TTS (speak()).
- IAiTranscriptionService interface: Implemented by providers that support STT (transcribe() + translate()).
- Provider support: OpenAI (TTS + STT), Mistral/Voxtral (TTS + STT), Groq/Whisper (STT + translation), xAI/Grok (TTS), Gemini (TTS + STT), ElevenLabs (TTS + STT — new dedicated audio provider).
- ElevenLabsService: New provider supporting high-quality TTS via eleven_multilingual_v2 and STT via scribe_v1. Use aiService("elevenlabs", apiKey).
- 6 new interception points: beforeAISpeech, afterAISpeech, beforeAITranscription, afterAITranscription, beforeAITranslation, afterAITranslation.
- audio settings block in module config: defaultVoice, defaultOutputFormat, defaultSpeechModel, defaultTranscriptionModel.
Audio Agent Tools — speak@bxai, transcribe@bxai, translate@bxai: New AudioTools class (models/tools/audio/AudioTools.bx) auto-registered in the global tool registry at module startup. speak@bxai converts text to speech and returns the saved file path (auto-generates a temp file when no outputFile is supplied). transcribe@bxai transcribes a local file or URL to plain text. translate@bxai translates any-language audio to English text. Opt-in by name: aiAgent( tools: [ "speak@bxai", "transcribe@bxai", "translate@bxai" ] ).
FileSystem Agent Tools — New FileSystemTools class (models/tools/filesystem/FileSystemTools.bx) with 19 @AITool-annotated methods covering the full filesystem lifecycle. NOT auto-registered — opt-in only via aiToolRegistry().scanClass() so agents never get filesystem access unless explicitly granted. Supports a path-guard constructor (allowedPaths: [...]) that canonicalizes and validates every path argument before execution, blocking directory-traversal attacks. Tool keys: readFile@bxai, readMultipleFiles@bxai, writeFile@bxai, appendFile@bxai, editFile@bxai, fileMetadata@bxai, pathExists@bxai, deleteFile@bxai, moveFile@bxai, copyFile@bxai, searchFiles@bxai, listAllowedDirectories@bxai, listDirectory@bxai, directoryTree@bxai, createDirectory@bxai, deleteDirectory@bxai, zipFiles@bxai, unzipFile@bxai, checkZipFile@bxai.
Async Runnables and Parallel Execution:
- runAsync() on all runnables (IAiRunnable, AiBaseRunnable): Every runnable now has a non-blocking runAsync(input, params, options) method that dispatches execution to the io-tasks virtual thread pool and returns a BoxFuture. Mirrors the existing aiChatAsync, loadAsync(), and seedAsync() patterns throughout the module.
- AiRunnableParallel class (models/runnables/AiRunnableParallel.bx): New runnable that accepts a named struct of runnables, fans them out concurrently via runAsync(), and returns a { name: result } struct once all futures complete. Mirrors LangChain's RunnableParallel — a structural parallel composition primitive that integrates cleanly into the existing pipeline system via .to(), .run(), and .runAsync().
- aiParallel() BIF: Creates an AiRunnableParallel from a named struct of runnables. aiParallel({ summary: summaryAgent, analysis: analysisAgent }).run("document") runs both concurrently and returns { summary: "...", analysis: "..." }.

🪲 Fixed

chatStream() across all providers never fires the onAITokenCount event, making streaming calls completely invisible to usage tracking, billing, and monitoring. The non-streaming chat() path fires it correctly.
AiModel.stream(): inject agent and model middleware into chatRequest, matching the existing pattern in run()
DockerModelRunnerService: capture arguments into local vars before retryOnModelLoading closure to prevent ArgumentsScope resolution failure
OpenAIService.chat(): capture chatRequest before nested .each() closures for tool calling
OpenAIService.chatStream(): scope callback and chatRequest for sendStreamRequest call and tool-calling .each() closure
CohereService.chat(): capture chatRequest before .map() tool closure
Standardized the data for the onAITokenCount event and add missing event on the following services: BedrockService, ClaudeService, CohereService, GeminiService
MCPServer scan() and scanClass() where not working accordingly with all cases and permutations.
Invalid location of directory for flight recorder tapes
aiAgent() bif, skills, availableSkills can now be an array or a single skill, we will normalize it to an array internally. This allows for more flexible agent construction with a single skill without needing to wrap it in an array.
ModuleConfig.bx listens now to onRuntimeStart() in order to setup skills and more, so caches and other things are properly loaded before the modules.
Docker Service issues with interface upgrades from previous version.

3.0.0 - 2026-04-02

2.4.0 - 2026-02-20

2.3.0 - 2026-02-18

Added

Pipeline _input System Variable: Auto-inject previous stage output into message templates via ${_input}. For struct outputs, individual fields are flattened as ${_input_fieldName} for template access. Enables clean, composable multi-stage AI pipelines without manual transformation steps.
aiTransform() needd to process instances of AiTransformRunnable and BaseTransformer classes, allowing for more flexible and reusable transformation logic.
Stricter and more defensive code when doing tool calling, to prevent errors when tools are called with invalid arguments or when the tool execution fails.

Fixed

Tool calling with streaming was not working because the tools were being executed in a different context that didn't have access to the request. Now the request is properly passed to the tool execution context, allowing tools to be called and executed correctly during streaming.
Agent stream() was not passing tools the correct request, now it does.
scoping issue on Agent streaming
fixed BaseMemory getRecent() where limit was not being used
SummaryMemory was not trimming messages when the summary threshold was exceeded, and it was recursing forever on summary. Now it properly trims messages until it gets under the threshold, then summarizes and adds the summary message back in.
BaseTransformer was missing it's internal constructor
Default for config on all BaseTransformer classes was missing.
Fixed a bug where if the aiTransform() BIF was called with a non-string or closure, the throw() was invalid.

2.2.0 - 2026-02-16

Added

AI Skills system (aiSkill() BIF + withSkills() / withAvailableSkills() APIs on AiModel and AiAgent): Composable, reusable knowledge blocks — following the Claude Agent Skills open standard — that can be injected into any model or agent system message at runtime.
- aiSkill( path | name, description, content, recurse ) — Creates or discovers AiSkill instances. Pass a file path to load a single SKILL.md, a directory path to auto-discover all skills recursively, or name/description/content for inline definitions with no files needed.
- aiGlobalSkills() — Returns the globally shared pool of skills auto-injected into every new agent's availableSkills pool. Populated via ModuleConfig.bx → settings.globalSkills.
- Always-on skills (withSkills() / addSkill()): Full skill content is injected into the system message on every call. Best for small, universally relevant guidance.
- Lazy skills (withAvailableSkills() / addAvailableSkill()): Only a compact index (name + description) is included in the system message. The LLM calls the auto-registered loadSkill( name ) tool to fetch full content on demand. Best for large or rarely needed skill libraries.
- activateSkill( name ) — Moves a skill from the lazy pool to always-on, promoting it for the rest of the session.
- buildSkillsContent() — Renders the combined skills system-message block for inspection or custom injection.
- SKILL.md format: Each skill lives in its own subdirectory under .ai/skills/. The file is Markdown with an optional YAML frontmatter block containing description. The body is the instruction content. If frontmatter is absent, the first paragraph of body text is used as the description.
- AiModel and AiAgent getConfig() now include activeSkillCount, availableSkillCount, and skills (a struct with activeSkills and availableSkills name/description arrays) for full introspection.
- aiAgent() BIF gains skills: [] and availableSkills: [] construction-time parameters. Global skills from aiGlobalSkills() are automatically prepended to every new agent's available pool.
- aiModel() BIF gains a skills: [] construction-time parameter.
MCP server seeding for agents and models: Agents and models can now be seeded directly with one or more MCP servers. All tools exposed by those servers are automatically discovered via listTools() and registered as MCPTool instances — no manual Tool construction required.
- New MCPTool class (models/tools/MCPTool.bx) implements ITool by proxying a single MCP server tool. It converts the MCP inputSchema to the OpenAI function-calling schema format and forwards invocations to the server via MCPClient.send().
- New withMCPServer( server, config ) fluent method on AiAgent and AiModel. Accepts a URL string or a pre-configured MCPClient instance. Optional config struct supports token, timeout, headers, user, and password.
- New withMCPServers( servers ) fluent method on AiAgent and AiModel for seeding from multiple servers in one call. Each entry can be a URL string, a config struct { url, token, timeout, … }, or a pre-configured MCPClient.
- New listMcpServers() method on AiAgent and AiModel returns the list of currently connected MCP servers with their exposed tools for introspection and debugging.
- aiAgent() and aiModel() BIFs gain an array mcpServers = [] parameter so servers can be provided at construction time.
- AiAgent now tracks connected MCP servers in a mcpServers property ([{ url, toolNames }]). This list is automatically injected into the system prompt so the LLM can correctly answer questions like "what MCP servers are you connected to?" and "which tools came from which server?"
- New listTools() method on AiAgent returns [{ name, description }] for all registered tools — useful for programmatic introspection.
- AiAgent|AiModel.getConfig() now includes tools (full name/description list) and mcpServers (server URL + tool-name list) alongside the existing toolCount.
Global AI Tool Registry: New singleton AIToolRegistry (accessible via aiToolRegistry() BIF) provides a module-scoped registry for AI tools. Tools can be registered by name with optional module namespacing (e.g. now@bxai), discovered at runtime by bare name or full key, and resolved lazily before LLM requests via aiToolRegistry().resolveTools(). This means tools can be referenced by string name in params.tools arrays and resolved automatically rather than requiring live object references.
BaseTool abstract base class: All tool implementations now extend BaseTool, which provides the shared invocation lifecycle (firing beforeAIToolExecute and afterAIToolExecute interception events), result serialization (primitives pass through, complex values serialize to JSON), and the fluent describeArg() / describe[ArgName]() schema annotation syntax.
ClosureTool class: Replaces the retired Tool.bx. A BaseTool subclass backed by any closure or lambda. Auto-introspects the callable's parameter metadata to generate an OpenAI-compatible function schema. Receives the originating AiChatRequest as _chatRequest for context-aware closures.
CoreTools built-in tools: Ships two tools out of the box. now (registered automatically as now@bxai on module load) returns the current date/time in ISO 8601 — ideal for giving the AI temporal awareness. httpGet (opt-in only, not auto-registered for security) fetches any URL via HTTP GET. Register it explicitly if your application requires web access.
Lazy tool resolution: params.tools arrays in aiChat(), aiModel().run(), and aiAgent().run() now accept string registry keys alongside live ITool instances. AIToolRegistry::resolveTools() converts any string keys to their registered ITool before the request is sent.
Two new interception points: onAIToolRegistryRegister and onAIToolRegistryUnregister.
Structured output for ollama tools, allowing for more complex and rich tool responses that can include multiple fields and nested data instead of just a single string output.
Streaming tools for ollama, allowing tools to return data in a streaming fashion for real-time processing and response generation.
Tools can now have non-required arguments in their schema
Tools can now access the full AiChatRequest object during invocation, allowing for more complex and context-aware tool behavior. They receive a _chatRequest argument that includes all the properties of the original request, such as messages, params, options, and more. This enables tools to make informed decisions based on the full conversation context and request configuration.
HuggingFace embeddings support
Ability to send a custom URL to the different senders in the base service.
Middleware support for AiModel and AiAgent, with agent middleware prepended ahead of model middleware.
Provider lifecycle hooks in preRequest(), postResponse(),for any custom logic before and after requests to change the shape of the request or response, log additional data, etc. These hooks are provider-specific and allow for custom behavior without needing to override the entire sendChatRequest() method.
Per-call identity routing on all memory types: add(), getAll(), clear(), trim(), seed(), and related methods on every IAiMemory and IVectorMemory implementation now accept optional userId and conversationId arguments. This follows the Spring AI ChatMemory pattern — a single memory instance can safely serve multiple tenants without creating a new instance per user. Construction-time values remain as fallbacks.
Provider capability interfaces: New models/providers/capabilities/ package introduces IAiChatService and IAiEmbeddingsService — scoped interfaces that let providers declare exactly which operations they support at the type level rather than through runtime throws.
getCapabilities() / hasCapability() on all providers: Every provider now exposes getCapabilities() (returns ["chat", "stream", "embeddings", ...]) and hasCapability( "chat" ) for clean, self-documenting runtime introspection. These are backed by isInstanceOf() checks and stay automatically in sync with the implements declarations on each provider — no maintenance required.
AiAgent parent-child hierarchy: AiAgent now tracks its position in a multi-agent tree through a parentAgent property and a full set of hierarchy helpers:
- setParentAgent(parent) — assign a parent with self-reference and cycle-detection guards
- clearParentAgent() — detach from a parent
- hasParentAgent() — returns true if the agent has a parent
- isRootAgent() — returns true for top-level agents
- getRootAgent() — walks up the tree and returns the root agent
- getAgentDepth() — returns the nesting depth (0 = root, 1 = direct child, …)
- getAgentPath() — returns a slash-delimited path string, e.g. /coordinator/researcher
- getAncestors() — returns an ordered array [immediateParent, …, root]
- addSubAgent() now automatically calls setParentAgent(this) on the sub-agent
- setSubAgents() now calls clearParentAgent() on replaced sub-agents before replacing them
- getConfig() now includes parentAgent (name string), agentDepth, and agentPath

Changed

Refactored all runnable objects to the runnables folder. This includes AiModel, AiAgent, and AiMessage. This better reflects their purpose as executable entities that can be run with different inputs, and allows for a cleaner separation between the core service logic and the runnable wrappers.
Refactored the BaseService to be truly a base and move all OpenAI specific logic to OpenAIService, which now serves as the default provider implementation. This allows for cleaner implementations of other providers that don't need to override every method.
AiAgent is now fully stateless: userId, and conversationId are resolved per-call from the options argument passed to run() and stream(), eliminating shared-state concurrency bugs in multi-user deployments. Seeding a memory with userId and conversationId is still supported, but these values will be overridden by any values passed in at call time.
resume() and resumeStream() now require threadId as an explicit required string argument instead of defaulting to the former instance property.
IAiService contract trimmed: The base interface now declares only identity/configuration/capability-discovery methods (getName(), configure(), getCapabilities(), hasCapability()). The operation methods (invoke(), invokeStream(), embeddings()) have moved to their respective capability interfaces where they belong.
VoyageService now extends BaseService directly and implements only IAiEmbeddingsService — it no longer extends OpenAIService with stubbed-out chat methods that threw at runtime. The type system now enforces the embeddings-only constraint at compile time.
aiChat(), aiChatStream(), and aiEmbed() BIF guards: Each BIF now checks the provider implements the required capability interface before attempting the call and throws a clear UnsupportedCapability exception instead of a cryptic provider error. Zero breaking changes to public BIF signatures.

Improvements

Renamed BaseService.sendRequest() to sendChatRequest().
Reduced duplicate payload fields in onAITokenCount.

Fixed

Model and Agent streaming was not announcing global pre/post events
Changelog corruption due to merge conflict.
MCP requestId null scope crash on JSON-RPC notifications for MCP servers
MiniMax chat errors (base_resp.status_code != 0) now surface correctly.
OllamaService stale postEmbeddingResponse() hook: The old hook was never wired to the current BaseService lifecycle and silently did nothing. Replaced with the proper postResponse( aiRequest, dataPacket, result, operation ) override that guards on operation != "embeddings", identical to how every other dual-capability provider handles this.

2.4.0 - 2026-02-20

Added

MiniMax AI Provider: Added support for MiniMax AI service with chat, streaming, and embeddings support. Use the minimax provider name and set your API key via the MINIMAX_API_KEY environment variable.
Updated getConfig() to not show sensitive info.

Fixed

BoxLang static constructs instead of inline to avoid issues with never versions.

2.3.0 - 2026-02-18

Added

Pipeline _input System Variable: Auto-inject previous stage output into message templates via ${_input}. For struct outputs, individual fields are flattened as ${_input_fieldName} for template access. Enables clean, composable multi-stage AI pipelines without manual transformation steps.
aiTransform() needd to process instances of AiTransformRunnable and BaseTransformer classes, allowing for more flexible and reusable transformation logic.
Stricter and more defensive code when doing tool calling, to prevent errors when tools are called with invalid arguments or when the tool execution fails.

Fixed

Tool calling with streaming was not working because the tools were being executed in a different context that didn't have access to the request. Now the request is properly passed to the tool execution context, allowing tools to be called and executed correctly during streaming.
Agent stream() was not passing tools the correct request, now it does.
scoping issue on Agent streaming
fixed BaseMemory getRecent() where limit was not being used
SummaryMemory was not trimming messages when the summary threshold was exceeded, and it was recursing forever on summary. Now it properly trims messages until it gets under the threshold, then summarizes and adds the summary message back in.
BaseTransformer was missing it's internal constructor
Default for config on all BaseTransformer classes was missing.
Fixed a bug where if the aiTransform() BIF was called with a non-string or closure, the throw() was invalid.

2.2.0 - 2026-02-16

Added

Consolidated AI request/response logging with execution time metrics for better performance insights.
Improved AI request/response to include other metrics in order to provide better insights into performance and potential bottlenecks.

Improved

Consolidation of options and settings, to have a single source of truth for configuration and to allow for better overrides and defaults.
Stream request logging to include execution time metrics for better performance monitoring and debugging insights.
If the chunk is empty, skip it (keep-alive or heartbeat) when doing chat streams. This prevents unnecessary processing of empty chunks and potential errors when parsing.

Fixed

Invalid use of request in the aiChatStream() BIF, which should have been chatRequest.
Extends for AiTransformRunnable was wrong.
AiModel extractMessages() was not flattening the messages correctly when the response had multiple choices with multiple messages. Now it properly flattens all messages from all choices into a single array.
Order of settings merging in aiChat() and aiChatStream() BIFs was incorrect, causing default options to override user-provided options. Now it merges in the correct order: user options → module settings → default options, allowing for proper overrides.
Error invoking population in schema builder, the third argument needs to be an array or struct, not a single value.
Fixed a bug where provider options in the configuration file were not being merged into the request options when creating a service instance.
Fixed a bug where the aiService() BIF was not correctly applying convention-based API key detection when options.apiKey was already set but empty. Now it checks if options.apiKey is empty before applying the convention key, allowing for proper fallback to environment variables or module settings.

2.1.0 - 2026-02-04

What's New: https://ai.ortusbooks.com/readme/release-history/2.1.0

Added

New event: onMissingAiProvider to handle cases where a requested provider is not found.
aiModel() BIF now accepts an additional options struct to seed services.
New configuration: providers so you can predefine multiple providers in the module config, with default params and options.

"providers" : {
	"openai" : {
		"params" : {
			"model" : "gpt-4"
		},
		"options" : {
			"apiKey" : "my-openai-api-key"
		}
	},
	"ollama" : {
		"params" : {
			"model" : "qwen2.5:0.5b-instruct"
		},
		"options" : {
			"baseUrl" : "http://my-ollama-server:11434/"
		}
	}
}

OllamaService now supports custom base URLs for both chat and embeddings endpoints via the options.baseUrl parameter.
AiBaseRequest.mergeServiceParams() and AiBaseRequest.mergeServiceHeaders() methods now accept an override boolean argument to control whether existing values should be overwritten when merging.
Local Ollama docker setup instructions updated to include the nomic-embed-text model for embeddings support.
Ollama Service now supports embedding generation using the nomic-embed-text model.
Multi-Tenant Usage Tracking: Provider-agnostic request tagging for per-tenant billing
- New tenantId option for attributing AI usage to specific tenants
- New usageMetadata option for custom tracking data (cost center, project, userId, etc.)
- Enhanced onAITokenCount events with tenant context for interceptor-based billing
- Works with all providers: OpenAI, Bedrock, Ollama, DeepSeek, etc.
- Fully backward compatible - existing code works unchanged
Provider-Specific Options Support: Generic providerOptions struct for provider-specific settings
- New providerOptions option for passing provider-specific configuration (e.g., inferenceProfileArn for Bedrock)
- New getProviderOption(key, defaultValue) method on requests for retrieving provider options
- Enables extensibility for any provider-specific features without polluting the common interface
OpenSearch Vector Memory Provider: Full integration with OpenSearch k-NN for semantic search
- Support for OpenSearch 2.x and 3.x with automatic version detection and space type mapping
- HNSW index configuration options (M, ef_construction, ef_search parameters)
- Space type options: cosinesimilarity, l2, innerproduct
- Basic authentication support (username/password)
- AWS region configuration for SigV4 authentication with AWS OpenSearch Service
- Multi-tenant isolation with userId and conversationId filtering
- Comprehensive test coverage for configuration, validation, and operations
OpenAI-Compatible Embedding Support: Vector memory providers now support custom embedding endpoints
- New embeddingOptions configuration in BaseVectorMemory for passing options to embedding provider
- Use embeddingOptions.baseURL for custom OpenAI-compatible embedding service URLs
- Allows using self-hosted or alternative OpenAI-compatible embedding services
- Works with providers like Ollama, LM Studio, and other compatible APIs
AWS Bedrock Streaming Support: Full streaming support for Bedrock provider
- Streaming via InvokeModelWithResponseStream API endpoint
- Support for all model families: Claude, Titan, Llama, Mistral
- AWS event-stream format parsing with base64 payload decoding
- OpenAI-compatible streaming response format for consistent callback handling
- Added more AiError exception handling for service json errors.

Changed

All AI provider services now inherit default chat and embedding parameters from the IAiService interface, ensuring consistent behavior across providers.
IAiService.configure() method now accepts a generic options argument instead of apiKey, to better reflect its purpose and support more configuration options.
AiRequest class renamed to AiChatRequest for clarity, and multi-modality support.

Fixed

Events for chat requests were incorrectly named in the ModuleConfig.bx file. Corrected to onAIChatRequest, onAIChatRequestCreate, and onAIChatResponse.
aiChat, aiChatStream BIF was not passing headers to the AiChatRequest.
aiChat, aiChatStream, aiChatAsync BIF was not using aiChatRequest() to build the request, but was building it manually.
According to the MCP spec prompts should return a key named "arguments" not "args".
AiRequest was not setting the model correctly from params.
API key was not being passed to the service in aiChat(), aiChatStream() BIF.
Typo of chr() --> char() in SSE formatting in MCPRequestProcessor and HTTPTransport.
AiModel.getModel() was not returning the model name correctly when using predefined providers from config.
Increased Docker Model Runner retry time to 5 seconds with 10 max retries to accommodate large model loading times
Fixed url parameter conflict in OpenSearchVectorMemory by using requestUrl for HTTP requests

2.0.0 - 2026-01-19

What's New: https://ai.ortusbooks.com/readme/release-history/2.0.0

One of our biggest library updates yet! This release introduces a powerful new document loading system, comprehensive security features for MCP servers, and full support for several major AI providers including Mistral, HuggingFace, Groq, OpenRouter, and Ollama. Additionally, we have implemented complete embeddings functionality and made numerous enhancements and fixes across the board.

Added

Document Loaders: New document loading system for importing content from various sources
- New aiDocuments() BIF for loading documents with automatic type detection
- New aiDocumentLoader() BIF for creating loader instances with advanced configuration
- New aiDocumentLoaders() BIF for retrieving all registered loaders with metadata
- New aiMemoryIngest() BIF for ingesting documents into memory with comprehensive reporting:
  - Single memory or multi-memory fan-out support
  - Async processing for parallel ingestion
  - Automatic chunking with aiChunk() integration
  - Token counting with aiTokens() integration
  - Cost estimation for embedding operations
  - Detailed ingestion report (documentsIn, chunksOut, stored, skipped, deduped, tokenCount, embeddingCalls, estimatedCost, errors, memorySummary, duration)
- New Document class for standardized document representation with content and metadata
- New IDocumentLoader interface and BaseDocumentLoader abstract class for custom loaders
- Built-in Loaders:
  - TextLoader: Plain text files (.txt, .text)
  - MarkdownLoader: Markdown files with header splitting, code block removal
  - HTMLLoader: HTML files and URLs with script/style removal, tag extraction
  - CSVLoader: CSV files with row-as-document mode, column filtering
  - JSONLoader: JSON files with field extraction, array-as-documents mode
  - DirectoryLoader: Batch loading from directories with recursive scanning
- Fluent API for loader configuration
- Integration with memory systems via loadTo() method and aiMemoryIngest() BIF
- Automatic document chunking support for vector memory
- Comprehensive documentation in docs/main-components/document-loaders.md
MCP Server Enterprise Security Features: Comprehensive security enhancements for MCP servers
- CORS Configuration:
  - withCors(origins) - Configure allowed origins (string or array)
  - addCorsOrigin(origin) - Add origin dynamically
  - getCorsAllowedOrigins() - Get configured origins array
  - isCorsAllowed(origin) - Check if origin is allowed with wildcard matching
  - Support for wildcard patterns (*.example.com)
  - Support for allowing all origins (*)
  - Dynamic Access-Control-Allow-Origin header in responses
  - CORS headers included in OPTIONS preflight responses
- Request Body Size Limits:
  - withBodyLimit(maxBytes) - Set maximum request body size in bytes
  - getMaxRequestBodySize() - Get current limit (0 = unlimited)
  - Returns 413 Payload Too Large error when exceeded
  - Protects against DoS attacks with oversized payloads
- Custom API Key Validation:
  - withApiKeyProvider(provider) - Set custom API key validation callback
  - hasApiKeyProvider() - Check if provider is configured
  - verifyApiKey(apiKey, requestData) - Manual key validation
  - Supports X-API-Key header and Authorization: Bearer token
  - Provider receives API key and request context for flexible validation
  - Returns 401 Unauthorized for invalid keys
- Security Headers: Automatic inclusion of industry-standard security headers in all responses
  - X-Content-Type-Options: nosniff
  - X-Frame-Options: DENY
  - X-XSS-Protection: 1; mode=block
  - Referrer-Policy: strict-origin-when-cross-origin
  - Content-Security-Policy: default-src 'none'; frame-ancestors 'none'
  - Strict-Transport-Security: max-age=31536000; includeSubDomains
  - Permissions-Policy: geolocation=(), microphone=(), camera=()
- Security Processing Order: Body size → CORS → Basic Auth → API Key → Request processing
- Comprehensive documentation in docs/advanced/mcp-server.md with examples
- Security configuration examples in main README.md
- 9 new integration tests covering all security features
Mistral AI Provider Support: Full integration with Mistral AI services
- New MistralService provider class with OpenAI-compatible API
- Chat completions with streaming support
- Embeddings support with mistral-embed model
- Tool/function calling support
- Default model: mistral-small-latest
- API key detection via MISTRAL_API_KEY environment variable
- Comprehensive integration tests
HuggingFace Provider Support: Full integration with HuggingFace Inference API
- New HuggingFaceService provider class extending BaseService
- OpenAI-compatible API endpoint at router.huggingface.co/v1
- Default model: Qwen/Qwen2.5-72B-Instruct
- Support for chat completions and embeddings
- Integration tests for HuggingFace provider
- API key pattern: HUGGINGFACE_API_KEY
Groq Provider Support: Full integration with Groq AI services for fast inference
- Uses OpenAI-compatible API at api.groq.com
- Default model: llama-3.3-70b-versatile
- Support for chat completions, streaming, and embeddings
- Environment variable: GROQ_API_KEY
Embeddings Support: Complete embeddings functionality for semantic search, clustering, and recommendations
- New aiEmbedding() BIF for generating text embeddings
- New AiEmbeddingRequest class to model embedding requests
- New embeddings() method in IAiService interface
- Support for single text and batch text embedding generation
- Multiple return formats: raw, embeddings, first
- Provider Support:
  - OpenAI: text-embedding-3-small and text-embedding-3-large models
  - Ollama: Local embeddings for privacy-sensitive use cases
  - DeepSeek: OpenAI-compatible embeddings API
  - Grok: OpenAI-compatible embeddings API
  - OpenRouter: Aggregated embeddings via multiple models
  - Gemini: Custom implementation with text-embedding-004 model
- New embedding-specific events: onAIEmbeddingRequest, onAIEmbeddingResponse, beforeAIEmbedding, afterAIEmbedding
- Comprehensive embeddings documentation in README with examples
- New examples/embeddings-example.bx demonstrating practical use cases
- Integration tests for embeddings functionality
ChatMessage now has the following new methods:
- format(bindings) - Formats messages with provided bindings.
- render() - Renders messages using stored bindings.
- bind( bindings ) - Binds variables to be used in message formatting.
- getBindings(), setBindings( bindings ) - Getters and setters for bindings.
Detect API Keys by convention in AIService() BIF: <PROVIDER>_API_KEY from system settings
OpenRouter Provider Support: Full integration with OpenRouter AI services
Automatic JSON serialization for tool calls that don't return strings
Ollama Provider Support: Complete integration with Ollama for local AI model execution
Comprehensive Provider Test Suite: Individual test files for each AI provider
Streaming Support Validation: Verified aiChatStream() functionality across all providers
Docker Compose Testing Infrastructure: Automated local development and CI/CD support
Enhanced GitHub Actions Workflow: Improved CI/CD pipeline with AI service support
BIF Reference Documentation: Complete function reference table in README
Comprehensive Event Documentation: Complete event system documentation

Fixed

If a tool argument doesn't have a description, it would cause an error when generating the schema. Default it to the argument name.
Model Name Compatibility: Updated OllamaService default model from llama3.2 to qwen2.5:0.5b-instruct
Docker GPU Support: Made GPU configuration optional in docker-compose.yml for systems without GPU access
Test Model References: Corrected model names in Ollama tests to match available models

1.2.0 - 2025-06-19

Added

New gradle wrapper and build system
New Tool.getArgumentsSchema() method to retrieve the arguments schema for use by any provider.
New logging params for console debugging: logRequestToConsole, logResponseToConsole
Tool support for Claude LLMs
Tool message for open ai tools when no local tools are available.
New ChatMessage helper method: getNonSystemMessages() to retrieve all messages except the system message.
ChatRequest now has the original ChatMessage as a property, so you can access the original message in the request.
Latest Claude Sonnet model support: claude-sonnet-4-0 as its default.
Streamline of env on tests
Added to the config the following options: logRequest, logResponse, timeout, returnFormat, so you can control the behavior of the services globally.
Some compatibilities so it can be used in CFML apps.
Ability for AI responses to be influenced by the onAIResponse event.

Fixed

Version pinned to 1.0.0 in the box.json file by accident.

1.1.0 - 2025-05-17

Added

Claude LLM Support
Ability for the services to pre-seed params into chat requests
Ability for the services to pre-seed headers into chat requests
Error logging for the services

Fixed

Custom headers could not be added due to closure encapsulation

1.0.1 - 2025-03-21

Fixed

Missing the settings in the module config.
Invalid name for the module config.

1.0.0 - 2025-03-17

First iteration of this module

Ortus Solutions
Published
Minimum Version: 1.8.0
3.1.0+11 is the latest of 18 release(s)
Published
Published on {{ getFullDate("2026-04-16T15:35:09Z") }}
Apache-2.0
ortus-boxlang/bx-ai

$ box install bx-ai

How do I do this?

No collaborators yet.

BoxLang AI
Created On: {{ getFullDate("2025-03-05T22:10:21Z") }}
Last Publish: {{ getFullDate("2026-04-16T15:35:09Z") }}
Views: 4,580
Installs: 1,816

Report Abuse

⚡︎ BoxLang AI

👋 Welcome

✨ Key Features

📃 Table of Contents

📃 License

🚀 Getting Started

OS

⚙️ All Available Settings

AWS Lambda

Web Applications

🤖 Supported Providers

📊 Provider Support Matrix

🔍 Provider Capability Discovery

📤 Return Formats

🥊 Quick Overview

💬 Chats

🤔 Why Use Chats?

💡 Quick Examples

📚 Learn More

🔗 Pipelines

🤔 Why Use Pipelines?

💡 Quick Examples

📚 Learn More

🧩 Middleware

🤔 Why Use Middleware?

📦 Core Middleware Included

📌 Registration

🪝 Hooks Reference

📬 AiMiddlewareResult

📚 Learn More

🗂️ Tool Registry

🤔 Why Use the Registry?

💡 Quick Examples

🧑‍💻 Custom Tools via BaseTool

📚 Learn More

🤖 AI Agents

🤔 Why Use Agents?

💡 Quick Examples

📚 Learn More

🎯 AI Skills

🤔 Why Use Skills?

📋 Skill File Format

💡 Quick Examples

Changelog

Unreleased

3.1.0 - 2026-04-16

🥊 New Features

🪲 Fixed

3.0.0 - 2026-04-02

2.4.0 - 2026-02-20

2.3.0 - 2026-02-18

Added

Fixed

2.2.0 - 2026-02-16

Added

Changed

Improvements

Fixed

2.4.0 - 2026-02-20

Added

Fixed

2.3.0 - 2026-02-18

Added

Fixed

2.2.0 - 2026-02-16

Added

Improved

Fixed

2.1.0 - 2026-02-04

Added

Changed

Fixed

2.0.0 - 2026-01-19

Added

Fixed

1.2.0 - 2025-06-19

Added

Fixed

1.1.0 - 2025-05-17

Added

🧑‍💻 Custom Tools via `BaseTool`