Details

Written by: Brandon J. Yaniz

Category: Open Source

• Open Source

yLLMKit

Local-First LLM Tools for Swift Apps

yLLMKit is a Swift package for building text and chat features into macOS and Apple platform apps. It gives developers a provider-neutral core for working with language models, while keeping local inference, remote providers, streaming responses, model settings, usage data, and context preparation behind a consistent application-facing interface.

The project is built around a local-first idea: applications should be able to use powerful language model features without assuming that every request must leave the user’s machine. Local Apple Silicon inference through MLX is the primary path, while optional hosted provider integrations make it possible to use services such as OpenAI or Anthropic when remote models are the better fit.

yLLMKit is designed for developers who want to add AI features without tying their application architecture to one model vendor, one transport style, or one deployment strategy.

Why yLLMKit Exists

Many AI features start as a direct connection between an app and a single model provider. That works for early prototypes, but it can become limiting quickly. The app begins to depend on provider-specific request formats, streaming behavior, model identifiers, error handling, usage reporting, and configuration details.

yLLMKit separates the application-facing shape of an LLM feature from the provider-specific implementation underneath it.

Application code can work with shared Swift types for chat messages, model identifiers, generation settings, streamed output, responses, usage metadata, and provider errors. Provider products then handle the specific details of local MLX inference, OpenAI requests, Anthropic requests, or other model backends.

This makes it easier to prototype with a mock backend, ship a local-first experience, add a hosted fallback, compare providers, or change model strategy later without rewriting the parts of the app that prepare conversations and consume responses.

What yLLMKit Provides

yLLMKit is organized as a set of Swift package products. Developers can add only the pieces their app needs.

yLLMKit

The core package provides the shared types and protocols for provider-neutral text and chat features.

It includes model identifiers, provider identifiers, model descriptors, chat messages, chat requests, streaming events, generation settings, usage metadata, and normalized provider errors. The core package is UI-neutral and provider-neutral, so it can sit behind SwiftUI, AppKit, command line tools, or service layers without pulling in unnecessary provider dependencies.

yLLMKitMLX

The MLX product provides local Apple Silicon inference support.

It owns the local model catalog, model preparation, download behavior, loading, streaming, cancellation, and MLX-specific mapping needed to run supported models on the user’s machine. This keeps local inference first-class instead of treating it as an afterthought modeled around cloud APIs.

yLLMKitOpenAI

The OpenAI product provides an optional hosted provider integration.

It maps yLLMKit’s shared chat request and streaming interfaces to OpenAI-specific transport behavior, response formats, usage reporting, and errors. Apps can use OpenAI models without spreading provider-specific networking and JSON handling throughout the application layer.

yLLMKitAnthropic

The Anthropic product provides an optional hosted provider integration.

It allows apps to use Anthropic text and chat models through the same shared request and streaming interface used by the rest of yLLMKit. This is useful for apps that want provider choice, remote fallback, comparison testing, or hosted model support beside local inference.

yLLMKitContext

The context product helps applications prepare source-aware prompts.

It is intended for apps that need more than a simple chat history. yLLMKitContext can help with app-supplied text sources, conversation transcripts, token-aware chunking, hierarchical summaries, source references, prompt budgeting, and GRDB-backed persistence.

The app remains the owner of its data. yLLMKitContext helps prepare useful context for the model, but it does not replace the application’s source-of-truth database, document store, review workflow, or user interface.

Who It Is For

yLLMKit is for Swift developers building applications that need text or chat-based AI features.

It is a good fit for:

• macOS apps that want local LLM features.
• SwiftUI or AppKit apps that need a shared model interface.
• Developer tools that need streamed AI output.
• Apps that may support both local and hosted models.
• Projects that need model provider flexibility.
• Apps that need structured context instead of ad hoc prompt strings.
• Experiments where model strategy may change over time.

It is not trying to be a complete AI platform. The v1 scope is intentionally focused on text and chat. Vision, audio, image generation, agents, realtime APIs, tool calling, function calling, embeddings, file uploads, workflow orchestration, and UI components are outside the current scope.

Local-First by Design

Local-first does not mean local-only.

It means the project treats local inference as a primary design path rather than a special case. Local model lifecycle, preparation progress, cache behavior, unloading, removal, and cancellation all need to be part of the developer experience.

Hosted providers are useful, and yLLMKit includes optional integrations for them. But the core architecture does not assume that hosted APIs define the shape of every AI feature. Local models have their own needs, and yLLMKit keeps those needs visible.

That approach gives app developers more control over privacy, offline capability, latency, cost, deployment, and long-term model strategy.

Core Features

yLLMKit currently focuses on a practical set of developer-facing capabilities:

• Provider-neutral chat request and response types.
• Shared model and provider identifiers.
• Streaming text responses through Swift concurrency.
• Generation settings that can be reused across providers.
• Normalized provider errors.
• Usage metadata where providers supply it.
• Optional local inference support through MLX.
• Optional hosted provider support.
• Local model preparation and lifecycle behavior.
• Context preparation for source-aware conversations.
• UI-neutral design for use in different kinds of Swift apps.

The goal is not to hide every provider difference. Some provider-specific behavior still belongs in the provider product or the app that chooses to use it. The goal is to give application code a stable, understandable layer to build against.

Example Use Cases

A writing app could use yLLMKit to offer local drafting assistance, then allow the user to switch to a hosted provider for more demanding work.

A research app could use yLLMKitContext to prepare source-aware prompts from notes, transcripts, or imported documents.

A developer tool could stream local model output into a command line or macOS interface without tying its application logic directly to MLX.

A private productivity app could use local inference by default, while keeping remote providers available as an explicit option.

A prototype could begin with one backend and later move to another without replacing the app’s conversation model, response handling, or settings interface.

Current Status

yLLMKit is currently pre-beta.

The project is under active development, and the public API should not be treated as stable yet. The current development focus is the text and chat foundation, local-first behavior, provider-neutral request and streaming types, optional provider products, and context support.

Developers interested in the project should expect change as the package moves toward a clearer beta contract.

Related Project

A companion demo project, yLLMKit Demo, shows how the package can be used from a small macOS SwiftUI chat application. The demo focuses on supported local models, model download progress, loading, model management, chat history, and streamed assistant responses.

Repository

The project is available on GitHub:

https://github.com/BrandonYaniz/yLLMKit