Search packages, repositories and app categories

Package detailsv1.8.6

flutter_gemma_litertlm

LiteRT-LM (.litertlm) on-device inference engine for flutter_gemma via dart:ffi (5 native platforms) + web. Opt-in InferenceEngineProvider. Also ships the LiteRT C API embedding backend (LiteRtEmbeddi [...]

AndroidiOSLinuxmacOSWebWindows
30-day downloads
29.7K
Likes
0
Pub points
130

Install package

flutter pub add flutter_gemma_litertlm

Package Overview

Package selection guidance compiled from public package information, community data, and project usage.

A lightweight, on-device inference engine for FlutterGemma using Dart:ffi, supporting five native platforms and web. It enables efficient local LLM inference via the LiteRT C API embedding backend with opt-in provider integration. Designed for low-latency, privacy-focused generative AI applications directly on devices without relying on cloud services. The package simplifies deployment across Android, iOS, Linux, macOS, Windows, and web environments.

Use cases

  • On-device language model inference
  • Privacy-preserving chat applications
  • Offline AI-powered apps
  • Edge computing for LLMs
  • Lightweight LLM deployment on mobile and desktop

Key features

  • Native platform support (5+ platforms)
  • Web compatibility via Dart:ffi
  • Opt-in InferenceEngineProvider
  • LiteRT C API embedded backend
  • Minimal runtime overhead
  • Cross-platform consistency

Suitable for

  • Developers building offline LLM apps
  • Apps requiring data privacy
  • Mobile and desktop AI prototypes
  • Edge AI projects with limited resources
  • Teams using Flutter for generative AI

Considerations

  • Requires native code compilation
  • Limited to models compatible with LiteRT
  • Web support depends on WASM availability
  • Performance varies by device hardware
  • No built-in model management or caching

Original README

Original README snapshot from the linked source repository. Visit the repository for the latest content.

Flutter Edge AI

pub package License: MIT

On-device AI for Flutter on Android, iOS, Web, macOS, Windows and Linux. Run any model a registered engine runs, or the model the operating system already ships, with text, vision, audio, function calling, embeddings, RAG and speech. No server, no cloud.

Website · Docs · Models · Codelabs · pub.dev

The core package is small: it installs models and hands them to engines. Every runtime, store and feature is a separate opt-in package, so an app ships only what it uses. Start with the core README or Getting started.

Packages

Package What it does Docs
flutter_edge_ai Core: model installation, the engine registry, sessions and chat. Required by every package except diagnostics and genkit_hybrid. pub.dev · site
Engines
flutter_edge_ai_litertlm LiteRT-LM (.litertlm) through dart:ffi on the five native platforms plus web; CPU, GPU and NPU; the LiteRT embedding backend. pub.dev · site
flutter_edge_ai_mediapipe MediaPipe (.task) on Android, iOS and web. Google's LLM Inference API is maintenance-only; new apps should use .litertlm. pub.dev · site
flutter_edge_ai_llama llama.cpp (.gguf) through llamadart on Android, iOS, macOS, Windows, Linux and web; native LoRA. pub.dev · site
flutter_edge_ai_onnx ONNX Runtime: GenAI inference and embeddings natively, Transformers.js and onnxruntime-web on web. pub.dev · site
flutter_edge_ai_builtin_ai The OS model, nothing to download: Gemini Nano (Android), Apple Foundation Models (iOS, macOS), Windows AI Foundry, the browser Prompt API. pub.dev · site
Embeddings and RAG
flutter_edge_ai_embeddings Embedding tokenizers (Gemma SentencePiece, BERT WordPiece) for .tflite and .onnx embedders. pub.dev · site
flutter_edge_ai_rag On-device RAG: RagIndex, embedding profiles, filters and the vector-store contract. pub.dev · site
flutter_edge_ai_sqlite SQLite (sqlite-vec) vector store on every platform, web included. pub.dev · site
flutter_edge_ai_qdrant Qdrant Edge vector store on native platforms. pub.dev · site
Speech, agents, diagnostics
flutter_edge_ai_speech Speech-to-text, text-to-speech and the VoiceSession voice loop. pub.dev · site
flutter_edge_ai_agent Agent skills (SKILL.md) the model picks and runs: text, JS, native intents and MCP tools. pub.dev · site
flutter_edge_ai_diagnostics Process memory diagnostics on Android, iOS, macOS, Linux and Windows. pub.dev · site
Genkit
genkit_flutter_edge_ai Genkit Dart plugin: on-device models and embedders as Genkit models. pub.dev · site
genkit_hybrid Hybrid routing for Genkit: on-device and cloud models behind one routing policy. Depends only on genkit. pub.dev · site

Which package you need for which task, and how they fit together: Packages and Capabilities.

Maintainers

Maintainer Looks after
Sasha Denisov (@DenisovAV) Lead maintainer: the core API, the engines, the native builds and every release
Jhin Lee (@leehack) flutter_edge_ai_llama
Francesco Vezzani (@kekko7072) flutter_edge_ai_builtin_ai
Samarthsinh Vala (@samarthsinh2660) flutter_edge_ai_diagnostics
Daisuke Majima (@john-rocky) The Hugging Face integration: installing a model straight from its repository

Everyone else who has contributed, and more about each maintainer: flutteredge.ai/contributors.

Repository

More: Migration guide · Desktop support · Contributing. The family shipped as flutter_gemma* up to core 1.11.4; those names stay published as they are.

License

MIT — see LICENSE.

Categories

Dependencies

Dependency list 7 items

Source repository

Projects using this package

No data available yet.

Rankings