Search packages, repositories and app categories

GitHub repositorygtbluesky/onnxruntime_flutter

onnxruntime_flutter

A flutter plugin for OnnxRuntime provides an easy, flexible, and fast Dart API to integrate Onnx models in flutter apps across mobile and desktop platforms.

onnxruntime_flutter project cover
Stars
130
Forks
73
Last push (UTC)
Dec 23, 2024
Project status
Active

Original README

English project snapshot. Visit GitHub for the latest content.

OnnxRuntime Plugin

pub package

Overview

Flutter plugin for OnnxRuntime via dart:ffi provides an easy, flexible, and fast Dart API to integrate Onnx models in flutter apps across mobile and desktop platforms.

Platform Android iOS Linux macOS Windows
Compatibility API level 21+ * * * *
Architecture arm32/arm64 * * * *

*: Consistent with Flutter

Key Features

  • Multi-platform Support for Android, iOS, Linux, macOS, Windows, and Web(Coming soon).
  • Flexibility to use any Onnx Model.
  • Acceleration using multi-threading.
  • Similar structure as OnnxRuntime Java and C# API.
  • Inference speed is not slower than native Android/iOS Apps built using the Java/Objective-C API.
  • Run inference in different isolates to prevent jank in UI thread.

Getting Started

In your flutter project add the dependency:

dependencies:
  ...
  onnxruntime: x.y.z

Usage example

Import
import 'package:onnxruntime/onnxruntime.dart';
Initializing environment
OrtEnv.instance.init();
Creating the Session
final sessionOptions = OrtSessionOptions();
const assetFileName = 'assets/models/test.onnx';
final rawAssetFile = await rootBundle.load(assetFileName);
final bytes = rawAssetFile.buffer.asUint8List();
final session = OrtSession.fromBuffer(bytes, sessionOptions!);
Performing inference
final shape = [1, 2, 3];
final inputOrt = OrtValueTensor.createTensorWithDataList(data, shape);
final inputs = {'input': inputOrt};
final runOptions = OrtRunOptions();
final outputs = await _session?.runAsync(runOptions, inputs);
inputOrt.release();
runOptions.release();
outputs?.forEach((element) {
  element?.release();
});
Releasing environment
OrtEnv.instance.release();
gtbluesky GitHub avatar
GitHub author

gtbluesky ↗

Interesting in Android、iOS、HarmonyOS、Flutter、AI...

NanJing, China
LanguagesC++DartCMakeCRubySwiftKotlinMakefileObjective-C

Categories

Packages published by this repository

Dependencies used

Dependency list 5 items

  • flutter{"sdk":"flutter"}
  • ffi^2.0.1
  • ffigenDevelopment^7.2.11
  • flutter_testDevelopment{"sdk":"flutter"}
  • flutter_lintsDevelopment^2.0.0

How do I choose a repository?

Start with your use case, compare repository activity, and check the original project before adopting it.

  1. Search by repository name, owner or description, then narrow the results by topic.
  2. Compare stars, forks and recent pushes. Stars show community interest, not a guarantee of quality.
  3. Read the README, license and issues on GitHub before using a project.

Sources and maintenance

“A repository is the most basic element of GitHub.”

GitHub Docs — About repositories ↗