gtbluesky/onnxruntime_flutter

https://github.com/microsoft/onnxruntime/raw/main/docs/images/ONNX_Runtime_logo_dark.png

OnnxRuntime Plugin

pub package

Overview

Flutter plugin for OnnxRuntime via dart:ffi provides an easy, flexible, and fast Dart API to integrate Onnx models in flutter apps across mobile and desktop platforms.

Platform	Android	iOS	Linux	macOS	Windows
Compatibility	API level 21+	*	*	*	*
Architecture	arm32/arm64	*	*	*	*

*: Consistent with Flutter

Key Features

• Multi-platform Support for Android, iOS, Linux, macOS, Windows, and Web(Coming soon).
• Flexibility to use any Onnx Model.
• Acceleration using multi-threading.
• Similar structure as OnnxRuntime Java and C# API.
• Inference speed is not slower than native Android/iOS Apps built using the Java/Objective-C API.
• Run inference in different isolates to prevent jank in UI thread.

Getting Started

In your flutter project add the dependency:

dependencies:
  ...
  onnxruntime: x.y.z

Usage example

Import

import 'package:onnxruntime/onnxruntime.dart';

Initializing environment

OrtEnv.instance.init();

Creating the Session

final sessionOptions = OrtSessionOptions();
const assetFileName = 'assets/models/test.onnx';
final rawAssetFile = await rootBundle.load(assetFileName);
final bytes = rawAssetFile.buffer.asUint8List();
final session = OrtSession.fromBuffer(bytes, sessionOptions!);

Performing inference

final shape = [1, 2, 3];
final inputOrt = OrtValueTensor.createTensorWithDataList(data, shape);
final inputs = {'input': inputOrt};
final runOptions = OrtRunOptions();
final outputs = await _session?.runAsync(runOptions, inputs);
inputOrt.release();
runOptions.release();
outputs?.forEach((element) {
  element?.release();
});

Releasing environment

OrtEnv.instance.release();