LocalLlmDatasource class
On-device LLM inference via llama_cpp_dart, which wraps a static-linked
llama.cpp xcframework on iOS (resolved through DynamicLibrary.process()
in LlamaEngine.spawnFromProcess) and an .aar-bundled libllama.so on
Android (loaded by name via LlamaEngine.spawn).
Replaces the previous flutter_gemma (MediaPipe LiteRT-LM) integration
that crashed on first inference on iPhone 15 under iOS 26.4.2 — see
plans/spike-local-llm.md "Findings — 2026-05-19".
Model files are GGUF Q4_K_M, downloaded from Hugging Face mirrors into
getApplicationSupportDirectory()/llama_models/<filename> on first use.
Downloads stream to a .partial file and rename on success; optional
SHA-256 verification (when _LlamaSpec.sha256 is non-null) catches
corruption before the first inference attempt.
- Implemented types
Constructors
- LocalLlmDatasource({Dio? dio})
Properties
- activeModelId → String?
-
no setter
- hashCode → int
-
The hash code for this object.
no setterinherited
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
Methods
-
deleteModel(
String modelId) → Future< void> -
Uninstall the GGUF file. No-op if not installed. If
modelIdis the currently-loaded engine's model, the engine is disposed too. -
dispose(
) → Future< void> -
downloadModel(
String modelId, {bool allowMetered = false, void onProgress(double progress)?}) → Future< void> -
Download + install the model. Idempotent: skips if the file already
exists and (when applicable) the SHA-256 verifies. Reports progress
0..1 via
onProgresswhen the server returns aContent-Length. -
isConfigured(
) → Future< bool> -
Whether the provider is configured (API key stored or OAuth signed in).
override
-
isModelDownloaded(
String modelId) → Future< bool> -
Whether the GGUF file backing
modelIdis installed locally. Verifies existence + (when populated in_LlamaSpec.sha256) integrity. -
listModels(
) → Future< List< AiModel> > -
Fetch available models from the provider's API.
override
-
modelFileSizeBytes(
String modelId) → Future< int?> -
File size in bytes if present, else
null. -
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
sendMessage(
{required String systemPrompt, required List< Map< messages, int maxTokens = 2048}) → Future<String, String> >String> -
Send messages and return the complete assistant response.
override
-
setActiveModel(
String modelId) → Future< void> -
Mark
modelIdas the active local model. Disposes any previously loaded engine when the active model changes. -
streamMessage(
{required String systemPrompt, required List< Map< messages, int maxTokens = 2048}) → Stream<String, String> >String> -
Stream a response, yielding text deltas as they arrive.
override
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited