classify_card

On-device document handling for Flutter, in three independent pieces:

	What you get
Card Gate V3	Given a photo, decides whether an identity document is the main subject — so you only spend a network round trip on photos that actually contain one.
Auto-crop	A photo goes in, a perspective-corrected photo comes out. No UI, no user interaction, on every supported device.

Use any one of them alone. They share no state, no platform interface and no thread, so the gate works without the cropper and the cropper works without the model.

The gate and the cropper run entirely on the device. Nothing is uploaded, and no network permission is needed.

What it does and does not do

Does	Does not
Answers `card` or `other` for a photo	OCR, read fields, or return text
Returns a calibrated confidence score	Identify the document subtype (NRC / passport / licence)
Runs offline in ~20 ms warm	Liveness, face matching, or anti-fraud
Crops a document out of a photo automatically	Guarantee a crop — see Automatic cropping

It is a pre-check and a cropper, not a replacement for your classify/OCR backend.

Requirements

	Minimum	Why
Android	`minSdkVersion` 21	LiteRT needs API 21.
iOS	12.0	Flutter 3.x minimum; the pod is set to match.
App size	+3.8 MB classifier, +4.5 MB corner model, +~14 MB per ABI ONNX Runtime	Both models ship inside the plugin and run offline.
Android ABI	arm only for `DocumentCropEngine.builtin`	ONNX Runtime publishes no x86/x86_64 build, so the no-UI crop does not run on an x86 emulator. Everything else does.

Still on API 21–22 or iOS 12? Pin classify_card: ^1.0.0. It has the classifier and the realtime scanner view, and neither needed the higher floors. Version 2.0.0 is a major bump for these two lines alone — every existing call still compiles unchanged.

On iOS, Info.plist needs NSCameraUsageDescription for CardGateScannerView.

You do not need to add noCompress 'tflite', register Xcode resources, or ship the model yourself — the plugin carries its own model and loads it in a way that does not depend on host-app packaging settings.

Install

dependencies:
  classify_card: ^2.0.0

One host-app change is required.

Android — android/app/build.gradle:

android {
  defaultConfig {
    minSdkVersion 21   // was flutter.minSdkVersion / 19
  }
}

iOS — nothing. The pod declares static_framework = true, so it installs under a plain use_frameworks! alongside other TFLite-based pods.

Only if you use the scanner (CardGateScannerView), add the camera purpose string to ios/Runner/Info.plist:

<key>NSCameraUsageDescription</key>
<string>The camera is used to scan your identity document.</string>

Only if your app gets camera from a git fork — ekyc_flutter_sdk does — pub cannot reconcile a git source with a hosted one. Point both at the same fork:

dependency_overrides:
  camera:
    git:
      url: https://github.com/Techainer/flutter-camera-package.git
      ref: lucas_handle_resolution_prest
      path: packages/camera/camera

Then rebuild the app — this plugin contains native code, so a hot restart is not enough.

Quick start

import 'package:classify_card/classify_card.dart';

final gate = ClassifyCard();

// Once, e.g. in initState of your capture screen. Never throws.
await gate.warmUp();

// After the user captures or picks a photo.
final decision = await gate.evaluate(photoPath, mode: CardGateMode.enforce);

if (!decision.allow) {
  showRetakeGuidance();   // no backend call
  return;
}

await myBackend.classify(photoPath);   // your existing flow, unchanged

photoPath should be the whole photo, upright. Do not crop it to the card first — the model was trained on full frames. A file:// prefix is fine, as is a bare path; Android also accepts a content:// URI. EXIF rotation is handled for you.

Realtime scanner

CardGateScannerView is a camera screen that photographs a card once it has been held still inside the guide. There is no shutter. The user brings the card into the guide, holds it steady for 1.2 seconds, and you get the photo through a callback.

Navigator.of(context).push(MaterialPageRoute(
  builder: (context) => CardGateScannerView(
    onCaptured: (capture) async {
      Navigator.of(context).pop();

      // capture.path is the full-frame JPEG. What happens next is entirely yours.
      final decision = await ClassifyCard()
          .evaluate(capture.path, mode: CardGateMode.enforce);
      if (decision.allow) await myBackend.classify(capture.path);
    },
    onCancel: () => Navigator.of(context).pop(),   // the back arrow, top-left
  ),
));

What has to be true before it fires

Three things at once, on every scored frame:

the classifier says card;
all four corners sit inside the guide and fill at least minFillRatio of it;
no corner has moved more than maxDrift since the countdown began.

Any of them lapsing puts the countdown back to zero. A ring traces the guide as it runs, so holding still visibly does something.

Requiring stillness is not decoration: a classifier that says card while the phone is being waved around will happily produce a blurred photo, and a blurred card is a failed OCR call rather than a saved one. Drift is measured against where the countdown started, not the previous frame - a slow creep passes every frame-to-frame check and still ends up somewhere else two seconds later.

Useful knobs: holdDuration (default 1.2 s), minFillRatio (0.55) and maxDrift (0.02) to tune how fussy it is, minFrameInterval (250 ms) to trade responsiveness for battery, overlay to restyle the guide, and the six hint strings.

The photo is the frame that was checked

There is no takePicture. When the countdown completes, the frame that completed it is encoded to a JPEG - upright, full resolution, rotation baked into the pixels rather than left as an EXIF tag.

Two reasons. A still on Android stops the stream, runs a fresh autofocus and exposure convergence, and commonly takes several hundred milliseconds - a dead pause at the exact moment a hands-free scanner should feel instant. And a still photographs the scene at the moment the shutter fires, which is not the scene that was checked: between the countdown completing and the still arriving, the card can move and the focus can hunt. Encoding the scored frame means the photograph you get is the one the corners were measured on.

The cost is resolution: the stream runs at the preview resolution, not the sensor's full one. resolutionPreset therefore sets the photo size too, and defaults to veryHigh (1080p on most devices) rather than the high a preview alone would need. Raise it if OCR wants more pixels; lower it if frame scoring is too slow on the hardware you support.

It does not crop

capture.path is the photo exactly as taken - the full frame. That is what evaluate needs, and cropping is a separate feature with its own API, so the screen does not quietly do both. Want a crop? Ask for one:

final crop = await DocumentCropper().crop(capture.path);
final toShow = crop.cropped ? crop.path : capture.path;

When there is no corner detector

The bundled detector runs on ONNX Runtime, which ships arm-only, so it does not load on an x86 Android emulator. Rather than never firing there, the screen notices corners have never arrived and falls back to holding on the classifier alone, reporting capture.usedGeometry == false.

Once corners have arrived even once, they are required from then on: their absence then means no card in the guide, not a missing feature.

Swapping the detector

cornerSource takes any CardGateCornerSource - one method, returning four corners in upright frame pixels or null. The widget, the countdown and the geometry are all detector-agnostic, so a different model drops in without touching them.

class MyDetector implements CardGateCornerSource {
  @override
  Future<List<Offset>?> detectFrame(CardGateFrame frame) async { ... }
}

CardGateScannerView(onCaptured: ..., cornerSource: MyDetector());

Auto or manual, the user's choice

A switch at the bottom lets the user pick. Auto is the hands-free countdown above. Manual gives them a shutter and no countdown - for a card the detector cannot see (bad light, an unusual document, a scratched laminate) they can reach for it the moment they want it, rather than waiting for a button to appear on its own.

Manual mode does not run the corner model at all: nothing is deciding when to fire, so the guide only has to recolour. On a slow phone that is most of this screen's cost.

capture.mode says which one took the photograph. showCaptureModeToggle: false locks the screen to initialCaptureMode; autoModeLabel and manualModeLabel are the two labels.

Three things worth knowing

A frame is a hint, not a verdict. capture.liveResult is the last frame the model scored. Do not act on it. capture.path is what you run evaluate on - that goes through the still-photo path the model was calibrated against.

The model sees the whole frame, not the guide. The guide helps the user aim; cropping to it would break the full-frame assumption the model was trained on, so the plugin does not.

Dart never touches a pixel. The plane bytes go straight from the camera to native code, which does the colour conversion, the rotation and the resize. Nothing is encoded to JPEG or PNG and nothing is written to disk, which is what makes several checks a second affordable.

Frames that arrive while one is still being scored are dropped, not queued - queueing at frame rate grows without bound and ends in an out-of-memory kill. The scanner ignores those silently.

If you want the live scoring without the UI, call classifyFrame yourself:

controller.startImageStream((image) async {
  if (busy) return;
  busy = true;
  try {
    final result = await gate.classifyFrame(
      CardGateFrame.fromCameraImage(image, rotationDegrees: description.sensorOrientation),
    );
    setState(() => status = result.label);
  } on CardGateException catch (e) {
    if (e.code != CardGateException.codeBusy) status = null;  // a dropped frame is normal
  } finally {
    busy = false;
  }
});

Cropping

// No UI: a file goes in, a cropped file comes out.
final result = await DocumentCropper().crop(photoPath);
await myBackend.upload(result.path);   // always a usable path

It runs DocAligner's lcnet100_h_e_bifpn_256 corner model (Apache-2.0, bundled at assets/document_corners_lcnet100.onnx, 4.5 MB) through ONNX Runtime, and warps the photo to the four corners it returns. It has nothing to do with ML Kit.

final result = await DocumentCropper().crop(path);

switch (result.reason) {
  case DocumentCropReason.cropped:           // result.path is a new file
  case DocumentCropReason.alreadyFullFrame:  // the photo was already just the document
  case DocumentCropReason.noDocument:        // nothing found; result.path is your original
  case DocumentCropReason.unavailable:
  case DocumentCropReason.timeout:
  case DocumentCropReason.failed:
}

result.path is always safe to use. When nothing was cropped it is the path you passed in, byte-for-byte unchanged, and result.cropped is false. crop never throws. That asymmetry is deliberate: a wrong crop destroys information permanently, a missing crop only fails to add any.

result.detector says which engine produced the corners — model, builtin (the geometric fallback, used only when the model cannot load), vision, or systemScanner.

How good it is. Measured against ground-truth corners on 24 real ID-card photographs plus 6 taken on a phone: median IoU 0.958, one photograph off by more than 30 px, and nothing cropped on the six photographs that contain no document at all. About 30 ms on a mid-range phone.

For comparison, the geometric detector this package shipped before shows median IoU 0.872, four photographs off by more than 30 px, and found nothing at all in any of the six phone photographs the model handles. It is still in the package, but only as the fallback for when the model cannot load.

What it costs. ONNX Runtime adds roughly 14 MB of native library per ABI, and the model 4.5 MB. ONNX Runtime ships arm only — there is no x86/x86_64 build — so this path does not run on an x86 Android emulator. Use an arm emulator or a real device.

Why this package carries its own corner model

Because it cannot. The entire public surface of the ML Kit document scanner is:

GmsDocumentScanning.getClient(options)           // -> GmsDocumentScanner
GmsDocumentScanner.getStartScanIntent(Activity)  // -> Task<IntentSender>
GmsDocumentScanningResult.fromActivityResultIntent(Intent)

One method, which takes an Activity and returns an IntentSender. There is no overload accepting a Bitmap, Uri, File or InputImage, so it can only ever open its own camera. Apple's VNDocumentCameraViewController is a view controller, with the same consequence.

"Photo in, cropped photo out" is a different problem, and neither platform ships an answer to it that takes a file. Verified three ways against the 16.0.0 AAR: no method accepts an image, there is no input extra, and the only IPC surface is com.google.mlkit.vision.docscan.**ui**.aidls.* — the model and the UI both live inside Play Services, reachable only by launching that UI.

So this package carries its own model instead: DocAligner's corner network, Apache-2.0, bundled here and run through ONNX Runtime. Nothing in this package depends on ML Kit or VisionKit.

The two rules that matter

1. It fails open

decision.allow is false for exactly one reason: CardGateReason.other, meaning the model is confident there is no document. Every other outcome lets the request through — a missing plugin, a corrupt model, a decode error, a timeout. A broken classifier must never block a customer holding a valid card.

`reason`	`allow`	Meaning
`card`	✅	Document found
`other`	❌	No document — the only blocking case
`shadow`	✅	A result was produced but `mode` was `shadow`
`disabled`	✅	`mode` was `off`
`unavailable`	✅	Plugin absent from the binary, or the classifier could not be built
`timeout`	✅	Native call did not settle
`failed`	✅	Decode or inference failed

reason.isFailure is true for the last three. Log those — a spike means something is broken.

2. Start in shadow, not enforce

mode defaults to CardGateMode.shadow: the model runs and reports, but never blocks. Ship that first, log the results next to what your backend decides, and only switch to CardGateMode.enforce once the two agree on your own traffic.

final decision = await gate.evaluate(path);         // shadow — cannot reject anyone
final r = decision.result;
if (r != null) {
  analytics.log('card_gate', r.toMap());            // safe to log: scores only, never the image
}

API

Three classes, three independent features. None of them touches the others.

`ClassifyCard` — the gate

Future<String?> warmUp();
Future<CardGateResult> classify(String imagePath, {Duration timeout});
Future<CardGateDecision> evaluate(String imagePath, {CardGateMode mode, Duration timeout});
Future<CardGateResult> classifyFrame(CardGateFrame frame, {Duration timeout});
Future<void> release();

Method	Throws?	Use when
`warmUp()`	No — returns `null` on failure	On screen mount, so the first photo is not charged the cold-start cost
`evaluate()`	No	Normal use. Handles fail-open for you
`classify()`	`CardGateException`	You want to own the error handling yourself
`release()`	No	Capture flow is done and you want the ~4 MB back

CardGateResult carries label, score (calibrated), rawScore (the model's own output), modelVersion, latencyMs and decodeMs. toMap() is safe to log — it never contains image data.

Do not re-threshold rawScore yourself. 0.5 is not the production operating point. score is the calibrated value that label was derived from, using the Platt transform and threshold in the bundled policy.

`DocumentCropper` — photo in, cropped photo out

Future<DocumentCropResult> crop(
  String imagePath, {
  int quality = 95,
  Duration timeout = const Duration(seconds: 8),
  bool preferPlatformDetector = false,
});
Future<void> clearCache();

Method	Throws?	Notes
`crop()`	Never	Returns the original path when nothing was cropped. Needs no `warmUp` and does not load the classifier.
`clearCache()`	No	Deletes only the files this package wrote

DocumentCropResult carries path, cropped, reason, detector, width, height, latencyMs, engineNote (why the requested engine did not run, when it did not), and corners — the four detected points as fractions of the source photo, ordered top-left, top-right, bottom-right, bottom-left. toMap() is safe to log.

preferPlatformDetector lets Apple's VNDetectDocumentSegmentationRequest try first on iOS 15+. It is off by default so both platforms run the same model and agree on the same photo.

Logging

The plugin deliberately does no logging of its own. It hands you the data; you decide what goes to your own telemetry. Nothing it returns contains image bytes.

How each platform runs the model

	Android	iOS
Runtime	`com.google.ai.edge.litert:litert` + `-support` + `-metadata` 1.4.0	`TensorFlowLiteSwift ~> 2.12.0`
Preprocessing	Hand-written	Hand-written
Matches the vendor reference exactly	Equivalent, not bit-identical	Equivalent, not bit-identical

Both platforms run the same pipeline: EXIF-upright, bilinear resize to 224x224 with half-pixel centres, the metadata normalization, then the input tensor's own quantization. The arithmetic is identical on both, so a photo should score the same either side.

Neither uses the TFLite Task Library, which is what the vendor reference is written against. org.tensorflow:tensorflow-lite-task-vision was never ported to LiteRT — there is no litert-task-vision — and shipping it alongside LiteRT puts two TFLite runtimes in one APK. On iOS TensorFlowLiteTaskVision is unusable in a host app for three separate reasons:

its 0.4.3 static framework exports the same TfLite* symbols as TensorFlowLiteC, so it collides with any app that already ships TFLite;
it redefines MLKit's GMLImage class, so it collides with any app using MLKit;
it has no arm64 simulator slice, so it cannot build for the iOS Simulator on Apple Silicon;
it has not been released since November 2022.

The consequence you should know about: the preprocessing is a faithful reimplementation, not the vendor's own code, so rawScore can differ slightly from the reference implementation. Measure it on your own images before enforcing — this is the main reason to run shadow first.

Verifying it before you enforce

Run the example/ app, set the mode selector to shadow.
Feed it ~10 real documents (including rotated 90° / 180° / 270°) and ~10 non-documents — portraits, landscapes, junk.
Check the label is right in every case, and compare rawScore between an Android device and an iPhone on the same photos.
Watch latencyMs. Target is p95 under 100 ms after warm-up.
Only then switch to enforce.

Integrity

The model is verified on every startup: its SHA-256 must equal metadata.model_sha256 in the bundled policy.json, and the policy's schema_version, score_transform.kind, scale and threshold are all range-checked. A mismatch makes the classifier unavailable — which fails open, it does not silently reject photos. The bundled model hashes to 361bbfc4e540e717d6a961ed119f47719416241ed939b2e0e90024ca14dd5855.

Model tensors, for reference: input input_1 UINT8 [1,224,224,3] scale 0.003921568859368563 zero-point 0; output Identity UINT8 [1,2] scale 0.00390625 zero-point 0.

Troubleshooting

reason is always unavailable

Either the plugin is not in the binary — do a full rebuild, not a hot restart — or the native setup failed. Check the device log for card_gate_unavailable; the message names the exact cause (missing resource, SHA-256 mismatch, bad policy field).

Android build fails on minSdkVersion

Raise minSdkVersion to at least 21 in your app's android/app/build.gradle.

CocoaPods cannot resolve TensorFlowLiteSwift

This pod asks for ~> 2.12.0 (>= 2.12.0, < 2.13.0), matching ekyc_flutter_sdk and tflite_flutter. If another pod in your app needs a different 2.x, CocoaPods cannot install both — one version of a pod is all it will take. Align the constraints, then re-run the verification steps above.

Note that changing a development pod's constraint is not picked up by pod install; run pod update TensorFlowLiteSwift to make CocoaPods re-resolve it.

transitive dependencies that include statically linked binaries

You should not see this: the podspec declares static_framework = true, so a plain use_frameworks! is enough. If it appears anyway, another pod depending on TensorFlowLiteSwift is missing that declaration.

Duplicate symbol errors mentioning TfLite

Something else in your app links its own TFLite copy — most often TensorFlowLiteTaskVision, MediaPipeTasksVision, or a vendored .framework. Only one TFLite runtime can be linked; consolidate on TensorFlowLiteSwift.

Android: duplicate libtensorflowlite_jni.so

Another dependency pulls org.tensorflow:tensorflow-lite* while this plugin uses LiteRT. They are different Maven coordinates, so Gradle keeps both. Move the other dependency to com.google.ai.edge.litert:*.

iOS Simulator on an Apple Silicon Mac

Works. TensorFlowLiteSwift ships a proper xcframework with an arm64 simulator slice — this is one of the reasons Task Vision was rejected.

License

See LICENSE. The bundled model and policy are covered by your agreement with the model provider, not by this package's license.

classify_card

What it does and does not do

Requirements

Install

Quick start

Realtime scanner

What has to be true before it fires

The photo is the frame that was checked

It does not crop

When there is no corner detector

Swapping the detector

Auto or manual, the user's choice

Three things worth knowing

Cropping

Why this package carries its own corner model

The two rules that matter

1. It fails open

2. Start in shadow, not enforce

API

`ClassifyCard` — the gate

`DocumentCropper` — photo in, cropped photo out

Logging

How each platform runs the model

Verifying it before you enforce

Integrity

Troubleshooting

License

Libraries

classify_card package

classify_card

What it does and does not do

Requirements

Install

Quick start

Realtime scanner

What has to be true before it fires

The photo is the frame that was checked

It does not crop

When there is no corner detector

Swapping the detector

Auto or manual, the user's choice

Three things worth knowing

Cropping

Why this package carries its own corner model

The two rules that matter

1. It fails open

2. Start in shadow, not enforce

API

ClassifyCard — the gate

DocumentCropper — photo in, cropped photo out

Logging

How each platform runs the model

Verifying it before you enforce

Integrity

Troubleshooting

License

Libraries

classify_card package

`ClassifyCard` — the gate

`DocumentCropper` — photo in, cropped photo out