Google ML Kit for Flutter: Complete Guide to Barcode Scanning, Face Detection & Text Recognition (2026)
Machine learning has moved from research labs into the pockets of billions of users. Yet, integrating ML into a mobile app used to mean training models, converting them to TFLite, managing input tensors, and handling edge cases across iOS and Android. Google ML Kit changes the equation dramatically – giving Flutter developers production-ready, on-device ML APIs that just work.
In this guide you will learn how to integrate three of ML Kit’s most practical capabilities – barcode scanning, face detection, and text recognition (OCR) – into a Flutter app. Every code snippet is real, runnable Dart. By the end you will have a solid understanding of the APIs, performance patterns, and architectural decisions that make ML Kit a first-class tool in your Flutter toolkit.
What is Google ML Kit & Why Use It?
Google ML Kit is a mobile SDK that bundles a suite of on-device machine learning features. Released as part of Google’s Firebase ecosystem (and now also available standalone through the google_mlkit_* pub.dev packages), it exposes pre-trained, highly optimised models that run entirely on the device – no internet connection required, no server round-trips, no user data leaving the phone.
On-Device Inference: Why It Matters
Processing data locally means:
- Privacy by default – camera frames never leave the device.
- Low latency – inference happens in milliseconds, not seconds.
- Offline-first – ML features work in aeroplanes, tunnels, and poor-coverage areas.
- Cost savings – no cloud Vision API bills that scale with usage.
ML Kit vs. Custom TFLite Models
When should you reach for ML Kit instead of a custom TFLite model?
| Factor | ML Kit | Custom TFLite |
|---|---|---|
| Training data required | None | Thousands of labelled samples |
| Domain coverage | Common tasks (text, faces, barcodes.) | Anything – including niche domains |
| Maintenance | Google-maintained | You maintain it |
| Model size | Bundled or streamed by Google Play | Bundled in APK/IPA |
| Cross-platform | iOS + Android out of the box | Requires platform channels for full parity |
The rule of thumb: if your task is a commodity ML problem – reading text, detecting faces, decoding barcodes – ML Kit saves you weeks of work. Reserve custom models for domain-specific needs where no pre-trained solution exists.
Key ML Kit Features Available in Flutter
- Barcode Scanning
- Face Detection & Mesh
- Text Recognition v2
- Image Labelling
- Object Detection & Tracking
- Pose Detection
- Language ID & Translation
- Smart Reply
Barcode Scanning Setup & Implementation
Real-time barcode scanning is one of the most requested features in retail, logistics, and ticketing apps. ML Kit’s barcode scanner supports QR codes, EAN-13, UPC-A, Code 128, PDF417, Data Matrix, and many more formats – all decoded on-device at camera speed.
Dependencies
# pubspec.yaml
dependencies:
flutter:
sdk: flutter
google_mlkit_barcode_scanning: ^0.12.0
camera: ^0.11.0
permission_handler: ^11.3.0
Run flutter pub get and add camera permission entries to your platform manifests:
- Android (
AndroidManifest.xml):<uses-permission android:name="android.permission.CAMERA"/> - iOS (
Info.plist):NSCameraUsageDescriptionkey with a human-readable string.
Full Barcode Scanner Implementation
import 'package:camera/camera.dart';
import 'package:flutter/material.dart';
import 'package:google_mlkit_barcode_scanning/google_mlkit_barcode_scanning.dart';
class BarcodeScannerScreen extends StatefulWidget {
const BarcodeScannerScreen({super.key});
@override
State<BarcodeScannerScreen> createState() => _BarcodeScannerScreenState();
}
class _BarcodeScannerScreenState extends State<BarcodeScannerScreen> {
late CameraController _cameraController;
late BarcodeScanner _barcodeScanner;
bool _isProcessing = false;
String _scannedValue = 'Point camera at a barcode';
bool _isCameraReady = false;
@override
void initState() {
super.initState();
_barcodeScanner = BarcodeScanner(
formats: [BarcodeFormat.all],
);
_initCamera();
}
Future<void> _initCamera() async {
final cameras = await availableCameras();
final backCamera = cameras.firstWhere(
(c) => c.lensDirection == CameraLensDirection.back,
orElse: () => cameras.first,
);
_cameraController = CameraController(
backCamera,
ResolutionPreset.high,
enableAudio: false,
imageFormatGroup: ImageFormatGroup.nv21,
);
await _cameraController.initialize();
if (!mounted) return;
setState(() => _isCameraReady = true);
_cameraController.startImageStream(_processFrame);
}
Future<void> _processFrame(CameraImage image) async {
if (_isProcessing) return;
_isProcessing = true;
try {
final inputImage = _buildInputImage(image);
if (inputImage == null) {
_isProcessing = false;
return;
}
final barcodes = await _barcodeScanner.processImage(inputImage);
if (barcodes.isNotEmpty && mounted) {
final barcode = barcodes.first;
setState(() => _scannedValue = barcode.displayValue ?? barcode.rawValue ?? 'Unknown');
}
} catch (e) {
debugPrint('Barcode scan error: $e');
} finally {
_isProcessing = false;
}
}
InputImage? _buildInputImage(CameraImage image) {
final camera = _cameraController.description;
final rotation = InputImageRotationValue.fromRawValue(
camera.sensorOrientation,
);
if (rotation == null) return null;
final format = InputImageFormatValue.fromRawValue(image.format.raw);
if (format == null) return null;
final plane = image.planes.first;
return InputImage.fromBytes(
bytes: plane.bytes,
metadata: InputImageMetadata(
size: Size(image.width.toDouble(), image.height.toDouble()),
rotation: rotation,
format: format,
bytesPerRow: plane.bytesPerRow,
),
);
}
@override
void dispose() {
_cameraController.stopImageStream();
_cameraController.dispose();
_barcodeScanner.close();
super.dispose();
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(title: const Text('Barcode Scanner')),
body: Column(
children: [
Expanded(
child: _isCameraReady
? CameraPreview(_cameraController)
: const Center(child: CircularProgressIndicator()),
),
Container(
width: double.infinity,
color: Colors.black87,
padding: const EdgeInsets.all(16),
child: Text(
_scannedValue,
style: const TextStyle(color: Colors.white, fontSize: 16),
textAlign: TextAlign.center,
),
),
],
),
);
}
}
Key Implementation Notes
_isProcessingflag: Camera streams deliver frames at 30+ fps. Without this guard, you will queue hundreds of concurrent ML operations and crash. Always skip a frame if the previous one has not finished processing.ImageFormatGroup.nv21: Usenv21on Android andbgra8888on iOS. The ML Kit plugin handles the difference, but you must set the correct group on theCameraController.ResolutionPreset.high: Gives a good balance between scanning distance and CPU load. Usemediumfor faster processing,veryHighonly when scanning small 1D barcodes at distance.- Always call
_barcodeScanner.close()indispose()to release native resources.
Face Detection with Bounding Box Overlay
Face detection is central to photo apps, AR filters, attendance systems, and accessibility features. ML Kit’s face detector locates faces in an image and – optionally – returns landmarks (eyes, nose, mouth), contours, and classification scores (smiling probability, eyes-open probability).
Dependencies
# pubspec.yaml
dependencies:
google_mlkit_face_detection: ^0.11.0
camera: ^0.11.0
FaceDetectorOptions Explained
final FaceDetector _faceDetector = FaceDetector(
options: FaceDetectorOptions(
enableClassification: true,
enableLandmarks: true,
enableContours: false,
enableTracking: true,
minFaceSize: 0.1,
performanceMode: FaceDetectorMode.fast,
),
);
Use FaceDetectorMode.fast for live camera feeds and FaceDetectorMode.accurate for processing still images where latency is acceptable.
Full Face Detection Implementation with CustomPainter
import 'dart:ui' as ui;
import 'package:camera/camera.dart';
import 'package:flutter/material.dart';
import 'package:google_mlkit_face_detection/google_mlkit_face_detection.dart';
class FacePainter extends CustomPainter {
FacePainter({
required this.faces,
required this.imageSize,
required this.isFrontCamera,
});
final List<Face> faces;
final Size imageSize;
final bool isFrontCamera;
@override
void paint(Canvas canvas, Size size) {
final paint = Paint()
..color = Colors.greenAccent
..strokeWidth = 2.5
..style = PaintingStyle.stroke;
final textPainter = TextPainter(textDirection: ui.TextDirection.ltr);
for (final face in faces) {
final rect = _scaleRect(face.boundingBox, imageSize, size);
canvas.drawRect(rect, paint);
if (face.smilingProbability != null) {
final label =
'Smile: ${(face.smilingProbability! * 100).toStringAsFixed(0)}%';
textPainter.text = TextSpan(
text: label,
style: const TextStyle(color: Colors.greenAccent, fontSize: 14),
);
textPainter.layout();
textPainter.paint(canvas, rect.topLeft - const Offset(0, 18));
}
}
}
Rect _scaleRect(Rect src, Size imageSize, Size canvasSize) {
final scaleX = canvasSize.width / imageSize.width;
final scaleY = canvasSize.height / imageSize.height;
double left = src.left * scaleX;
double right = src.right * scaleX;
if (isFrontCamera) {
left = canvasSize.width - src.right * scaleX;
right = canvasSize.width - src.left * scaleX;
}
return Rect.fromLTRB(left, src.top * scaleY, right, src.bottom * scaleY);
}
@override
bool shouldRepaint(FacePainter oldDelegate) =>
oldDelegate.faces != faces || oldDelegate.imageSize != imageSize;
}
class FaceDetectionScreen extends StatefulWidget {
const FaceDetectionScreen({super.key});
@override
State<FaceDetectionScreen> createState() => _FaceDetectionScreenState();
}
class _FaceDetectionScreenState extends State<FaceDetectionScreen> {
late CameraController _cameraController;
late FaceDetector _faceDetector;
bool _isProcessing = false;
List<Face> _faces = [];
Size _imageSize = Size.zero;
bool _isCameraReady = false;
bool _isFrontCamera = true;
@override
void initState() {
super.initState();
_faceDetector = FaceDetector(
options: FaceDetectorOptions(
enableClassification: true,
enableLandmarks: true,
performanceMode: FaceDetectorMode.fast,
),
);
_initCamera();
}
Future<void> _initCamera() async {
final cameras = await availableCameras();
final frontCamera = cameras.firstWhere(
(c) => c.lensDirection == CameraLensDirection.front,
orElse: () => cameras.first,
);
_isFrontCamera = frontCamera.lensDirection == CameraLensDirection.front;
_cameraController = CameraController(
frontCamera,
ResolutionPreset.medium,
enableAudio: false,
imageFormatGroup: ImageFormatGroup.nv21,
);
await _cameraController.initialize();
if (!mounted) return;
final size = _cameraController.value.previewSize!;
_imageSize = Size(size.height, size.width);
setState(() => _isCameraReady = true);
_cameraController.startImageStream(_processFrame);
}
Future<void> _processFrame(CameraImage image) async {
if (_isProcessing) return;
_isProcessing = true;
try {
final inputImage = _buildInputImage(image);
if (inputImage == null) return;
final faces = await _faceDetector.processImage(inputImage);
if (mounted) {
setState(() => _faces = faces);
}
} catch (e) {
debugPrint('Face detection error: $e');
} finally {
_isProcessing = false;
}
}
InputImage? _buildInputImage(CameraImage image) {
final camera = _cameraController.description;
final rotation =
InputImageRotationValue.fromRawValue(camera.sensorOrientation);
if (rotation == null) return null;
final format = InputImageFormatValue.fromRawValue(image.format.raw);
if (format == null) return null;
final plane = image.planes.first;
return InputImage.fromBytes(
bytes: plane.bytes,
metadata: InputImageMetadata(
size: Size(image.width.toDouble(), image.height.toDouble()),
rotation: rotation,
format: format,
bytesPerRow: plane.bytesPerRow,
),
);
}
@override
void dispose() {
_cameraController.stopImageStream();
_cameraController.dispose();
_faceDetector.close();
super.dispose();
}
@override
Widget build(BuildContext context) {
if (!_isCameraReady) {
return const Scaffold(body: Center(child: CircularProgressIndicator()));
}
return Scaffold(
appBar: AppBar(title: Text('Face Detection (${_faces.length} faces)')),
body: Stack(
fit: StackFit.expand,
children: [
CameraPreview(_cameraController),
CustomPaint(
painter: FacePainter(
faces: _faces,
imageSize: _imageSize,
isFrontCamera: _isFrontCamera,
),
),
],
),
);
}
}
Front Camera Mirroring
The front camera captures an unmirrored image (the raw sensor output), but Flutter’s CameraPreview displays it mirrored for a natural “selfie” feel. Your bounding boxes must apply the same horizontal mirror transform – see the _scaleRect method above – otherwise boxes will appear on the wrong side of the face.
Text Recognition (OCR)
Text recognition – often called OCR (Optical Character Recognition) – lets your app read printed text from camera frames or static images. Use cases include business card scanners, document digitisation, receipt parsers, and real-time translation overlays.
Dependencies
# pubspec.yaml
dependencies:
google_mlkit_text_recognition: ^0.13.0
camera: ^0.11.0
Frame Throttling – Why It Is Essential
OCR is considerably heavier than barcode scanning. On mid-range devices, a single recognition call can take 80-200 ms. Running it on every frame (30 fps) would queue frames faster than they can be processed, leading to memory pressure and dropped UI frames. The solution is a timestamp-based throttle: only submit a new frame if at least 500 ms have passed since the last submission.
Full OCR Implementation
import 'package:camera/camera.dart';
import 'package:flutter/material.dart';
import 'package:google_mlkit_text_recognition/google_mlkit_text_recognition.dart';
class TextRecognitionScreen extends StatefulWidget {
const TextRecognitionScreen({super.key});
@override
State<TextRecognitionScreen> createState() => _TextRecognitionScreenState();
}
class _TextRecognitionScreenState extends State<TextRecognitionScreen> {
late CameraController _cameraController;
final TextRecognizer _textRecognizer =
TextRecognizer(script: TextRecognitionScript.latin);
bool _isProcessing = false;
String _recognisedText = '';
DateTime? _lastProcessedAt;
bool _isCameraReady = false;
static const Duration _throttleDuration = Duration(milliseconds: 500);
@override
void initState() {
super.initState();
_initCamera();
}
Future<void> _initCamera() async {
final cameras = await availableCameras();
final backCamera = cameras.firstWhere(
(c) => c.lensDirection == CameraLensDirection.back,
orElse: () => cameras.first,
);
_cameraController = CameraController(
backCamera,
ResolutionPreset.medium,
enableAudio: false,
imageFormatGroup: ImageFormatGroup.nv21,
);
await _cameraController.initialize();
if (!mounted) return;
setState(() => _isCameraReady = true);
_cameraController.startImageStream(_processFrame);
}
Future<void> _processFrame(CameraImage image) async {
final now = DateTime.now();
if (_lastProcessedAt != null &&
now.difference(_lastProcessedAt!) < _throttleDuration) {
return;
}
if (_isProcessing) return;
_isProcessing = true;
_lastProcessedAt = now;
try {
final inputImage = _buildInputImage(image);
if (inputImage == null) return;
final RecognizedText result =
await _textRecognizer.processImage(inputImage);
final buffer = StringBuffer();
for (final block in result.blocks) {
for (final line in block.lines) {
buffer.writeln(line.text);
}
}
if (mounted) {
setState(() => _recognisedText =
buffer.toString().trim().isEmpty ? 'No text found' : buffer.toString().trim());
}
} catch (e) {
debugPrint('OCR error: $e');
} finally {
_isProcessing = false;
}
}
InputImage? _buildInputImage(CameraImage image) {
final camera = _cameraController.description;
final rotation =
InputImageRotationValue.fromRawValue(camera.sensorOrientation);
if (rotation == null) return null;
final format = InputImageFormatValue.fromRawValue(image.format.raw);
if (format == null) return null;
final plane = image.planes.first;
return InputImage.fromBytes(
bytes: plane.bytes,
metadata: InputImageMetadata(
size: Size(image.width.toDouble(), image.height.toDouble()),
rotation: rotation,
format: format,
bytesPerRow: plane.bytesPerRow,
),
);
}
@override
void dispose() {
_cameraController.stopImageStream();
_cameraController.dispose();
_textRecognizer.close();
super.dispose();
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(title: const Text('Live OCR')),
body: Column(
children: [
Expanded(
flex: 2,
child: _isCameraReady
? CameraPreview(_cameraController)
: const Center(child: CircularProgressIndicator()),
),
Expanded(
flex: 1,
child: SingleChildScrollView(
padding: const EdgeInsets.all(12),
child: Text(
_recognisedText.isEmpty ? 'Waiting for text.' : _recognisedText,
style: const TextStyle(fontSize: 15),
),
),
),
],
),
);
}
}
Handling Multiple Scripts
The TextRecognitionScript enum supports latin, chinese, devanagari, japanese, korean, and georgian. Each script uses a different bundled model. You can instantiate multiple TextRecognizer objects simultaneously if your app needs to handle mixed-script input, though this increases memory usage.
Performance Optimisation
Combining camera streams with on-device ML is demanding. Without careful engineering, your app will drain the battery and drop frames. Here are the essential patterns.
The _isProcessing Flag Pattern
All three implementations above use a boolean guard to prevent frame queue build-up. This is the single most important pattern for camera-based ML in Flutter:
bool _isProcessing = false;
void _processFrame(CameraImage image) async {
if (_isProcessing) return;
_isProcessing = true;
try {
// ... ML Kit call ...
} finally {
_isProcessing = false;
}
}
The finally block ensures the flag is always cleared, even when an exception is thrown – preventing permanent lock-up of the pipeline.
Choosing the Right ResolutionPreset
| Preset | Typical Resolution | Best for |
|---|---|---|
low |
240p | Fast prototyping only |
medium |
480p | OCR, face detection |
high |
720p | Barcode scanning, small text |
veryHigh |
1080p | High-quality still captures |
ultraHigh |
2160p+ | Rarely needed; very slow |
Always pick the lowest preset that satisfies your accuracy requirement. Halving the resolution (e.g., from 1080p to 480p) reduces pixel count by ~80% and typically more than doubles processing speed.
Closing Detectors in dispose()
Every ML Kit object holds native resources – model weights loaded into memory, interpreter sessions, GPU delegate handles. If you forget to call .close(), you will see memory leaks that grow with each navigation to and from the screen.
@override
void dispose() {
_cameraController.stopImageStream();
_cameraController.dispose();
_barcodeScanner.close();
super.dispose();
}
Always stop the image stream before disposing the camera controller; otherwise callbacks may fire after disposal and cause a use-after-free crash.
Using compute() for Heavy Post-Processing
ML Kit’s inference runs on a native thread and does not block the Dart VM. However, if you perform heavy post-processing of results (e.g., building a complex structured document from OCR blocks), offload it to an isolate using Flutter’s compute():
Map<String, dynamic> parseOcrResult(RecognizedText recognizedText) {
final lines = <String>[];
for (final block in recognizedText.blocks) {
for (final line in block.lines) {
lines.add(line.text);
}
}
return {'lines': lines, 'count': lines.length};
}
// In your widget:
final parsed = await compute(parseOcrResult, result);
Battery Optimisation Tips
- Pause the stream when the app goes to background – listen to
AppLifecycleStateand callstopImageStream()/startImageStream()accordingly. - Throttle aggressively when a result has been found. For example, once a barcode is decoded, pause scanning for 2-3 seconds before resuming.
- Disable unneeded detector options – enabling contours or landmarks in the face detector roughly doubles processing time and power draw.
- Use
FaceDetectorMode.fastoveraccurateunless you specifically need landmark precision for an AR use case.
Conclusion
Google ML Kit removes the biggest barrier to mobile ML adoption – the need to train, optimise, and deploy your own models. With the google_mlkit_* Flutter packages, you get:
- Real-time barcode scanning across dozens of formats with a handful of lines of Dart.
- Face detection with bounding boxes, landmarks, and emotion classification – rendered live with
CustomPainter. - On-device text recognition supporting Latin and several CJK scripts, throttled to stay smooth on any device.
The architectural patterns – the _isProcessing guard, timestamp throttling, correct dispose() ordering, and appropriate ResolutionPreset – are the difference between a demo that crashes and a production feature your users trust.
As ML Kit continues to expand (on-device translation, document scanning, subject segmentation), mastering these foundations puts you in an excellent position to add new capabilities with minimal effort.
Want more Flutter tips? Explore more tutorials on FlutterExperts.com and level up your app development skills today!
