- Personal Data
- Augmented Reality
Gone are the days when machine learning (ML) was something that only seasoned developers and software engineers could understand. Modern technologies can now train software to recognize objects, images, speech, gestures, and even emotions — effectively taking mobile apps to another level.
Several frameworks on the market now cut the time needed to train ML models from weeks to hours. Each has its pros and cons. iOS machine learning stands out as one of the best, as Apple’s Core ML framework offers a convenient drag-and-drop interface to streamline the training process and harness the power of ML models for embedded apps.
Core ML is Apple’s foundational machine learning framework for integrating ML models into iOS applications. It’s fully compatible with all Apple products and offers fast performance and easy integration of trained ML models.
An iPhone app uses Core ML APIs and stored data to analyze information, detect patterns, and make predictions. This means that inferences are made directly on the mobile device. Models are also trained on the user’s device.
A Core ML model is the result of applying an ML algorithm to training data. You can build a model using the Create ML app bundled with Xcode. In this case, no conversion is required, as models trained with Create ML are already in a format suitable for use in an app. There are also Core ML tools and third-party ML libraries available for converting your model into Core ML format.
The framework is built on three main technologies:
One of the good things about Core ML is that it allows you to switch between the CPU, GPU, and ANE on the fly. This optimizes on-device performance and means you don’t have to decide which one to run for each model.
Since its introduction in 2017, Core ML has gone through three iterations as Apple has upgraded the framework and added new tools. It’s currently in its fourth version, known as Core ML 4.0.
Core ML supports four training domains that define its architecture: vision, NLP, speech recognition, and sound analysis.
Here’s what that looks like:
The list of Core ML support features and models grows each year. Here are the main categories:
If you’re using third-party training libraries, you can easily convert them to Core ML with the coremltools Python package.
Every year, the ML team at Apple extends their list of supported libraries. In 2020, they announced support for the most popular neural network libraries, TensorFlow and PyTorch, and turned coremltools into a one-stop shop for converting to Core ML.
That same year, the company introduced a single converter stack called Model Intermediate Language (MIL). MIL was designed to streamline the conversion process and make it easy to add support for new frameworks. It unifies the stack by providing a standard interface for capturing information from different frameworks.
To integrate a Core ML model into your app, follow these steps:
Since its introduction, iOS machine learning has enjoyed wide implementation. Here are some successful use cases:
At WWDC 2020, Apple introduced its Action Classifier model and the Action & Vision app. The app uses Object Detection and Action Classification in Create ML along with the Body Pose Estimation, Trajectory Detection, and Contour Detection features in the Vision framework. The app analyzes your movements during physical activity and helps improve your training performance.
The video below shows a developer experimenting with an app powered by Core ML and ARKit, a framework based on facial recognition technology. The model is trained to recognize emotions and showcases the power of the Core ML Vision framework.
Style Art is a library that processes images using Core ML with a set of pre-trained ML models and converts them to ArtStyle.
Image Colorizer colorizes grayscale images using the Core ML, Vision, and Core Image frameworks.
SeeFood is an app that uses CoreML to detect various dishes.
Apple is constantly upgrading its machine learning technology, and Core ML is getting better every year as it offers users more and more opportunities to implement it in apps.
If you’re still not sure whether it’s for you, check out this list of the main pros and cons of Core ML implementation:
Get actionable insights for your product
Thank you for reaching out, !
Make sure to check for details.