WWDC 2016: Rich Notifications in iOS 10

Posted 2020-08-09

Notifications have gotten more than a visual refresh in ios 10. As part of the new UserNotifications framework, Apple has given us the ability to make notifications that contain images, sound, video, or even custom content generated on the device. Much of this capability comes via three new features showcased at WWDC: media attachments, notification service extensions, and notification content extensions.

 

Media Attachments

A media attachment is an object we can attach to a notification that contains the URL of a media file. Supported media file types include common audio, video, and image formats. There is even built-in support for animated gifs–a surprising fact considering the lack of such support in classes like UIImageView. iOS 10 presents media attachments in a way that will look familiar if you’ve ever received a notification for a photo message from Apple’s Messages app.

Like a Messages alert, a notification containing an image or video attachment shows a thumbnail version of the attachment. This is a nice preview, but the really special part happens when you 3D Touch the notification. New in iOS 10, the notification now expands and displays a full-size version of the attachment with animation, action buttons, and playback controls enabled automatically.

Why They Are Awesome

Media attachments open up new opportunities for user engagement with our apps. Before, we were limited to just a couple lines of text; we had to rely on the user opening the app to do anything more. However, in keeping with Apple’s push for “preview” style interactions using 3D Touch, we can now engage users with rich content without them having to open the app. This is a win-win for app makers and users. App makers get to deliver more compelling content in their notifications, and users get more choice in their level of notification interaction.

How They Work

Media attachments work slightly different between local notifications and remote notifications. In the case of a local notification, its media attachment must contain the URL of a file on disk at the time that an app creates the notification. The file is copied when the notification is scheduled, and that file is delivered along with the notification. No extra effort is required.

For a remote notification, a remote notification service delivers information about the media attachment as part of the APNS notification payload. This includes the attachment URL, which, importantly, does not have to be the URL of a file already on the device. However, iOS 10 will not automatically deliver an attachment with a notification if that attachment is not stored locally. Apple’s solution to this problem is something called the Notification Service Extension.

Notification Service Extensions

A Notification Service Extension is something we can build into an app that may download or change the content of that app’s remote notifications without even opening the app. At 4KB in size, a remote notification’s payload is too small to deliver a media attachment file itself. Instead, we define an attachment URL in the remote notification payload. Once a device receives the remote notification payload for our app, the app’s service extension gets the chance to download the file at the URL and attach it to the notification before the device displays the notification to the user.

Why They Are Awesome

My first thought when looking at this feature was, why do we need this? Why doesn’t iOS just download the attachment automatically and add it to the notification? However, looking at some of the code generated when adding this extension in Xcode reveals Apple’s intentions. Below is one of the functions we need to fill in when implementing a Notification Service Extension.

 

 

1 2 3 4 5 6 7

override func serviceExtensionTimeWillExpire() {     // Called just before the extension will be terminated by the system.     // Use this as an opportunity to deliver your "best attempt" at modified content, otherwise the original push payload will be used.     if let contentHandler = contentHandler, let bestAttemptContent =  bestAttemptContent {         contentHandler(bestAttemptContent)     } }

 

Network calls are inherently unreliable, and this method is one way Apple has given us to guard against poor user experience as a consequence. In the case where our attachment takes too long to download, the OS executes this serviceExtensionTimeWillExpire() callback, allowing us to fail gracefully and deliver a notification with “best attempt” content. What that content should be will depend on the app, but the Notification Service Extension provides us the ability deliver it if necessary.

How They Work

On the notification server side, sending a notification that will be processed by a service extension is as simple as setting two fields in the APNS payload.

 

 

1 2 3 4 5 6 7

{   aps: {     alert: {...}     mutable-content: 1   }   my-attachment: "https://foo.bar/baz.jpg" }

 

The mutable-content fields tells iOS that the notification has content that can be changed by a service extension before delivery, and the my-attachment field contains the URL for the content to be downloaded.

To make an app extension that will handle that notification, we just add a notification service extension as a new target in the app’s Xcode project. Xcode will generate a file containing a subclass of UNNotificationServiceExtension with two functions to be overridden.

 

 

1 2 3 4 5 6 7

override func didReceive(_ request: UNNotificationRequest, withContentHandler contentHandler:(UNNotificationContent) -> Void) {   // ... } // ... override func serviceExtensionTimeWillExpire() {   // ... }

 

We use the first function to download the content from the URL and add it to the notification. We should also handle network errors here. We use the second function, mentioned earlier, to handle the case where the first function runs out of time (as determined by iOS). The end result is a remote notification with a media attachment that, from the user’s perspective, looks, feels, and works the same as a local notification with locally stored content.

But what if we don’t just want to serve up simple media files? What if we want to generate new content that has the same look and feel as our app? That’s where Notification Content Extensions come in.

Notification Content Extensions

A Notification Content Extension is similar to a Notification Service Extension in that it gives us a chance to process a notification and supply new content to the notification before it is displayed to the user. However, these extension types differ in two important ways. First, a content extension may operate on both remote and local notifications. Second, a content extension does not merely alter a media attachment; it supplies an entire UIViewController to be rendered upon notification expansion.

Why They Are Awesome

Content extensions give us the power to render notifications with app-native content–that is, content with the style, capabilities, and local context of the app itself–without actually having to load the app. iOS 10 renders a content extension’s view inside the same kind of expanded notification used by the previous two features. Notifications on Apple’s own Calendar app make for a great example.

We see here a local notification about a calendar event that uses the same kind of visualization as the Calendar app itself would use to display the event. And, because the view is rendered locally with Calendar app user data, the view can show information about the event in the context of other events on the user’s calendar. Such content extensions present a great opportunity for the personalization of notifications.

There is one crucial limitation in content extensions, however, which speaks strongly to the way Apple feels content extensions should be used by developers. User interaction is all but disabled for content extensions, save for notification action buttons and optional media playback controls. Even with this limitation, however, we have ample opportunity to make rich, engaging notifications using content extensions.

How They Work

Like a service extension, adding a content extension to an app is as simple as creating one as a new target for our app in Xcode. Xcode generates a new storyboard file and corresponding source code file as well as an “info.plist” file for our content extension. The source code file contains a subclass of UIViewController that implements the UNNotificationContentExtension protocol. To make our content extension work, we need only flesh out the required protocol function, customize the view in the storyboard, and set the category ID of the notifications we want to handle in the “info.plist” file. We may also implement optional functions that handle action button presses or display media playback controls.

Summary

Apple has given developers three great new features for delivering notifications with rich, engaging content to app users. We can show full-size images, play audio and video, or even display custom visualizations in notifications before users even open our apps. Apple is pushing hard for this kind of tiered interaction using 3D Touch in its own apps, and users are going to come to expect it in all the apps they use. If your app displays notifications for any reason, you’re going to want to explore how you can take advantage of these great features.

Helpful Links

Xcode 8 Beta

NotificationsDemo project

Apple’s Local and Remote Notifications Programming Guide

UserNotifications Framework

UserNotificationsUI Framework

WWDC 2016: Introduction to Notifications Video

WWDC 2016: Advanced Notifications Video