SiriKit Resolutions with Swift 3 and iOS 10 – SiriKit Tutorial (Part 2)

SiriKit Resolutions with Swift 3 in iOS 10 – SiriKit Tutorial (Part 2)

This tutorial written on June 20th, 2016 using the Xcode 8 Beta 1, and is using the Swift 3.0 toolchain.

This post is a follow-up in a multi-part SiriKit tutorial. If you have not read part 1 yet, I recommend starting there.

Resolving requests from SiriKit

In order to make our Siri integration more useful, we can help fill out the content of our message using a callback method from the INSendMessageIntentHandling protocol. Investigating this protocol you can see this show up an optional methods.

resolveRecipients(forSendMessage intent: INSendMessageIntent, with completion: ([INPersonResolutionResult]) -> Swift.Void)
 
resolveContent(forSendMessage intent: INSendMessageIntent, with completion: (INStringResolutionResult) -> Swift.Void)
 
resolveGroupName(forSendMessage intent: INSendMessageIntent, with completion: (INStringResolutionResult) -> Swift.Void)
 
resolveServiceName(forSendMessage intent: INSendMessageIntent, with completion: (INStringResolutionResult) -> Swift.Void)
 
resolveSender(forSendMessage intent: INSendMessageIntent, with completion: (INPersonResolutionResult) -> Swift.Void)

So we can provide SiriKit with further information by implementing as many of these resolutions as we wish. Effectively enabling us to provide information regarding the recipients, content, group name, service name, or sender. These should be relatively self-explanatory.

Let’s try providing some static data for our title and content, to demonstrate how resolutions work.

First, let’s add the resolution for the content of the message, by implementing the resolveContent protocol method.

func resolveContent(forSendMessage intent: INSendMessageIntent, with completion: (INStringResolutionResult) -> Void) {
    let message = "My message body!"
    let response = INStringResolutionResult.success(with: message)
    completion(response)
}

Here we create a string resolution result, and call the success function. This is the simplest way to proceed, but we also have the option of returning a disambiguation, confirmationRequired, or unsupported response. We’ll get to those later, but first let’s actually use the data Siri is providing us.

Siri will send in it’s own transcription of our message in the intent object. We’re interested in the content property, so let’s take that and embed it inside of a string.

func resolveContent(forSendMessage intent: INSendMessageIntent, with completion: (INStringResolutionResult) -> Void) {
    let message = "Dictated text: \(content!)"
    let response = INStringResolutionResult.success(with: message)
 
    completion(response)
}

The content property is an optional, and as such we need to make sure Siri actually provided a transcription. If no transcription was provided then a message won’t be entirely useful, so we need to tell Siri that the information is missing and we need this value. We can do this by returning a resolution result calling the needsValue class method on INStringResolutionResult.

func resolveContent(forSendMessage intent: INSendMessageIntent, with completion: (INStringResolutionResult) -> Void) {
    if let content = intent.content {
        let message = "Dictated text: \(content)"
        let response = INStringResolutionResult.success(with: message)
        completion(response)
    }
    else {
        let response = INStringResolutionResult.needsValue()
        completion(response)
    }
}

SiriKit requesting additional information

Now SiriKit knows when we try to send a message, that the content value is a requirement. We should implement the same type of thing for the recipients. In this case, recipients can have multiple values, and we can look them up in a variety of ways. If you have a messaging app, you would need to take the INPerson intent object that is passed in and try to determine which of your own user’s the message is intended for.

This goes outside the scope of this Siri tutorial, so I’ll leave it up to you to implement your own application logic for the resolveRecipients method. If you want to see an example implementation, Apple have released some sample code here.

More iOS 10 Tutorials

We’ll be continuing to investigate iOS 10 and publish more free tutorials in the future. If you want to follow along be sure to subscribe to our newsletter and follow me on Twitter.

Thanks,
Jameson


Sign up now and get a set of FREE video tutorials on writing iOS apps coming soon.

Subscribe via RSS

Creating an iMessage Sticker App on iOS 10 with Swift – Tutorial (Part 1)

Creating iMessage Apps with XCode 8

This is part of a series of tutorials introducing new features in iOS 10, the Swift programming language, and the new XCode 8 beta, which were just announced at WWDC 16

Intro

Among the most exciting additions in iOS 10 are some of the new app extension types, and the new extension type that really caught my attention was the iMessage app, which allow you to add functionality to the built-in Messages app on iOS.

Unlike other extension types, which requires you to bundle your extension with a standard app, iMessage apps can be released as standalone apps through the the new iMessage App Store, which will be accessible from within the Messages app. While you can include your Messages app extension as part of a standalone app, with this new branch of the app store, for the first time, you don’t have to.

When users use your iMessage app to communicate with people who don’t have it, the recipient will be prompted with a link to download it from the store.

The purpose of this tutorial is to create a quick, searchable guide for creating a sticker app (which you can do without writing a line of code), and very briefly introduce the new Messages framework (Messages.framework) for building interactive apps.

Creating Content and Apps for iMessage

iMessage Sticker Apps

There’s a very easy option for artists (even non-programmers) who want to release content for the iMessage app store. You can offer sticker apps: purchasable packs of stickers (even animated stickers) that users can “peel” (using a long press) and slap onto any balloon in a conversation. Like all iMessage apps, these packs can be a bonus bundled with your existing app, or a standalone package offered for sale. Apart from creating the assets, releasing a sticker app couldn’t be much easier.

Still in Beta

A sticker pane, with a sick computer stuck onto a message

The sticker packs themselves don’t require any code to create. All you really need is a copy of XCode 8 and an Apple Developer account. The few simple guidelines for creating the assets can be found under “Sticker Packs” in the Messages Framework reference. For convenience, I’ve copied a slightly-edited version here.

Each sticker image should meet this criteria:

  • The image must be a PNG, APNG, GIF or JPEG file.
  • The file must be less than 500KB.
  • The image must be between 100×100 and 206×206 points

PNG or APNG (Animated PNG) will generally look best. Make sure your stickers look good against different color backgrounds; they can be applied to photos.

Apple uses points (rather than pixels) for screen coordinates and dimensions on their devices. 10×10 points (pt) @ 2x equals 20×20 pixels (px). The original retina display crammed 4 pixels into the space of one (1×1 became 2×2). Simply by providing ‘@2x’ copies of images at that higher resolution, developers were able to quickly update their old apps to support the sharper display.

Always provide @3x images – between 300×300 and 618×618 pixels. The system generates the @2x and @1x versions by downscaling the @3x images at runtime.

In Xcode’s Attributes inspector, set the Sticker Size for the entire Sticker pack. The system lays out the stickers in the browser based on these sizes. To best match the browser, use sticker images of the specified size. The Sticker Size defaults to Medium.

The recommended maximum dimensions for each size setting are:

  • Small: 100x100pts.
  • Medium: 136x136pts.
  • Large: 206x206pts.

Attributes Inspector

Xcode’s Attributes Inspector after selecting the “Sticker Pack” group in Stickers.xcstickers

Building the App

Start a new project in XCode, and you’ll see there’s a new template for creating a standalone sticker pack.

Sticker Pack Template

You’re almost done. Select Stickers.xcstickers and add some icons so that your sticker pack looks good in Messages and on the store, then drag your stickers into the “Sticker Pack” group.

Note: You can create animated (APNG) stickers with just XCode. Right click inside your sticker pack group and select Add Assets -> New Sticker Sequence. Use the attributes inspector on the right to set options like number of frames, framerate and repetition count.

Sticker Pack Template

Note: If the app won’t run, try the following:

  • Select your “Run” build scheme, ensure that ‘Executable’ is set to [projectname].app (stickers.app in my case)
  • Provide images for at least the 1x versions of the 27×20, 32×24, and 1024×768 point icons, as well as the 2x icon for “Settings” and “Messages” for your target platform
  • Note that 2x means to double the requested dimensions (2x for 29x29pt means provide a 58×58 px image)

If you are ramping up to release your app, make sure to provide images for all icon versions and sizes.

The new version of the simulator for iOS 10 includes a special version of the Messages app which allows you to see both sides of a conversation as you test your iMessage apps.

My sticker app (containing an animated gif and the obligatory cat photo) may not win any awards, but it works. Here is is in action, with the sticker tray open and a few stickers strategically applied:

AAAAAAHHHHH!!

If you’ve made it this far (and provided more/better art) your app is now ready for submission to iTunes Connect and Apple’s review process (pending availability of the release version of XCode 8).

Customizing the Sticker Browser

Developers should be aware that it is possible to customize the appearance of the sticker browser (the window that users peel stickers from) in a full-fledged iMessage app by subclassing MSStickerBrowserViewController. You can also load stickers dynamically by setting your StickerBrowserViewController’s data source to a class which conforms to MSStickerBrowserViewDataSource.

Developing Interactive iMessage Apps

There’s a lot more you can do with iMessage Apps. For now, I just want to briefly discuss Messages.framework. I hope to dive much deeper in the near future.

Note: There is one major limitation that may kill your idea. You won’t be allowed to read the content of user’s messages or even, necessarily, know anything about the participants in the session. Apple takes privacy very seriously; this is as it should be.

A number of the built-in apps provide good examples. Apart from the sticker features, the new drawing capabilities, the music sharing app and the Bing image search, in particular, are good examples of the type of thing that third-party developers could realistically build today.

There is a new, relatively bare-bones, project template in XCode 8: “Messages Application”. MessagesViewController a subclass of MSMessagesAppViewController is your main entry point and does at least provide a number of comments. This is the view controller behind your app’s view in the app drawer and when expanded.

The documentation on these classes is there for anyone getting started and a helpful introduction and associated sample project are now available.


Sign up now and get a set of FREE video tutorials on writing iOS apps coming soon.

Subscribe via RSS