Facebook Portal Puts AR Messenger Video Calls In Alexa Smart Displays

Facebook has revealed its long-rumored video calling devices, Facebook Portal and Portal+, the social network's first homegrown hardware not to mention its first entrant in the smart display space. Promising a more natural way to hold video calls between Messenger friends and family, the two new devices also feature Amazon Alexa support, taking on products like the 2nd Gen Echo Show in the process.

The biggest difference between the two is the size. Facebook Portal has a 10-inch 1280 x 800 display in landscape orientation, much like the new Echo Show or Lenovo's Smart Display. It's Facebook's entry-level model, priced at $199.

Facebook Portal+, however, is altogether more interesting. It gets a 15-inch, 1920 x 1080 display which can pivot between landscape and portrait orientation, and is priced at $349. The mechanism itself is pleasing in its tactility, with just the right amount of counterbalancing to swing smoothly between the two positions.

Indeed, ease of use is Facebook's primary focus with both gadgets. Video calling typically comes with friction, Rafa Camargo, Vice President of Portal explained to me. Whether it's the headaches involved in setting up a phone, tablet, or laptop camera to capture you properly, or the feeling that the calls themselves need to be organized and planned for in a way that feels more businesslike than friendly, they're issues Facebook believes it can address with dedicated hardware and some AI smarts.

The Portal and Portal+ devices themselves put functionality at the fore. Both have front-facing speakers – 10W on the smaller unit, 20W on the larger plus a 4-inch bass speaker – and a 4-way microphone array. They're finished in either matte black or white plastic, with Facebook's logo discreetly placed on the back rather than emblazoned across the front.

At the top of each is a 12-megapixel camera with a particularly wide, 140-degree field of view. The combination of that high resolution and broad perspective allows for Portal's big magic trick, dubbed Smart Camera. Effectively it's a combination of 2D pose recognition to identify bodies – and the heads atop those bodies – in the frame, and then digital cropping and zooming to make sure they're centered in the picture.

The result is a virtual cameraman, always making sure you're in view. If you're on your own in the call, Smart Camera automatically follows you around. With two or more people in frame, Portal will zoom so that everybody is visible. Alternatively, it's possible to select a person and have Smart Camera focus entirely on them. It does so with surprising smoothness and grace; according to Carmago, Facebook called upon the expertise of an Academy Award-winning cinematographer to help get that right.

At the same time, the Smart Sound system aims to do the same with the microphone array. That means, Camargo explained to me, that you shouldn't need to shout, even if you're on the opposite side of the room.

After all, the goal is to make Portal calls – which can take place between Portal and Portal+ units, or to anybody running Facebook Messenger – more akin to casual conversations than organized video chats. That includes sharing more than just talk. Log into your Spotify, Pandora, or iHeartRadio account, and both sides of the call can listen to music together. Newsy, Food Network, and Facebook Watch will do the same with headlines, recipes, and other videos.

There's something more fun, however. Facebook's Spark AR-based augmented reality stickers, for example, will allow you to don silly hats or other add-ons, tracking you as you move around the room. Even more entertaining is Story Time: one person gets a teleprompter for a children's story, while at the other end of the call there are AR effects, graphics, and sound effects. Portal will launch with "Three Little Pigs," "Five Little Monkeys," "Itsy Bitsy Spider," "Professor Right," and "Trouble Bubble."

The primary method of control for all this will be voice. Facebook has built its own interactive assistant, summoned by calling out "Hey Portal," for a hands-free experience. Alexa, meanwhile, fills in the gaps for everything else, including on-screen graphics for things like weather forecasts and conversions. Alternatively, though, you can tap the Portal and Portal+ touchscreens.

It's a simple UI, but Carmago and the team have been thorough with it. The core interface is thumbnails of contacts, complete with availability indicators. Portal can suggest those for you, based on who you speak to most often, or you'll be able to pin your own favorites to the top of the list. Multiple Facebook users will be able to log in simultaneously, with Portal supporting switching between friends lists. If you're on a video call through Messenger on your phone, you can transfer it – without interruption – to Portal, and vice-versa.

When you're not in a call, Portal switches into what Facebook is referring to as Superframe mode, effectively becoming a digital photo frame. It'll also show birthday reminders, and suggest friends who are available online at that moment, and whom you might want to have an impromptu call with. Selecting which images and galleries you want to be included in Superframe will be done manually: Facebook is looking to avoid any sort of unpleasant "why is it showing that?" surprises when you turn the corner and see what Portal is displaying.

While the Portal team has been hard at work, it's fair to say that Facebook has been going through a rough period when it comes to perceptions of privacy. Unsurprising, then, is just how much emphasis it's placing on data security and how personal information is being taken care of, all the more important when you consider Facebook is hoping users will bring an internet-connected camera into their homes.

Privacy was the first consideration when Portal and Portal+ were being designed, Carmago insisted to me, both in hardware and software. Hit the mute button, for instance, and the camera and microphone are "physically disconnected" the Facebook VP explained. If you still want Portal to be able to hear and respond to voice instructions, but not see you, there's a slide-on camera cover included in the box. A screen lock passcode – of up to 12 digits – can be added to limit access.

On the software side, Facebook says all Portal video calls are encrypted, and it can't access or view any of the conversations. Both Smart Camera and Smart Sound rely on on-device processing, rather than Facebook's servers, and the automatic framing is based on pose recognition rather than identifying and recalling faces. Like an Echo or Google Home, Portal does use server-side processing for voice instructions, though only after the device itself has recognized the "Hey Portal" wake-phrase. Your voice history can be accessed – and deleted – through the Facebook Activity Log.

At launch, Portal+ and Portal will be fairly focused in their purpose. They'll do video calls, play music, act as an Alexa smart speaker, and allow you to stream music over Bluetooth from your phone or laptop. However Carmago and the team is already thinking about future possibilities. Story Time, for instance, has proved a hit when he and the Portal group have discussed it with publishers, so we can likely expect more interactive storybooks in future.

Similarly, some of the Portal test group – Facebook has around a thousand out in the wild right now, for real-world feedback – have asked for a way to record calls. Having an AI videographer to film you could certainly be an interesting prospect for YouTubers and vloggers, after all, though Carmago says one key point there will be making it abundantly clear at which times you're being recorded.

I suspect that caution is well placed. Facebook may namecheck Portal's privacy credentials, but it can't have escaped Carmago and his team's attention that all it would really take to scupper the video calling devices' prospects would be a data scandal. The social network isn't short on critics these days, some of whom wield regulatory powers, and one misstep could completely pull the rug out from under Portal.

Even with those safeguards and promises, Portal and Portal+ likely face an uphill challenge. The reality the team has observed, marketing lead Dave Kaufman told me, is that families given access to the devices quickly come to rely on them. The number of video calls made increases noticeably, and such chats become less planned and more casual and impromptu. All the same, Facebook will need to convince people that a standalone device is worth spending $200 or more on.

That'll include competing with rivals like Echo Show and other smart displays. The smaller Portal undercuts Amazon's 2nd Gen model, not to mention the Lenovo Smart Display, but there's no denying that Portal+ really is the star. It comes with a leading man price tag, however.

Facebook is quietly confident it has aced the equation, though, and there's not long to wait to see if the social network is right. Portal and Portal+ are going up for preorder today in the US, both from Facebook direct and via Amazon and Best Buy. They'll begin shipping in November. As well as Portal for $199 and Portal+ for $349, there'll be bundle deals of any two units with a $100 saving on the pair.