This Is The Cozmo SDK: I Think It'll Do For Robots What iOS Did For Apps

Robots with personality aren't new, but Anki's Cozmo has broader goals than just entertainment: it wants to bring advanced coding techniques to the mass market. Announced last month, Cozmo – a palm-sized robot that uses your smartphone for a brain – may look like a toy but its companion SDK is arguably just as ambitious as that of iOS or Android.

Cosmo's raw appeal is obvious. With its animated expressions, chipmunk-esque chattering, and a cutesy forklift design fit for the the Doozers from Fraggle Rock, it has all the engagement potential of a Pixar character.

Preorders kicked off two weeks ago; Anki isn't talking exact sales numbers but, even with internal expectations that the company was on to something potentially big, demand has proved greater than predicted.

"It's a lot more than we thought," Hanns Tappeiner, co-founder and president of Anki, told me ahead of the SDK's big reveal. In fact, in year one the company is already expecting demand to exceed supply, as production ramps up.

"Cozmo is a really complex product," Tappeiner explains, "so the manufacturing is really complex too." Mass product started a couple of weeks ago, but one of the major bottlenecks is testing. There are over twelve different test points in the factory, and every robot goes into a "playpen" at the end during which its visual, motor, and other abilities are assessed.

Out of the box, Cozmo will be able to play a variety of games, recognize and remember faces, and generally act as a tiny desktop companion. Yet Anki's goal was always more than just making a better Furby: the company wanted, even before the first Anki Drive went on sale, to package robot development in a way that was open to the most inexpert of people.

I'm one of those "inexperts"; I've loved robotics and the idea of robots since I was a kid, and I've always wanted a "robot friend", but I don't have the technical skill to package those ambitions into something that gets anywhere near what you see in sci-fi movies and TV shows.

Perhaps I shouldn't feel so bad for lacking that ability. When you contrast the current state of robotics with app development, Tappeiner tells me, the scale of the problem becomes clear. Getting started with apps is fairly linear, from the "Hello World" basics through to something actually useful.

Developing for robots, in contrast, involves an exponential jump. Yes, the fundamentals – driving, and turning at obstacles – are straightforward, but there's a vast gulf between that and the "next step" of things like face recognition, path planning, developing a 3D world model, and other skills.

That jump, in fact, is about equivalent to a PhD in robotics, Tappeiner says, and even then it's challenging stuff for someone who has made robots their profession.

"There's no fundamental difference between apps and robotics," Tappeiner argues, "but it's because people at Apple and Google have written thousands of lines of code, so you don't have to write your own. In order to advance robotics as a whole, we need to move away from a point where only robotics experts can take part."

The result is the Cozmo SDK, more than a million lines of code that collectively boil down some of the most advanced technologies in modern robotics into something a relative novice can implement. It's the spawn of something Anki had no choice but to develop: the company's software engineers needed to make something simple enough that its game developers could pick up and run with without having to do a robotics PhD of their own.

Robotics software isn't new, of course, but what differentiates the Cozmo SDK is how it hides its complexity. "The trick was to make those million lines of code available to people who don't know robotics," Tappeiner explains, "with just a single line of code."

A good example is Cozmo's ability to count the number of faces he sees with his camera. Someone using the SDK can call that ability with a single instruction – "numFaces" – but that one line represents thousands of lines of code. [Updated to clarify the amount of code "numFaces" represents]

Developers still get all the low-level access they might want – they can turn on Cozmo's backpack light, drive him around the desktop, play sounds, and show anything they want on his OLED face display – but they also get the sum intelligence of the Anki team in straightforward shortcuts.

Say, for instance, you want Cozmo to pick up a certain cube someone on the desk. That might involve a simple scurry from point A to point B, but more likely it's a fairly complex act of navigation around obstacles and into the right position where the robot's grapples can engage with the cube. In fact, it's a combination of localization, real-time path planning, 3D pose estimation, and more that's enough to give even seasoned roboticists a migraine, but you can ask Cozmo to do it with a single keyword.

"The high level functionality people couldn't develop on their own," Tappeiner says, "they have access to."

Cozmo may be small, and surprisingly cheap for a robot with a fully-fledged development environment, but the list of talents you can unlock with the SDK is vast.

For a start you can tap into the vision system, for face tracking, planning and AI, manipulation of objects, animations, and even pet detection – Cozmo can differentiate between cats and dogs, though not remember specific animals – in addition to marker and motion detection, color detection, and direct access to streaming video from the camera.

It can identify up to 44 different face markers, as well as identify age, gender, eye-openness, and facial expression – classifying that into happy, sad, angry, surprised, or neutral – of the people around it, as well as figuring whether you're looking at Cozmo or elsewhere.

There's a real-time path planner, simultaneous location and mapping (SLAM) to both navigate and build a map at the same time, and world exploration, plus 3D position tracking, and the ability to push, pick up, stack, and roll the three light-up cubes that will come in the box with Cozmo. Finally, there's the emotion engine and behavior engine, which call upon the handiwork of Carlos Baena, responsible for animating Wall-E, Buzz Lightyear, and other Pixar characters, and which give Cozmo his lifelike personality.

Using the SDK promises to be straightforward. It uses the same app as Anki relies on to control Cozmo from your iOS or Android device, tethered via USB to a laptop; there's nothing to install on your computer.

Even those who haven't coughed up $180 for a Cozmo themselves will be able to try their hand at developing for the robot. Anki plans to add a simulator – effectively a virtual Cozmo that does exactly what the real 'bot would do on your desk – about a month after the SDK's launch.

It's all been built with flexibility baked in from the outset, too. "If people don't like Python they can use their choice of programming language," Tappeiner says. "Anybody can go and look through our code – the messages are really well documented – and in a few hours you could have Swift, or Scratch, to Go interact with the Cozmo engine in the same way our Python SDK does."

"You can combine it with any other SDKs which are out there," Tappeiner adds, describing how his web developer wife, in her first ninety minutes with Cozmo, whipped up a program that linked the robot with Philips' hue SDK so that it could remotely control the lights.

If you'd rather use your own text-to-speech engine, or your own computer vision system, that's possible too. Even if you bypass Cozmo's native abilities, you still get the benefits of the personality engine.

That makes a huge difference. Watching as Tappeiner slotted a few single lines of code into the SDK, I knew to expect the basics: the robot would look for a toppled block, and upon seeing it move to roll it so that it was the right way up.

What Cozmo actually did, however, was much more engaging. When he caught sight of the block, his eyes went wide and he span to face Tappeiner – having remembered where he last saw him in the room – and babbled some excited gibberish, before spinning back around, scooting forward to grab his toy and tip it, before cooing proudly.

"It makes you feel like you're programming a little thing and not a dead machine," Tappeiner observes.

That's going to be particularly important when Anki moves into phase two of its roll-out for Cozmo. Initially, the target audience are hackers, makers, students, and researchers. People who don't really need to know anything about robotics beyond that they have an interest in it, but who have ideally used a Python-based SDK before.

Beyond that, though, Anki is looking to K-12 classrooms, where the company envisages Cozmo being a huge boon to STEM (science, technology, engineering, and math) education. That'll probably involve a wrapper around the core SDK that makes constructing programs for the robot as straightforward as dragging blocks together on-screen.

Finally, phase three embraces third-party content developers. While the SDK will be released on October alongside Cozmo itself, it'll only be a beta at that stage. The full SDK isn't expected until perhaps another six months after; that, Tappeiner explains, should give the company plenty of time to iterate with feedback from developers.

The primary limitations will be the reliance on a tethered laptop to actually run apps on Cozmo, and the absence of access to the Anki app store. That won't happen until the SDK itself is finalized.

Given the four years of effort that has gone into programming Cozmo, I asked Tappeiner whether the company was considering turning the SDK and the robot's AI into a standalone product to use with other hardware. That, he says however, presents a greater challenge than you might think.

"Robots are complete different," Tappeiner says, "you have Cozmo here, you have a humanoid robot there, you have a huge robotic arm. There are some common parts, but in order to make the SDK work for Cozmo we had to tailor it really, really well to Cozmo's hardware."

Introduce a different camera module, for instance, or a new set of servos, or even just a chance in size and balance, and the core engine just won't do what it should. "It was the only thing to make sure it works really, really well in all conditions," Tappeiner concludes.

That's hardly bad news for Anki, of course, which wants to sell robots, but it's also pretty good news for robotics developers who have never had access to this level of advanced hardware, with these companion software skills, at such an affordable price. Traditional research robots are thousands of dollars apiece and, in all but the richest of labs, are shared between numerous users; Cozmo is cheap enough that everybody could have one of their own.

NOW READ: Why Cozmo is ambassador for robo-human relations

Anki isn't the only company trying to make companion robots more personable – the delayed Jibo, for instance, has its own take on using lifelike animation to ease a 'bot into your daily life – but on a bang-for-buck basis there's not much out there that can compete.

Whether individuals, researchers, and schools alike all embrace Cozmo in the way Anki hopes remains to be seen, but this tiny, engaging toy has the potential to have a big impact on how robotics is brought into the mainstream.