Mark Zuckerberg Explains How He Built His Jarvis AI System

At the beginning of 2016, Mark Zuckerberg set out to build an AI system for his home, likening such a system to Jarvis from the Iron Man movies and comic books. Now that we're just two weeks out from the end of the year, Zuckerberg has delivered another update on his system, announcing that it's been almost fully realized. In a lengthy Facebook note, Zuckerberg walks us through everything he needed to do and what his AI system is capable of after a year of work.

For starters, Zuckerberg says that he needed to figure out how to communicate with all of his devices. This included reverse engineering APIs to make them play nice with his PC, in some cases, even making hardware modifications to make them internet capable. In the end, Zuckerberg says that we need common APIs and industry standards if we're going to make connected-home AI systems.

Once he was communicating with his devices through his PC, he developed ways to issue commands through natural language. This system is more than one that just recognizes keywords, as it quickly became clear that he needed his Jarvis AI to pick up on contextual clues. As the AI got better at picking up context, Zuckerberg and his wife Priscilla Chan were able to begin using open-ended requests more frequently. Zuckerberg notes that "no commercial products I know of do this today, and this seems like a big opportunity."

From here, he implemented a number of other features, whether it's a visual system that allows Jarvis to identify when his friends are the front door and let them into the house, or voice recognition that allows Jarvis to continually listen to what he says. It also sounds like this project helped him realize just how many tools Facebook engineers have at their disposal, and how well organized those engineers keep their code.

One of the more interesting features of his Jarvis AI is the Messenger bot he built for it. With it, he can use the Messenger platform to communicate directly with his system from his phone, which gives him an advantage over home device AI systems like Amazon Echo and Alexa. He also says that he found he communicated with Jarvis more through text than voice, which he think says something about the future of AI needing support for both text and voice-based communications.

Zuckerberg goes into much more depth over in his Facebook post, and if you have a few minutes to read through the whole thing, it's definitely worth checking out. From here, he says he'll continue to improve the system, which includes developing an Android app and adding more voice terminals around his house. Even though he's largely finished his vision for Jarvis, we may be hearing more about it in the future, so stay tuned.

SOURCE: Mark Zuckerberg