With the Jarvis project, Zuckerberg wanted to use his voice to control everything in his house, from the music to the lights to the temperature. He also wanted Jarvis to swing open the front gate for friends by recognizing their faces. Essentially, the Jarvis project is like Zuckerberg’s homemade version of Amazon’s Alexa service or Google’s Home.
From building “Jarvis,” his personal AI-powered butler
What if your security camera could not only see who’s at your door, but also identify whether it’s a guest you’re expecting, alert you when they arrive, and let them in? Or how about a speaker system that automatically plays music as your child wakes up?
That’s the type of functionality Facebook CEO Mark Zuckerberg is trying to build into his virtual butler, Jarvis, which he’s been developing throughout the year as part of his New Year’s resolution.
With 2016 coming to a close, Zuckerberg published a lengthy blog post detailing the types of tasks Jarvis can accomplish. He also wrote about the biggest challenges that he faced when developing his artificial intelligence (AI) software, and where he believes AI is heading. One of his most significant takeaways was that, although AI is advancing quickly, it still requires a decent amount of human guidance.
“We know how to show a computer many examples of something so it can recognize it accurately, but we still do not know how to take an idea from one domain and apply it to something completely different,” wrote Zuckerberg.
Here’s a look at what else the Facebook CEO now believes about the state of AI, virtual assistants, and the potential they have for the future.
Home AI Has to Be Able to Connect Everything
Zuckerberg noted that one of the struggles he encountered while building Jarvis was creating a system that could connect all of the types of appliances and devices in his home, from Sonos speakers to a Samsung TV and Nest camera.
“For assistants like Jarvis to be able to control everything in homes for more people, we need more devices to be connected and the industry needs to develop common APIs and standards for the devices to talk to each other,” he wrote.
Some tech companies are trying to address this problem by building their own smart home platforms, like Apple with HomeKit and Alphabet through its “Works With Nest” program and its Weave language. But we’re still far from a reality in which every single appliance in the home can communicate with one another. HomeKit, for instance, is currently only compatible with devices like alarms, heating and cooling devices, thermostats, lights, and switches, not laundry or cooking appliances. And with multiple smart home platforms available, buyers usually must commit to one, limiting their functionality.
The Phone Is the Perfect Home for an AI Assistant
Digital helpers like Apple’s Siri and Amazon’s Alexa have made their way to all types of gadgets, from laptops to smartwatches, and home speakers. Zuckerberg, however, feels that mobile devices are the right vessel for AI assistants because you’re not restricted to only engaging with them when you’re home.
“I want to communicate with Jarvis when I’m not home, so having the phone be the primary interface rather than a home device seems critical,” he wrote.
Text Input Is Just as Important as Voice
While much of the tech industry has been fixated on the rise of the voice-centric interface following the success of Amazon’s Echo, Zuckerberg believes text input will still play a large role in how we interact with AI. A helpful assistant should support both text and voice input, similar to Microsoft’s Cortana.
“One thing that surprised me about my communication with Jarvis is that when I have the choice of either speaking or texting, I text much more than I would have expected,” he wrote.
