Skip to main content

Facebook's Mark Zuckerberg pulls back the veil on the Jarvis AI system he created

1143461 autosave v1 2 mark zuckerberg ceo at facebook
Image used with permission by copyright holder
In a lengthy post published to his nearly 13-year-old pet project, Facebook, social network wunderkind Mark Zuckerberg has officially unveiled the ins and outs of his built-from-scratch home artificial intelligence system. Dubbed Jarvis — a nod to the AI system Tony Stark employs in Marvel’s Iron Man comics — Zuckerberg designed his personalized home automation network to make accomplishing such tasks as playing music or dimming the lights easy while also allowing the AI to personally learn his “tastes and patterns.” Though he’d personally given updates (and even run public tests) on the system in the past, his recent briefing serves as the project’s most in-depth explainer.

Built using Objective C, PHP, and Python, the Jarvis AI system is essentially an ubertalented version of Amazon’s Alexa or Google’s Home device. Capable of operating a number of smart products in Zuckerberg’s home — such as his Crestron lighting system, his Sonos and Spotify setup, and the Nest cam he uses to check in on his daughter — Zuckerberg acknowledged that one of the hardest parts of the entire process was figuring out a way for each smart home system to communicate effectively with each other.

“Before I could build any AI, I first needed to write code to connect these systems, which all speak different languages and protocols,” Zuckerberg wrote. “We use a Crestron system with our lights, thermostat and doors; a Sonos system with Spotify for music; a Samsung TV; a Nest cam for Max; and, of course, my work is connected to Facebook’s systems. I had to reverse engineer APIs for some of these to even get to the point where I could issue a command from my computer to turn the lights on or get a song to play.”

Zuckerberg talking about his Jarvis AI system in Italy over the summer. Facebook

After overcoming this initial hurdle, Zuckerberg turned his attention to allowing Jarvis to heed and interpret communication as similarly as possible to that of a human. To accomplish this, the system had to have a method for receiving text messages while maintaining the ability to translate actual speech. Zuckerberg felt the text message and speech combination was an incredibly important decision, admitting he’s surprised at how much more he texts Jarvis instead of talking to it.

“One thing that surprised me about my communication with Jarvis is that when I have the choice of either speaking or texting, I text much more than I would have expected,” he continued. “This is for a number of reasons, but mostly it feels less disturbing to people around me. If I’m doing something that relates to them, like playing music for all of us, then speaking feels fine, but most of the time text feels more appropriate.”

What may be most impressive about Jarvis’ ability to receive texts is the fact Zuckerberg built a dedicated Messenger bot that allows him to chat with the system no matter where he’s actually located. Not only does this allow him to preemptively turn on his lights but it also provides a supremely effective way to keep up with what’s going on at his house. For instance, with the inclusion of an advanced facial- and speech-recognition feature, Jarvis keeps Zuckerberg in the loop should someone stroll up to his house — known or unknown. Furthermore, if the system recognizes the person, it automatically lets them in — after taking their photo and adding it to the Messenger conversation, of course.

Though the system isn’t perfect — during an exclusive Fast Company demo, it took Zuckerberg four tries to turn off his lights — he says he’s committed to staying focused on continuously improving Jarvis since he uses it every day. Be it the addition of more voice terminals or the creation of an Android app (currently, Zuckerberg uses an iPhone to control Jarvis), there’s still plenty to be accomplished. However, from the looks of the schematic Zuckerberg published, it appears as though his system is essentially perfect. How so? Because he’s programmed Jarvis to be compatible with a T-shirt cannon that fires a new gray tee to Zuckerberg when he asks for one. If you ask us, that’s peak artificial intelligence.

Rick Stella
Former Digital Trends Contributor
Rick became enamored with technology the moment his parents got him an original NES for Christmas in 1991. And as they say…
Eve MotionBlinds Upgrade Kit and Blinds Collection join the Matter lineup
The Eve MotionBlinds Upgrade Kit being installed.

Eve is a prominent name in the smart home world, with the company responsible for a variety of devices covering everything from smart plugs and security cameras to smart light switches and motion sensors. Its catalog is growing even more enticing today, with the Eve Blinds Collection and MotionBlinds Upgrade Kit gaining Matter support.

The MotionBlinds Upgrade Kit will likely appeal to more customers, as it costs just $200 and works with most existing setups. Installation is as simple as removing your shades, placing the MotionBlinds Upgrade Kit through the roller rod, then reinstalling the unit on your windows. Blinds with a roller between 1.5 inches and 2.0 inches are compatible with the upgrade kit.

Read more
Philips Hue lights have a rare discount, but there’s a catch
A TV with ambient backlighting from Philips Hue.

One of the cool things about Philips Hue lighting is how much freedom it gives you over creating the perfect ambiance in your home. One of the cool things about buying Philips Hue products over at Best Buy right now is how much freedom Best Buy is giving you over what you buy to save money. Right now, when you buy two or more select Philips Hue smart lighting product in Best Buy's offer builder, you can save 15%. Tap the button below to start building your own bundle and see all of the products available. While they don't include the best smart light bulbs that Philips produces, you'll be amazed by the versatility of the lightstrips. Or keep reading for an overview of some of the top picks from the bundle.

What you should purchase from the Philips Hue offer builder
The place to start with this deal is the and the if you don't already have them. These are the devices that take your movies, music, and games and syncs them to your lights.

Read more
Samsung’s futuristic Jet Bot AI+ robot vacuum is $400 off
The Samsung Jet Bot AI+ Robot Vacuum cleaning the floor.

For families who need a lot of help in keeping their floors spotless, the Samsung Jet Bot AI+ is an investment that you'll want to make. It's pretty expensive at its original price of $1,300, but a $400 discount from Samsung pulls it down to $900. It's still not cheap, but this futuristic robot vacuum could be the only device that you need to keep dirt, debris, and pet hair away from your sight. You're going to have to complete your purchase for it as soon as possible though, as there's no telling when the offer expires.

Why you should buy the Samsung Jet Bot AI+ robot vacuum
The Samsung Jet Bot AI+ checks all the boxes in the features that are recommended by our guide on how to choose a robot vacuum. It's equipped with powerful sensors and object recognition technology that allows it to avoid furniture when it's cleaning your floors, while LiDAR technology allows it to create accurate maps of the rooms in your home so that it can properly navigate and clean more efficiently. The robot vacuum also features a five-layer filtration system that allows it to release clean air, and when it's done, it goes back to its Clean Station where it empties its contents.

Read more