Meta's vision for AR glasses and the future of human connection

I'd love to start with these. Ten years of work right there. Someone on your team call these the real-life Tony Stark glasses. Very hard to make each one of these. That makes me feel incredibly optimistic. In a world where AI gets smarter and smarter, this is probably going to be the next major platform after phones. I miss hugging my mom. Yeah, haptics is hard. How does generative AI change how social media feels? We haven't found the end yet. The average American has fewer friends now than they did 15 years ago. Why do you think that's happening?

I mean, there's a lot going on to unpack there. I'm about to interview Meta's CEO, Mark Zuckerberg. Mark Zuckerberg. There are not that many people with more power over what our future might look like. Nearly half the total human population now uses Meta products. And I just tested some of their new tech that feels like science fiction. Stop!

This is crazy. Mark Zuckerberg and the team at Meta are imagining a future that billions of other people might actually end up living in. So my goal for this conversation is to try to figure out what that future really looks like, to paint a picture of the future Mark Zuckerberg is trying to build so that you can decide for yourself what you think of it. Welcome to the first episode of our new series, Huge Conversations.

Okay, good to meet you. Yeah, looking forward to it. Awesome. I'd love to tell you what my goal is of this conversation. Go for it. So we have a show called Huge If True, which is this very optimistic show about science and technology and the potential futures that we can build. And in every episode we're sort of exploring what does it look like if you play a certain technological future out. And so my goal in this conversation is to

Try to help people see the future that you're imagining when you're building the products that you and the meta team are building. What are you imagining this looks like in the future? How are you imagining people use this? All of that. Cool. Awesome. So I'd love to start with these. Let's do it. 10 years of work right there. I got to demo them a little bit earlier today. I heard someone on your team call these the real life Tony Stark glasses. We're getting there.

but I'd love to just hear in your voice, what are these? Well, these are the first full holographic augmented reality glasses, I think, that exist in the world. We've made, I think it's a few thousand or something, right? Very hard to make each one of these. But this is the culmination of 10 years of research and development that we've done to basically miniaturize

all the computing that you need to have glasses, not a headset, but glasses that can put full holograms into the world with a wide field of view. So you can imagine sort of in the future, we'd be having a version of this conversation where, you know, maybe I or you are not even here. It's like one of us is physically here and the other one is here as a, as kind of a full body hologram. And it's not just a video call. You can actually interact. You can do things. I mean, in the demo, we had the,

you know ping pong and games and things like that but I mean you could you can interact you can work together you can you know play poker play chess whatever like the holographic cards holographic board game I just think it's gonna be wild it's gonna remake I think so many different fields that we think about today from how we work and productivity to a lot of things around science a lot of things around education entertainment fun gaming so but this is just the beginning you know this is the first version it's like

It's a prototype version that we've made in order to develop the next version, which is hopefully going to be the consumer one that we sell to a lot of people. Why build these? Well, I think it's going to be the next major computing platform. So if you look at like the grand arc of computing over time, you've gone from like mainframes to computers that basically like live on your desk or on a tower to phones that you have in your hand that you basically like

can take with you everywhere that you want, but it's pretty unnatural, right? It takes you away from the world around you. And I think that the trend in computing is it gets more ubiquitous, it gets more natural, and it just gets more social, right? So you want to be able to interact with people in the world around you. And I think that this is probably going to be the next major platform after phones. Look at these two. These are the clear ones that show all the technology. Yeah, the whole thing is a special edition, and this is like a really special edition.

there's not a single millimeter of space. It's, you know, everything in here from the micro projectors that basically shoot light into the wave guides, right? It's a special type of display system. I mean, these aren't normal displays like you have in a phone or a TV or computer, like the type of displays that people have been building for decades. It's a wave guide system. The projector that's shooting light

basically goes into these nano etchings across the waveguide that are what catches and creates the holograms in order to synchronize that with your where you're looking there's eye tracking and little cameras they illuminate your eyes and then of course there's all the basic stuff that you need all the computing the batteries to power the whole thing microphones the speakers because it needs to be able to play audio and speak with you

and the cameras and sensors to see things around you in the world so that way when it's placing holograms in the world it can do that in the right place and understand where you are so that probably is still not covering everything because there's a lot of things that need to go into syncing up the holographic images between the two displays because you don't just have a single display like you have in a phone or TV you have two and it moves around and physical things are hard and need to be synced up

there's also the radio that has to communicate with your other computing devices to do heavier computing and the wrist-based neural interface that you probably got to try out we kind of miniaturized all of this and fit it into a you know normal looking pair of glasses which is uh you know when i told the team that we were going to do this 10 years ago um you know people weren't sure if we were going to be able to but i think you know not only we're going to be able to do this but i think we're going to be able to get a cheaper and higher quality and

even smaller and more stylish over time. So I think this is going to be a pretty wild future. There are so many versions of trying to get at a similar idea of digital objects in physical space. I'm thinking of, for example, glasses that have heads-up displays where it's headlocked and it's moving with my eyes. Glasses that are really creating digital objects in physical space that don't move as I move.

I'm thinking of these. I'm also thinking of the Snapchat spectacles that they just announced. Then on the other hand, there are headsets like the Quest and also like the Apple Vision Pro that seem to fall into a different category. I'm curious how you would organize this landscape for people and how you think about people using these tools in their real lives in the near future. Yeah. So when we were getting started on this about 10 years ago, I thought that something like this was going to be the ultimate product for everyone, right? You get to

um you know normal looking pair of glasses and we'll continue improving that that can have full holographic images yeah um i think it's it's super powerful and it is sort of the science fiction future that i think we all hope to get to um on the journey we took a few other approaches as well um to help us develop towards that including building glasses that don't have displays um

to try to learn, just take a stylish pair of glasses today and put as much technology into it as you can, but really focus on the form factor. And that's the Ray-Ban meta glasses and it's doing really well. And initially we thought that that was sort of intro product for us to learn how to build this. But one of the things that's clear now is you're going to be able to make that product a lot more affordable than this.

Probably permanently. So I actually think that there are going to be a bunch of different of these paths that we've taken are going to be kind of permanent product lines that people will choose. I think you'll see displayless glasses like the Ray-Ban Meadows continue to get better and better. Great for AI. No display, but you can talk to it. It can talk back. I know there's going to be something between these.

that's basically a heads up display. So it's not a 70 degree field of view. Maybe it's a 20 degree or 30 degree field of view. So that's not going to be what you want for putting kind of a full hologram of a person or interacting with the world around you, but it's going to be great for, you know, when you're talking to AI, not just having voice, but also being able to see what it's saying or, um, being able to text someone with your risk-based neural interface,

and then have their text show up rather than having it read to you, which is we read faster than we can listen. Or getting directions, right? Or just being able to search for information and get all that. So there's a lot of value for a heads-up display. That will be somewhat more expensive than the displayless glasses, but somewhat cheaper than this. Then I think you're going to get this. It's going to be probably the most premium and expensive of the glasses products, but hopefully still something that, you know, like a computer...

is generally accessible to most people in the world, but I think that there are going to be all of those, and I think people will like them. I also think that the headsets that people are using around mixed reality will continue to be a thing too, because no matter how good we get at miniaturizing the tech for this, you're just going to be able to fit more compute into a full headset. Fundamentally, our mission is not build something that is advanced and only a few people can use. We want to take it

in the last mile and do all the innovation to get it to everyone we you know just shipped or announced quest 3s the new mixed reality headset we basically are delivering high quality mixed reality for 299 dollars i was really proud last year when we delivered quest 3 the first kind of really high quality high resolution color mixed reality device um for 500 dollars right it was like

It's like a fraction of the cost of what the competitors are doing. And I think it's actually higher quality in a lot of ways. And now we've just doubled down on that. So I think that they're all actually going to end up being important long-term product lines. Display lists, heads-up display, full holographic AR, full headsets. I think that they're all going to be important. Yeah. If you play out the future of not just the hardware that we've been talking about, so...

MetaRay bands, Quest, Orion, but also the Llama models. If everything goes according to you and the team's wildest dreams. I'd love for you to just begin to describe what that feels like. I mean, I think that there are two primary values that we're trying to bring. On the AR and kind of mixed reality side, the main value we're trying to bring is this feeling of presence, right? So there's something that I think is just really deep.

about being physically present with another person that you don't get from any other technology today. And I think that that's the thing when people have a very visceral reaction to experiencing virtual or mixed reality, what they're really reacting to is that they actually, for the first time with technology, feel a sense of presence. Like they're in a place with a person and that's super powerful. Like I've focused on designing social apps and experiences for 20 years.

that's sort of like the holy grail of that is being able to build a technology platform that delivers this like deep sense of of social presence the other big track is around personalized ai and for that and that's sort of where llama and med ai and all those things are going there's all this development that's going into making the models smarter and smarter over time but i think where this is going to get really compelling is when it's personalized for you and

in order for it to be personalized for you, it has to have context and understand what's going on in your life, both kind of at a global level and like what's physically happening around you right now. And in order to do that, I think that glasses are going to be the ideal form factor because they're positioned on your face in a way where they can let them see what you see and hear what you hear, which are the two most important senses that we use for, for kind of taking an information and context about the world.

I think that this is all going to be kind of really deep and profound stuff. But it's basically those two things. It's this feeling of presence and this capability of really personalized intelligence that can help you. I'd love to talk about each of those two things. The first on presence, I owe a lot to being able to connect with people online, right? This job that I have is by definition there. Also with my family. My parents don't live anywhere close to me. I video call them a lot. And when I think about

the progress of technology like this in a timeline from the telegram to the telephone to video call to some feeling of presence with another person who feels like they're right there in front of me. That makes me feel incredibly optimistic. I would love a future where I can lose in Scrabble to my mom and feel like she's really there in front of me. And it feels like we're not that far away from something that persuades my brain that that's happening.

And also, I miss hugging my mom. Right? Like, that never goes away. Yeah, haptics is hard. Yeah. And so my question is, my question is about that. It's about this feeling of like, it's hard for me to imagine a future where real physical presence is not different and special in some way, where I don't miss literally hugging my mom. Yeah. And I'm curious how you think about

the parts of human connection that are eye contact and physical touch and things that our eight brains value for connection with other people yeah well eye contact i think we're going to get to a lot before the the touch part for haptics i do think we'll make progress on that but it's it's obviously there's a spectrum there too from kind of hands which is where if you if you draw out the

kind of, like, homunculus version of a person in terms of, like, what are our kind of sensory, you know, what's, like, the majority of what we're sensing. Yeah, it looks funny. Yeah, yeah, yeah. So I think being able to do that for your hands is...

probably the most important place to start and you have a rough version of that with controllers today um i think that that'll get even more over time we have this demo playing ping pong where you have a controller where as the digital ball hits the ping pong paddle you feel it hit the as if it's hitting the ping pong paddle wherever it is so you actually have a sense of like where it's it's hitting the the the paddle so i think that that was that was just a wild demo so i think we'll get some of that

the most extreme version of this is wanting force feedback right so I mean like for doing a lot of sports right it's like okay we can kind of do a good approximation of like boxing today where you get like good feedback on your hands but it would be hard to do a virtual reality version of jujitsu where you're like grappling with someone and you need like real kind of force feedback on it so that's probably like the hardest thing to go do but I think we'll get there you know I think like

most science fiction it's not this binary thing that you just like wake up one day and we're like oh we've realized all the dreams but but i i do think that these platforms are going to be the first time that i think that there's a realistic sense of presence in all the ways that that's special to people um for most things that people want to do which are not the most physical ones and even some of the basic physical ones i think we'll get

But then there's a long tail of other stuff. I mean, smell is also really important for people. I think it's disproportionately important for memories. And that's not really a thing that I think in the next few years we're going to have in any of these devices. I mean, that's a very difficult and challenging thing on its own. What is the piece of that that you feel most interested in that you keep coming back to in your mind? This has the frustrating property to develop that

the sense of presence