Designing the Future: A Journey into the Revolutionary UX of the Mercedes Vision AVTR

Vera Schmidt
Director Advanced UX Design
Automatic Summary

Design and Innovation at Mercedes Benz: A Focus on User Interface

Welcome to the world of Mercedes Benz's Design and Innovation, where we constantly push the boundaries of new technological solutions in an effort to create desirable, cutting-edge vehicles. Our main focus? The user interface of the car. How does the driver interact with the car? What's the input? What's the output? This involves not only the interface for the production cars that customers can purchase, but also future vehicles. We explore questions such as: What kind of technologies will be available in the future? What kind of impact will AI have on our design process and the user experience within the car?

Our Stance: A Vision of Human-Centered Design - The Mercedes AVTR

The Mercedes AVTR is our vision of human-centered design. A product of a global collaboration between Avatar, one of the most innovative film brands in the entertainment industry, and Mercedes Benz, the AVTR is about a symbiosis between the human, the machine, and the nature around us. The autonomous concept vehicle enables a completely new interaction between man, machine, and nature. We aimed to create a vehicle that extends our senses and make us aware of our world's beauty that we might ignore or not experience in the typical way.

Our Method: The Human-Centered Innovation Approach

At Mercedes Benz, our method of choice is human-centered innovation. The explosion of features due to technological advancement brings with it a dire need for radical simplification on the user's side. UX and UI design are major drivers for this innovation, simplifying the information that the driver needs to interact with.

The Inside: A Look into the Mercedes AVTR

With its many technological features, the Mercedes AVTR is designed to be an immersive experience. Some highlights include:

  • Haptic feedback in the seats
  • A curved dashboard for ample space for projection
  • Merge control for a completely new navigation interface
  • 3D spatial sound
  • A magic pool for the child seat to connect

The experience, of course, begins even before you enter the car. On approach, the car recognizes you and wakes up, signaling readiness for a ride. The interior lights up, the sensors prepare, and a living organism is designed.

The Magic: How interaction with the car works

The Mercedes AVTR offers a new way to interact with your vehicle. Once inside, you can place your palm on top of the merge control, synchronizing yourself with the car. Your profile is set up, your heartbeat detected, and the car can see if you are nervous or stressed and then adapt to cater to your needs.

Making Use of Innovation in Car Entertainment

The advent of features such as movement in car interaction make car entertainment truly enchanting. The goal is to include the movement of the car to any entertainment features, providing a unique user experience compared to stationary home entertainment systems such as TVs. If you have more questions or are intrigued by our innovative approach to car design, feel free to connect with us on LinkedIn. We love to interact and exchange ideas, and we are always on the lookout for talent.


Video Transcription

Pleasure for me to give you a glimpse into our design and innovation approach at Mercedes Benz. Currently I am located here in our headquarter design headquarter in Zindler, which is near Stuttgart in Germany.And I want to give you a small yeah, transparency about what we do here in our center. We are focusing um for on the user interface of the car with our engineering colleagues. So I'm a designer and it's our design department design center and um we are focusing on the interface. How do you interact with the car? Uh what's the input, what's the output, what's the interface, it's for the production cost customers can buy. But at the same time and it's especially in my department, we are focusing also on the future vehicles. So how do you as a human interact with the car in the future? What kind of technologies will be available? Um You heard about A I today already, what kind of impact will have A I on our designing uh the human uh user experience uh within the car, but also beyond the car. So um today and for in this part, as especially we are constantly push the boundaries to find and visualize a new technological uh solutions uh for desirable future. And we are creating cars like you will see in a moment to ex make our ideas and prototypes experience.

So in, in my talk today, I will introduce you to the vision AVTR. It's a car where our vision of Human Centered design becomes very visible and tangible. In short, it becomes experience. The Mercedes AVTR is the result of a very extraordinary global collaboration between Avatar. It's uh you guys might know Avatar. It's one of the most innovative film brands in the entertainment industry. And of course, a Mercedes Benz, the name of this groundbreaking concept vehicle stands for advanced Vehicle transformation. And I have to say from my personal experience, it's one of the best collaboration I ever experienced such a creative uh people from the film industry to collaborate with them from a very different field. It was a very highlight. It's one of my highlights in my career to be honest and, and some of the slides I will show you in a second, you will see how we got inspired by the Avatar movie and the ideas of the creatives. So uh division AVTR is about a symbiosis between the human, the machine and the nature around us and the biometric connection, the autonomous concept vehicle. It's, it's driving autonomous, enables a complete new interaction between man machine and the nature and we ask ourselves.

So what if our vehicle would extend our senses and make us aware of our world's beauty which we might not see or might not experience in this way. Looking ahead, it's all about reducing the whole complexity by expanding the functionalities that matter. Our method of choice is here, the human centered innovation, we have an explosion of features due to the technology that brings uh about the need for radical simplification on the user side. And the UX and U I design is one of the major driver for this innovation. Here, we have to, yeah, simplify uh the the information, the content you want to interact with the user. Naar is often often interacting as an operator. You are approaching the machine. So you are reaching out to a display, you're touching the screen, you are approaching the machine.

So this is something you are touching called metal, the glass. But our vision here is how could the machine let us explore and support us in our desires, adjusting to how we interact with our environment. So it's not longer that the man and woman adapting to technology, but the technology is embracing the mankind. It's as if the machine is reaching out to you. And so you will be part of the interface and we called it from hm I to hmm Human machine merged for us. The digital luxury also means that we can support each individual needs and personal preferences and create an interface that can adapt, that can extend your senses and it's uh can generate an incredible immersive experience. It's very personalized. It's adapting to your, to your needs as a human.

Well, the experience of course already starts before you enter the car, you're approaching the car that recognizes you. It wakes up, it's signaling that it becomes ready to go on a ride with you. You see also the interior is lighting up, the sensors are getting ready. There are a lot of sensors in there and there, it's introducing you really to enter the car for us. It really means that we are designing kind of a living organism. If you look at the back of the car, you see like these 33 movable flags, they're integrated. And what they can do is they're reacting of your movements. So if you are approaching the car from the left or the right, the flaps start moving. So it's almost like this, this creature can detect you and start communicating with you. And uh of course, it cannot only sense you, you see the different colors here. But also if you're sitting in inside a vehicle, you can communicate to your environment, to other cars. It's not like a stiff yeah metal anymore, but it starts to really reacting to environment communicating.

Um And especially if the car in the future can drive autonomous. Uh that would be very interesting feature um to develop if you are looking into the inter we integrate it very different technology. So it really feels like this is an organism you can communicate to the car, can sense you, but it also can react to you. So it become the whole inter becomes really an immersive experience space. And I want to explain you what's inside a car because it's full of technology we have, for example, in inside the seats, we have haptic perception in the seats. There are sensors detecting your heartbeat. There are sensors which uh can um or technology which is vibrating you giving you haptic feedback. We have a curved dashboard with for a lot of uh space for projection, we have a merge control and we have a very natural interface which is completely new and visual extension for the dashboard screen. We have 3d spatial sound. It's also very, very interesting source for your senses and we have a very interesting magic pool for the child to connect.

And uh I would jump to some of these to explain you further what our ideas were, how we can design user experience in the future to a more human centered approach. So if you are entering the car you saw there was no steering wheel, it looked pretty empty. The only thing you saw was like um we call it merge control. And once you're approaching the car, the car starts to wake up, we call it wake up. It's more like an organism and this merge control started to pulsate. It's like you're breathing, it starts to become alive and it's uh showing you it's ready, it's awake, it's detecting you. It's sensing the environment. Once you sit inside the car, you can put your palm on top of the merge control. And that way you are synchronizing yourself and you see the moment here with the car. So your profile is getting set up, uh your heartbeat is detected. The car can see if you are nervous. If you are not stressed, it can then uh adapt to your needs. But at the same time, we said the car is driving mostly autonomous. But if you don't want to drive autonomous, you want to steer the car. You can use this merge control to uh to move forward to um accelerate or to break, to turn left, to turn right. That's our new um human machine merge and we call this uh element merge control. It's for us more, yeah, more adaptive to how we could interact with the car. It's not the steering wheel anymore. You have to turn. If you move on.

One of the very interesting features is one of my favorites in this car is that we integrated the hand projection before. Remember you said we said um usually you are the one who's reaching out to the machine. You, you touch a touchscreen or you're trying to turn a knob, you're trying to turn the steering wheel But how about if the car is an interface is really, is approaching its more reaching out to you. So you don't have to reach out to stretch out, especially in a car. You know why you are driving? It's, it's difficult anyways. But for example, if you, you raise your hand, the projection, the interface will be projected onto your palm. And even if you move your hand left and right, the interface will follow you, it will stay and stick to your hand. And for us, it's very interesting because suddenly you're not touching a cold glass or like a material, but you're touching or you're feeling your own warm, soft skin, which is very different in how we interact. Uh usually. And by uh if you want to select the icon, you can just um close your hand hand and uh select the desirable um or desired icon for the interface which you see in the font moving forward. We see a very luxurious uh dashboard.

This dashboard is really huge, not only in front, but you can see and it's really expanding into the back of the of the rear. So to the children's seats and uh that way, the whole information which is coming in the front can float into the magic pool in the rear where the kids can interact and play with it. But it was very interesting to create an immersive ecosystem within the car where the information can really come onto the back of the little seat you see on the left, on the magic pool you see on the left side and the kids can play with it and then they act and really experience the environment around them with very interesting information for us.

It's also that the whole interior should become kind of a digital membrane. So we want that the car is extending our senses. There are things we might not see, there are things we might not hear that's we integrated uh the 3 3d spatial sound. So for example, if an element maybe in in deer or another animal can or is really um walking close to us or flying, maybe this is something we can hear because the the car can, can give the sound to it or the seat is V bring on the sound to know. Hey, on the left side, there is something happening. Let's see what we uh created for the huge dashboard. We think it's uh can be very interesting to use the car, as we said as a membrane to visualize maybe information we usually are not able to see. So maybe you can see and the car is detecting with all the sensors, the car has the environment. And for example, you can see nature forces things like the magnetic field or the air flow energy flows, which you are usually not seeing with your own eyes, but the car can see it. Maybe it's something interesting it can detect to you and show you the environment and together with the sound and with the vibration and a seat, it would get really an immersive immersive experience.

At the same time, you said, well, we are driving around with the car but maybe there are areas you can't drive around just because there are no roads or it's a secured nature you shouldn't destroy. But how about your drive through it virtually. So what we could um imagine is to extend the view to places which you can't really visit. So you are visiting, for example, the pink lake um or different very special um natures and places within the, within the earth. And of course, because we are collaborating with avatar, he said, hm, maybe there are views usually or perspectives you can't see as an example from nature. We have bees and bees see very different colors and structures on a flower or an environment which we as human can't see.

How about we can jump into this perspective. And in the collaboration, we use the uh from, from the Pandora, the Dire horse or for example, the mountain Benji. And then you can fly through the area like the mountain Benji or you run through the area like a dire horse for us just to experience this and show what kind of yeah ideas there could be how you could really use the car to drive through the environment would be very interesting at the end, I would say for all these interaction you see in the car and the designs, the new approaches for the, for the Human, for the Human Centered design and the aesthetics we generated.

It's almost like a piece of art and you almost sit inside it as a piece of art and it, it was a great collaboration and gave us a huge impact also on what we could do and how we can design and engineer our cars in the future. Thank you for your time. Thank you for listening. It was a pleasure. Thank you so much for

your presentation. I was very impressed by the design and the presentation and I just see how much effort and thought you put into coordinating different parts of the product project together and truly inspiring. It's like you, you are at the forefront of the transformation, innovation and the way changing the way we uh drive and agreement and, and um you know, um totally rethinking the future. Thank you so much. Um Here's a question. And do you, how do you envision in car entertainment by uh uh how do you vision in car entertainment by Ilena Praca?

So it's, it's, it's really what we just said, the car entertainment. I mean, the, the, the, the special thing on a car is it's, it's moving right? You're not sitting on the couch and watching TV, you are moving for an environment. So for us, it would be really interesting to include the movement of the car to any entertainment features. If it's around you, if you're wearing glasses, whatever it is to really use the situation that you are in a car moving around would be for us. Uh Very interesting.

Yeah, thank you so much. Do you have any more questions? Let me also have a look. I saw that people were thanking you in the chat and I'm truly honored and happy to have you with us today at the Chief in Tech Summit. What is the best way for you to follow, to follow up with you if people have more questions? So maybe we're a bit shy to ask them now in the chat,

you guys, I'm, I'm on linkedin. You can reach out to me anytime and uh we have different design studios around the world. We have huge entering parts. So we would love to interact and exchange and of course, we're also looking for talent. So I'm very happy about it.

So connect with Sarah Schmidt. Don't be shy and maybe who knows tomorrow, you're working at Mercedes and it's you presented at the Women in Tag Global Conference. Yeah, it's, I think it's very exciting to be working on your team and I'm sure many people will, you know, explore these opportunities and yeah, thank you so much for being with us today. Truly impressed with your design and presentation and everything that you do. Uh You see us So thank you so much

for take care. Have fun.