As the automotive industry evolves, more consumers – especially millennials and digital-native Gen Z –are seeking connected vehicles and personalized car experiences. What once differentiated vehicles, such as horsepower, design or even trunk space, is now shifting toward software capabilities. With the rise of software-defined vehicles (SDVs), the race to deliver the most connected, safe and enjoyable driving experience is becoming increasingly important for OEMs and influencing buying decisions.
SAIC Motor, a leader in the automotive industry, is at the forefront of this transformation. Their vision for the next-generation driving experience centers around leveraging cutting-edge technology to create user-centric vehicles that seamlessly integrate digital and physical environments. A critical component of this vision is their 3D HMI – a technology fueled by intuitive design that enhances in-car systems to be more adaptive and responsive, with virtual reality features to enrich user interactions.
At Star, we have been an innovation partner with SAIC for many years, collaborating on groundbreaking projects. This includes developing a scalable HMI design system for SAIC MG's global market, an in-car AI assistant, and most recently, a revolutionary 3D HMI concept. This latest concept transforms the daily commute into an immersive experience, allowing passengers to engage in interactive challenges and enjoy a vibrant, gamified interface enriched with 3D elements.
This article explores how technologies like Unreal Engine are driving innovative HMI experiences in the automotive industry. We also assess the value of 3D HMI, identifying the scenarios where it enhances user experience and how it differentiates from traditional applications.
SAIC’s vision for 3D HMI: Beyond functionality, toward immersion
SAIC understands that as vehicles become more advanced and feature-rich, there’s a risk of overwhelming users with complex interfaces. The challenge lies in creating an HMI system that balances advanced functionality with ease of use, providing an interface that is both intuitive and immersive. SAIC’s industry-leading vision for the future of in-car experiences goes beyond traditional 2D interfaces. Their 3D HMI enhances the driving experience by integrating AR-driven navigation and AI-assisted interaction to improve safety and personalization.
In a recent conversation, Yidan Bao, Product Manager at SAIC MOTOR Overseas Intelligent Mobility Technology Co., Ltd (a software development company invested in by SAIC Motor), highlighted the importance of designing an HMI that feels natural and offers meaningful interactions. “We aim to create a system where the user experience is as immersive as it is seamless,” she explained. “This means leveraging 3D technology to offer clearer, more intuitive visualizations of data, real-time surroundings and personalized driving modes that adapt to the user's preferences.”
Yidan emphasized that SAIC’s vision is to create a holistic car experience – one where drivers feel connected not just to their vehicles, but also to the world around them. Features such as augmented reality (AR) overlays for navigation, AI-driven personalization and dynamic driver assistance position 3D HMI as the gateway to a new era of driving, where technology and user experience are deeply intertwined.
Beyond these core features, VR in automotive is also playing a significant role in shaping broader mobility ecosystems. While not always directly embedded into in-car HMI, virtual reality is being leveraged by OEMs in areas like design and testing, creating immersive simulations for vehicle development. This technology helps OEMs like SAIC refine and enhance 3D HMI systems before they reach consumers, ensuring that both the interface and overall user experience are optimized for real-world conditions.
Yidan also highlighted the potential of semi-mixed reality (SMR) in blending physical and digital worlds to create new in-car experiences. With the right technological infrastructure, SMR could overlay digital information onto the real world, revolutionizing navigation, surround views and notifications – all while keeping important information within the driver’s line of sight. She believes that over the next 5-10 years, the automotive industry will experience a shift towards smarter, more connected, and autonomous vehicles. In this evolution, 3D HMI will play a pivotal role by offering intuitive and immersive interfaces that enhance user interaction with advanced technologies. As vehicles become more autonomous, 3D HMI will improve both entertainment and productivity, creating personalized environments for different purposes. This innovation will be essential in differentiating brands as vehicles increasingly function as mobile living spaces rather than just modes of transportation.
“Ultimately, our vision is to set new standards in smart mobility by blending advanced technology with user-centric design, ensuring that our vehicles are not only technologically advanced but also deeply responsive to individual needs,” added Yidan. This approach is setting a new industry benchmark.
Exploring new dimensions with 3D
At Star, we believe that 3D HMI will play a pivotal role in shaping the future of the in-car user experience, offering intuitive and immersive interfaces that enhance interaction with vehicles. Technologies like Unreal Engines with real-time 3D rendering play a key role in bringing dynamic visualizations to life. These representations not only improve safety by providing clearer views of the vehicle's surroundings but also act as key drivers of brand differentiation and user engagement.
Navigation
3D visualizations of maps and landmarks can provide a more intuitive and clear representation of the driver’s surroundings, making it easier to navigate unfamiliar areas.
Driver assistance
3D visualizations of sensor data, such as from cameras and lidar, can provide a more detailed and accurate representation of the car’s environment, improving the effectiveness of driver assistance features.
Entertainment & comfort
3D visualizations can also enhance in-car entertainment, from immersive gaming or virtual reality experiences to interactive control of interior settings, offering a richer experience for passengers.
Safety & maintenance
3D HMI can deliver real-time warnings and alerts through visual representations of hazards or obstacles, increasing driver awareness. The system can also support maintenance tasks, such as indicating when to refill fluids or replace fuses, with clear visual guides.
Dynamic surround view
By merging data from multiple cameras and sensors into a cohesive 3D model, 3D HMI can provide a dynamic surround view of the vehicle. This gives drivers an unparalleled perspective of their surroundings, aiding in parking, maneuvering, and detecting hidden obstacles.
Trends and technologies shaping the future of mobility – a 10Forward view
Throughout our 10Forward trend report research process, two key themes that consistently emerged in interviews with industry experts worldwide: Car as a third space and hyper-personalization & gamification. These trends, coupled with advances in 3D HMI technology, mark a fundamental shift in how users interact with vehicles and provide key opportunities for OEMs to lead the next wave of innovation.
Car as third space
As vehicles become increasingly autonomous, they are transforming into mobile living spaces, and becoming multifunctional environments where people can work, relax, socialize or be entertained while in transit. The implications for OEMs are profound: cars are no longer just vehicles—they are extensions of people’s digital and physical lives.
3D HMI plays a critical role in bringing this vision to life. With AR-driven navigation, AI-assisted interactions and immersive environments, 3D HMI enables vehicles to provide a seamless experience between the car and the outside world. Whether it’s real-time updates on traffic or surroundings, interactive productivity tools or entertainment options, 3D HMI creates an environment where the vehicle itself becomes a dynamic, responsive space. For OEMs, this means a new way to differentiate their brand by creating vehicles that serve as connected ecosystems.
Hypersonalization & gamification
Consumers don’t just want connectivity, they also want relevant and personalized experiences. By leveraging real-time data from the driver and various vehicle sensors, 3D HMI can elevate routine tasks such as navigation into tailored interactions.
Gamification is also becoming a key feature in the automotive space, turning everyday driving into interactive experiences. Features like interactive challenges, achievements and point systems can motivate safer driving habits or improve fuel efficiency. With 3D HMI, OEMs can create personalized, dynamic environments that enhance users’ lives, reimagining cars into more than just modes of transportation.
Co-creating the future driving experiences
At Star, we are committed to co-creating with our clients, ensuring that every project we undertake is rooted in both innovation and sustainability. As we continue to push the boundaries of HMI technology and user experience design, we look forward to driving the future of mobility together.
