Mixed Reality with Microsoft Mesh

Microsoft Mesh is a new mixed-reality platform powered by Azure, with which people in different physical locations can collaborate and share holographic experiences on different types of devices. It uses 3D capture technology to beam life-like images into a virtual scene. With this technology, people can experience events as avatars while they unfold in a shared holographic world. Developers can build immersive, multiuser, cross-platform mixed reality apps with Microsoft Mesh, and users can enhance virtual meetings, conduct virtual design sessions, assist remotely, learn together virtually, host virtual social gatherings, and meet-ups. Let us understand Microsoft Mesh and how it will revolutionize mixed reality (MR) in the time to come.

What Are the Capabilities of Microsoft Mesh?

Microsoft Mesh is designed to solve challenges that prevent developers from enabling MR experiences for multiuser scenarios, keeping holograms stable in shared MR spaces across time and devices, bringing high-fidelity 3D models into MR, and synchronizing people’s actions and expressions in a geographically distributed MR session. Here are some of the capabilities of Microsoft Mesh.

Providing Immersive Presence

Participants have to be represented in distinct forms based on their devices in multiuser MR scenarios. With Mesh, it is possible to create representative avatars using inside-out sensors of the devices. The avatar rig and a customization studio in the Mesh platform can be used for creating out-of-the-box avatars. AI-powered motion models are used to capture motions and expressions of users. Similarly, outside-in sensors help in enabling photorealistic 360-degree holoportation that can be used within Mesh-enabled apps on immersive mixed reality headsets, smartphones, PCs, and tablets.

Creating Spatial Maps

Mesh can be used to merge and optimize spatial/ environmental information in local caches in multiple devices and build 3D models and share device point-of-views. The holographic content in spatial maps persists across time, space, and devices, and provides a common perspective of the space around each participant, along with an understanding of the physical world.

Supporting Holographic Rendering

In most cases, the local device’s computing and thermal budget have a major role in effective holographic rendering. However, Microsoft Mesh provides developers to choose between local-standalone rendering and cloud-connected remote rendering within their app for each scene and model. This lends flexibility to design apps that can optimize for latency vs fidelity based on the device used by the end-user. Additionally, Mesh supports most 3D file formats allowing users to render their existing models natively in Mesh-enabled apps.

Enabling Multiuser Sync

Multiuser sync in Mesh enables a shared context for creating a common perspective of the hologram among users in a collaborative session. Changes in poses, motions, expressions, and other holographic transformations, including augmentation with spatial audio, happen within 100 milliseconds of latency regardless of the physical space in a multiuser scenario.

Apps4Rent Can Help with Microsoft Cloud Solutions

With Mixed Reality going mainstream across consumers and commercials, screen-bound experiences have already started giving way to instinctual interactions in space and among things with people, and Microsoft Mesh will play a fundamental role in facilitating the creation of immersive experiences. Microsoft is introducing Mesh in HoloLens Mesh App, Partner Apps, and AltspaceVR (a solution that introduces new enterprise capabilities) and will be enabling it in Microsoft Teams and Microsoft Dynamics 365 soon.

As a Tier 1 Microsoft CSP with Gold Competency in specializations such as Cloud Platform and Cloud Productivity, Apps4Rent helps businesses adopt and implement Microsoft cloud solutions with managed Azure services and other offerings. Contact our Microsoft certified cloud practitioners available 24/7 via phone, chat, and email for assistance.