By democratizing access to scalable computation, storage, and networking infrastructure and services, cloud computing is enabling a new era of data and AI. Organizations can now collect data on an unprecedented scale and utilize it to train complicated models and produce insights thanks to the cloud.

While the increased need for data has opened up new opportunities, it has also raised privacy and security issues, particularly in regulated areas such as government, banking, and healthcare. Patient records, which are used to train models to assist physicians in diagnosis, are one area where data privacy is critical. Another example is in banking, where models used to assess borrower creditworthiness are being developed using more comprehensive information such as bank records, tax filings, and even social media profiles. To guarantee that this data remains private, governments and regulatory organizations are enacting strict privacy rules and regulations to control the use and sharing of data for AI, such as the General Data Protection Regulation (GDPR) and the planned EU AI Act.

Commitment to a confidential cloud

Microsoft acknowledges that trustworthy AI necessitates a trustworthy cloud—one with built-in security, privacy, and transparency. Confidential computing is a critical component of this vision—a collection of hardware and software capabilities that provide data owners with technical and provable control over how their data is shared and utilized. Confidential computing is based on trusted execution environments, a novel hardware abstraction (TEEs). Data in TEEs is encrypted not just at rest or in transit, but also while in use. TEEs also provide remote attestation, which allows data owners to remotely validate the setup of the TEE’s hardware and software and authorize specified algorithms access to their data.

Microsoft is dedicated to creating a secure cloud, where secure computing is the default for all cloud services. Azure now provides a rich confidential computing platform that includes various types of confidential computing hardware (Intel SGX, AMD SEV-SNP), core confidential computing services like Azure Attestation and Azure Key Vault managed HSM, and application-level services like Azure SQL Always Encrypted, Azure confidential ledger, and confidential containers on Azure. However, these options are restricted to the use of CPUs. This presents a problem for AI workloads, which rely significantly on AI accelerators such as GPUs to deliver the speed required to handle enormous volumes of data and train complicated models.

The Microsoft Research Confidential Computing group identified this problem and proposed a vision for confidential AI-powered by confidential GPUs in two papers, “Oblivious Multi-Party Machine Learning on Trusted Processors” and “Graviton: Trusted Execution Environments on GPUs,” which we share in this post. We also go through the NVIDIA GPU technology that is assisting us in realizing this goal, as well as the partnership between NVIDIA, Microsoft Research, and Azure that allowed NVIDIA GPUs to become a part of the Azure confidential computing ecosystem.

Vision for confidential GPUs

TEEs may now be created using CPUs from companies like Intel and AMD, which can isolate a process or an entire guest virtual machine (VM), essentially removing the host operating system and the hypervisor from the trust barrier. The goal is to extend this trust barrier to GPUs, allowing CPU TEE programs to safely offload computation and data to GPUs.

Unfortunately, expanding the trust boundary is not simple. On the one hand, we must protect against a variety of attacks, such as man-in-the-middle attacks, in which the attacker can observe or tamper with traffic on the PCIe bus or an NVIDIA NVLink connecting multiple GPUs, and impersonation attacks, in which the host assigns an incorrectly configured GPU, a GPU running older versions or malicious firmware, or one lacking confidential computing support for the guest VM. Meanwhile, it must guarantee that the Azure host operating system has sufficient authority over the GPU to carry out administrative operations. Furthermore, the additional security must not impose major performance overheads, raise thermal design power, or need significant modifications to the GPU microarchitecture.

According to the research, this vision may be accomplished by equipping the GPU with the following capabilities:

• A new model in which all important GPU state, including GPU memory, is isolated from the host.
• A hardware root-of-trust on the GPU chip that can provide verifiable attestations capturing the GPU’s entire security-sensitive state, including all firmware and microcode.
• GPU driver extensions for verifying GPU attestations, establishing a secure communication channel with the GPU, and transparently encrypting all communications between the CPU and GPU.
• All GPU-GPU connections over NVLink have transparently encrypted thanks to hardware support.
• Support for securely attaching GPUs to a CPU TEE in the guest operating system and hypervisor, even if the contents of the CPU TEE are encrypted.

Confidential computing with NVIDIA A100 Tensor Core GPUs

With a new technology called Ampere Protected Memory (APM) in the NVIDIA A100 Tensor Core GPUs, NVIDIA and Azure has made a critical step toward fulfilling this ambition. We detail how APM enables secret computing within the A100 GPU to ensure end-to-end data privacy in this section.

APM introduces a new confidential mode of execution in the A100 GPU, which marks a region in high-bandwidth memory (HBM) as protected and helps prevent leaks through memory-mapped I/O (MMIO) access into this region from the host and peer GPUs. Only authenticated and encrypted transmission to and from the area is authorized.

The GPU can be paired with any external entity, such as a TEE on the host CPU, in confidential mode. The GPU features a hardware root-of-trust to enable this pairing (HRoT). NVIDIA assigns the HRoT a unique identification and a related certificate throughout the manufacturing process. The HRoT additionally provides authenticated and measured boot by measuring the GPU’s firmware as well as the firmware of other microcontrollers on the GPU, including a security microcontroller known as SEC2. In turn, SEC2 can issue attestation reports that incorporate these measurements and are signed by a new attestation key that is validated by the unique device key. Any external party can use these reports to confirm that the GPU is in confidential mode and running the latest known good firmware.

The NVIDIA GPU driver in the CPU TEE checks if the GPU is in confidential mode when it boots. If this is the case, the driver requests an attestation report and verifies that the GPU is a genuine NVIDIA GPU with known good firmware. Once authenticated, the driver opens a secure channel with the SEC2 microcontroller on the GPU, employing the SPDM-backed Diffie-Hellman-based key exchange protocol to generate a new session key. When that exchange is finished, the GPU driver and SEC2 both have the same symmetric session key.

The GPU driver encrypts all subsequent data transfers to and from the GPU using the shared session key. Because CPU TEE pages are encrypted in memory and therefore unreadable by GPU DMA engines, the GPU driver creates pages outside the CPU TEE and writes encrypted data to those pages. The SEC2 microcontroller on the GPU is in charge of decrypting the encrypted data sent from the CPU and transferring it to the protected zone. Once the data is in cleartext in high bandwidth memory (HBM), GPU kernels can freely use it for computation.

Accelerating innovation with confidential AI

The introduction of APM is a significant step toward attaining greater use of secure AI in the cloud and beyond. APM is the fundamental building piece of Azure Confidential GPU VMs, which are currently in private preview. These VMs, developed in conjunction with NVIDIA, Azure, and Microsoft Research, have up to four A100 GPUs with 80 GB of HBM and APM technology, allowing customers to run AI workloads on Azure with increased security.

However, this is only the beginning. We are excited to take our partnership with NVIDIA to the next level with NVIDIA’s Hopper architecture, which will let clients ensure the confidentiality and integrity of data and AI models in use. We think that secret GPUs can allow a secure AI platform on which many businesses may collaborate to train and deploy AI models by pooling sensitive information while maintaining complete control over their data and models. A platform like this can unleash the value of enormous volumes of data while protecting data privacy, allowing enterprises to drive innovation.

Bosch Research, the company’s research, and advanced engineering branch is building an AI pipeline to train models for autonomous driving. Personal identifying information (PII), such as license plate numbers and people’s faces, is common in most of the data it employs. Simultaneously, it must comply with GDPR, which needs a legal justification for processing PII, namely data subjects’ consent or legitimate interest. The former is difficult since it is nearly hard to obtain consent from pedestrians and drivers captured by test cars. Relying on real interest is also difficult since it involves demonstrating, among other things, that there is a less invasive manner of attaining the same purpose. This is where confidential AI shines: Using private computing may help decrease risks for data subjects and data controllers by restricting data exposure (for example, to specified algorithms), allowing businesses to train more accurate models.

Microsoft Research is dedicated to collaborating with the secret computing ecosystem, including collaborators such as NVIDIA and Bosch Research, to increase security, allow smooth training and deployment of confidential AI models, and power the next generation of technology.

Microsoft announced today the addition of 12 new languages and dialects to Translator. With these upgrades, the service can now translate between more than 100 languages and dialects, making text and document information available to 5.66 billion people globally.

“One hundred languages is a good milestone for us to realize the goal of allowing everyone to interact regardless of language,” said Xuedong Huang, Microsoft technical fellow and Azure AI chief technology officer.

Today, translators cover the most commonly spoken languages in the world, including English, Chinese, Hindi, Arabic, and Spanish. Advances in AI technology have enabled the business to expand its language library with low-resource and endangered languages, such as Inuktitut, a dialect of Inuktut spoken by around 40,000 Inuit in Canada.

Bashkir, Dhivehi, Georgian, Kyrgyz, Macedonian, Mongolian (Cyrillic), Mongolian (Traditional), Tatar, Tibetan, Turkmen, Uyghur, and Uzbek (Latin) are the new languages and dialects that have taken Translator beyond the 100-language mark.

Removing language barrier

Thousands of organizations throughout the world have used Translator to connect with their members, workers, and clients. For example, the Volkswagen Group uses machine translation technology to service its users in over 60 languages. Each year, the effort includes translating more than 1 billion words. The firm began with conventional Translator models and is now fine-tuning these models with industry-specific terminology utilizing the Translator’s custom functionality.

According to Huang, the ability for enterprises to fine-tune pre-trained AI models to their unique needs was central to Microsoft’s ambition when it released Azure Cognitive Services in 2015.

Azure Cognitive Services, in addition to language, contain AI models for voice, vision, and decision-making activities. These models enable enterprises to use capabilities such as Optical Character Recognition, a Computer Vision technology (OCR). This service retrieves text input on a form in any of the Translator’s more than 100 languages and utilizes it to create a database.

“We celebrate not only what we have accomplished in translation – reaching 100 languages – but also in voice and OCR,” Huang remarked. “We aim to break through language boundaries.”

Multilingual model

According to Huang, the cutting edge of machine translation technology at Microsoft is a multilingual AI model dubbed Z-code. The model blends various languages from the same linguistic family, such as Hindi, Marathi, and Gujarati from India. As a result, the different language models learn from one another, reducing the amount of data required to create high-quality translations. For example, when the translation model is trained with related French, Portuguese, Spanish, and Italian data, the quality of translations to and from Romanian improves.

“We can exploit the shared transfer learning potential and apply it to develop the whole language family,” Huang added.

The lowered data requirements also allow the Translator team to create models for languages with limited resources or that are endangered owing to declining native speaker numbers. Several of the languages supported by translators have limited resources or are endangered.

Huang stated that Z-code is part of a bigger effort to merge AI models for text, vision, audio, and language in order to create AI systems that can talk, see, hear, and comprehend, and hence complement human skills more effectively. The continuous release of additional languages produced using multilingual model training technology, he claims, is proof of this so-called XYZ-code vision coming into focus.

“It’s bringing people together,” Huang added. “Because of our XYZ-code vision, this capacity is now in production.”

The ease and speed with which we may move around a city have a huge effect on individuals who live and work there, affecting our livelihoods, wellness, and quality of life. Mobility is critical to the efficient operation of cities.

According to Heiko Huettel, Microsoft EMEA’s automotive head, innovation enabled by a mix of digital technology and collaboration between local governments, transportation providers, and mobility firms will help offer the answers required. Mobility is evolving into a smart platform, fueled by connection, real-time data, and artificial intelligence.

Cities are growing in size and in importance

According to the latest TomTom Traffic Index, traffic has been at all-time lows in many places around the world since the beginning of the pandemic, with one-fifth (19%) less congestion globally and congestion down by an average of 26% during rush hour (Europe: 24%; North America: 40%; Asia 11%). However, the roads will not remain calm for long. TomTom’s vice president of traffic and travel, Ralf-Peter Schäfer, believes it will recover.

“That is why now is the moment for city planners, policymakers, businesses – and drivers – to assess what they will do to make future roads less crowded,” he says.

According to UN figures, more than half (54%) of the world’s population already lives in cities, with that figure expected to climb to 70% by 2030.

Economic potential is a big draw for cities. New York City contains more wealth than the whole country of Canada. London has the same GDP as the Netherlands, and if Tokyo were a country in its own right, it would be the world’s 15th largest economy.

Cities have typically become more economically productive due to the concentration of enterprises, talent, and other resources. London, for example, has the greatest productivity in the United Kingdom, more than 50% more than many of the country’s former industrial heartlands, according to the Institute for Public Policy Research.

Cities are facing mounting challenges as mobility hits the buffer

According to Open Data Institute research, some cities are no longer benefiting from scale-related productivity increases. The reason: strain on public transportation infrastructure. In terms of economic production, Birmingham, the United Kingdom’s second-biggest city, acts as a city with a third of its population during peak travel hours. It is far from an outlier.

Not only are expanding populations putting a burden on mobility, but e-commerce has roughly tripled in the previous five years (even before adjusting for the pandemic), resulting in a spike of congestion induced by last-mile distribution.

As a result, there is increasing pressure for mayors and municipal authorities to take action to decrease traffic, reduce pollution, reduce carbon emissions, and assist in urban redevelopment to protect the health and economics of their cities.

As part of its aspirations to become carbon neutral, Copenhagen is investing in a variety of micro-mobility programs, aiming to employ this form of transportation for more than half of all journeys by 2025.

Mobility needs are evolving, are cities ready?

Regulatory and policy demands, such as the need to cut carbon emissions and improve air quality, adapt to changes in people’s wants and behaviors, increase in e-commerce delivery, a rise in remote working, and other factors, are increasing the complexity.

During the pandemic, lockdowns served as a behavioral deterrent. “circuit breakers,” adds Huettel, forcing many individuals to rethink their mobility demands and options.

With flexible, “hybrid” employment on the rise (76% of European organizations currently have remote work rules), many office workers will likely commute less frequently or during off-peak hours. And this is intensifying pre-crisis tendencies like the transition away from private automobile ownership and fixed-schedule public transportation and toward new mobility alternatives like micro-mobility, subscription- and shared-transport services, he argues.

Huettel adds that it will take time for the effects of these changes to be realized and for us to identify between disruption induced by the pandemic and longer-term adjustments in mobility. However, one pre-crisis trend remains the demand for improved methods for moving people and things throughout cities.

Routing mobility through the cloud

Dr. Matthias Kempf, a founding partner of automotive consultancy Berylls Strategy Advisors, emphasizes the significance of acting quickly to fulfill increasing demands.

“We predict that cities and present mobility networks throughout the world will be unable to satisfy expanding demand, resulting in a travel capacity deficit of 150–200 billion person kilometers in the next 15 years,” he adds. “That’s the equivalent of 25,000 trips to the moon and back.”

He also claims that cities are designed largely for vehicles rather than people.

“Mobility must be designed with people in mind,” he argues. “Because people dislike changing ‘vessels,’ we want a quick, pleasant, and easy integrated transit system that encourages people to utilize the most efficient means of transportation.” What is very crucial in this case is the availability of high-quality mobility data that allows mobility systems to be planned and operated as needed.”

However, innovation is changing this. While cars will continue to be a major mode of transportation for the foreseeable future, they are being reimagined “as-a-service” and consumed as a bundle of kilometers or minutes by companies like Volkswagen’s daughter company Urban Mobility International (UMI), which has launched an electric vehicle car-sharing service called WeShare. The idea is to minimize the number of automobiles on city roadways while making better use of those that do remain. Customers use their smartphones to hire a car, which they then return to any location within the operational area. The company uses AI technology to analyze demand and parking behavior trends, allowing it to enhance its service and save operational expenses.

Philip Reth

“Our concept of how we use and experience transport is transforming,” says Philipp Reth, CEO of UMI, who says the current generation dubbed “Generation Share” is choosing to rent on demand and pay as you go.

“It’s cost-effective, convenient, and environmentally savvy,” Reth says. “It means car ownership is becoming less and less relevant for urban mobility – it’s increasingly more about a multi-modal mix of options that are interlinked in a meaningful and intelligent way.”

As cities invest in fast wireless connectivity and the cloud to create new digital highways, “digital vehicles” – vehicles that are as much made of software as they are of metal and rubber and able to intelligently interact with and navigate the outside world – will be key to realizing the potential of smart mobility, says Christoph Hartung, CEO of Bosch’s software subsidiary, ETAS.

He claims that “cars are soon becoming linked components in a connected mobility system.”

Bosch has announced a collaboration with Microsoft to create a software platform that would easily link automobiles to the cloud. The purpose of this partnership is to simplify and speed the development and deployment of vehicle software in compliance with automotive quality standards throughout a car’s lifespan. The new platform, which will be built on Microsoft Azure and include Bosch software components, will allow the software to be produced and distributed to control units and vehicle computers. The partnership will also focus on the creation of technologies to improve the efficiency of the software development process.

Christoph Hartung

“As cars get smarter, they open up a limitless number of options for innovation,” adds Hartung. “Having access to open, standardized platforms lower the ‘barrier to entry and shortens the time to market for new mobility services and operators.” This will assist customers by providing more options and will allow us to become wiser about how things travel across cities in linked vans and trucks.”

According to Hartung, the ecosystems to which cars are connected must be open and standardized, or else solutions would be fragmented and adoption will be blocked.

“People are traveling not only inside London, Berlin, or Paris, but from Paris to Berlin, and they want a seamless experience,” he adds.

Shashi Verma, Chief Technology Officer at Transport for London (TfL), spends a lot of time thinking about how to encourage healthy customer behaviors and choices through experiences.

“We want to guide people to the most efficient means of transportation, which is often public transportation or active modes like walking and cycling,” he explains. “However, finding out how the public transportation system works in a new city may be difficult, so many people just go and join the cab waiting.”

“That is something we do not want to happen.” We’re focused on making things easy for customers in London, such as open payments and making the experience of paying for transportation similar to buying coffee or anything else. People do utilize it, and we’ve seen big gains from bringing the right sort of ease to the right kind of product.”

Shashi Verma

TfL’s goal, according to Verma, is not only to administer London’s transportation system but also to boost the city’s productivity. To keep the city running, a constant focus on balancing capacity and demand is required. TfL requires a variety of Microsoft tools and services to run its daily operations. For example, using Microsoft Teams to conduct remote inspections and engineering design assessments allows new infrastructure to be guaranteed and brought into service during lockdowns without the requirement for on-site presence. TfL also utilizes Azure, Microsoft’s secure cloud hosting environment, to host a variety of apps and data, including Power BI to help make data on recent passenger journeys and road collisions more accessible to the public.

“The most important use of technology is to ensure that capacity is being used properly and that we, as operators, are getting the most out of that available capacity,” Varma adds. “Digital signaling, for example, has increased the capacity of the Victoria line by over 40%.” It is difficult and expensive to establish a new train line, but if you can achieve the same by improving the technology within existing rail lines, it is a significant benefit to any city.”

Smarter mobility, according to Verma, comes down to “better use of data to lead people to make the correct decisions, and better use of technology to drive those choices and strengthen infrastructure.”

Many parties believe that public-private collaboration on open data and data sharing is important to the development of innovative mobility solutions.

We have witnessed how adversity can lead to creativity as the globe continues to face multiple obstacles as a result of the COVID-19 epidemic. We went from early vaccination studies and testing to providing real doses to vulnerable people all around the world in less than a year. More than 560 million doses have been administered — and counting. This makes it the world’s largest organized vaccination program.

Along with the scientific achievements that have gotten us to this point, vaccination deployment requires a massive logistical effort. One critical component in this chain is the quick manufacturing of enough glass vials to hold the vaccinations and syringes to administer them. This must be accomplished without sacrificing quality, which is a non-negotiable when dealing with something as important and sensitive as a vaccine. Stevanato Group, an Italian corporation, is looking for new technologies to help deliver on both.

Positive pressure from the pandemic

The company, which has offices in the United States, Mexico, and Japan, got involved in the pandemic response early on. They produce plastic components used in virus detection kits and build essential vaccine inspection equipment used by several global pharmaceutical companies to ensure the integrity of their vaccines in addition to delivering glass vials and syringes to more than 70% of the global treatment and vaccine programs in the most advanced development stages.

Given that the company’s goods are used at nearly every stage of the process, from testing to immunization, ensuring business continuity while keeping on-site staff safe has been a top goal. Collaboration solutions like Microsoft 365 and Microsoft Teams have played an important role in minimizing interruption and connecting colleagues and customers as much as possible.

“We all recognize the very real impact that our job has on the globe,” says Raffaele Pace, Stevanato Group’s Engineering Vice President of Operations. “For example, if we deliver vaccine inspection equipment to our clients late, it has an impact on vaccine supply chains.” That type of positive pressure is advantageous because it gives you the impression that you can make a difference.”

Necessity is the mother of innovation

The pandemic also prompted the corporation to switch to remote testing of the inspection equipment it sells to pharmaceutical companies all around the world.

The factory acceptance test (FAT) is the final pre-delivery test that ensures the customer’s equipment is operational and allows the firm to fix any remaining difficulties. Before the final delivery, it is usually done in person at Stevanato Group’s facilities in Italy or Denmark, but as the pandemic swept through Italy and the rest of Europe in early 2020, it became clear that this way of working was no longer viable, although the need for quality inspection equipment was more pressing than ever. To address this issue, Stevanato Group began providing clients the option of remotely attending factory acceptance testing utilizing mixed reality technology for all of its equipment portfolios, including glass converting lines and assembly and packaging solutions.

Members of the company’s engineering teams wear a Microsoft HoloLens 2 headset and walk each customer through their equipment inspection process, demonstrating the general machine components, demonstrating how to run it, and sharing important documentation in real-time – all while the customer may be on the other side of the world.

“Some clients were apprehensive at first, but after they saw the technology in action, they saw the benefits extended beyond the pandemic, particularly in terms of cost and time savings when you eliminate the need to travel,” Pace explains.

This breakthrough has had a huge impact – not only are all FATs for Stevanato Group’s vision inspection devices now done entirely digitally, but the technology has also shown its worth early in the inspection process. Because conventional, in-person FATs are one of the final processes before equipment delivery, misunderstandings or difficulties that must be handled on a very short timeline may develop. Customers may now “view” the equipment for themselves an unlimited number of times.

The team has also started to provide remote audits and mixed reality meetings to incorporate clients earlier in the manufacturing process. For example, by using HoloLens 2 for early design review meetings, they can clarify the process, address any concerns, and provide the customer with a real-time sense of how their equipment is evolving – all of which reduces the likelihood of delays and issues affecting the overall production of medicines.