• Courses
    • Oracle
    • Red Hat
    • IBM
    • ITIL
    • PRINCE2
    • Six Sigma
    • Microsoft
    • TOGAF
    • Agile
    • Linux
    • All Brands
  • Services
    • Vendor Managed Learning
    • Onsite Training
    • Training Subscription
  • Managed Learning
  • About Us
    • Contact Us
    • Our Team
    • FAQ
  • Enquire

OUR BLOG


Using the Power10 Chip to Speed Up AI Inferencing

Posted by Marbenz Antonio on March 20, 2023

IBM Archives - The Next Platform

An inferencing model refers to a type of model that has been trained to identify patterns of interest in data, with the goal of gaining insights from the data.

Compared to training an artificial intelligence (AI) model, inferencing doesn’t require as much computing power. As a result, it’s feasible and even more energy-efficient to perform inferencing without additional hardware accelerators, like GPUs, and to do so on edge devices. It’s not uncommon for AI inferencing models to run on smartphones and similar devices using just the CPU. In fact, many picture and face filters found in social media phone apps rely on AI inferencing models.

IBM’s Power10 chip

IBM was a trailblazer in incorporating on-processor accelerators for inferencing into its IBM Power10 chip, which it dubbed the Matrix Math Accelerator (MMA) engines. By doing so, the Power10 platform is able to outpace other hardware architectures in terms of speed without requiring the use of additional GPUs, which would consume more energy. This means the Power10 chip can derive insights from data more quickly than any other chip architecture while consuming significantly less energy than GPU-based systems. That’s why it’s an optimal choice for AI applications.

When using IBM Power10 for AI, particularly for inferencing, AI DevOps teams don’t need to exert any additional effort. This is because data science libraries, including openBLAS, libATen, Eigen, and MLAS, among others, have already been optimized to utilize the Matrix Math Accelerator (MMA) engines. Consequently, AI frameworks that leverage these libraries, such as PyTorch, TensorFlow, and ONNX, are already able to take advantage of the on-chip acceleration. These optimized libraries can be accessed through the RocketCE channel on anaconda.org.

IBM Power10 can accelerate inferencing by utilizing reduced-precision data. Rather than using 32-bit floating point data, for instance, the inference model can be fed with 16-bit floating point data, which enables the processor to process twice as much data for inferencing simultaneously. This approach can be effective for some models without compromising the accuracy of the inferred data.

Inferencing is the final phase of the AI DevOps cycle, and the IBM Power10 platform was purposefully designed to be AI-optimized. As a result, clients can extract insights from data in a more cost-effective manner, both in terms of energy efficiency and by reducing the requirement for additional accelerators.

Conclusions

Leveraging the IBM Power10 chip can significantly accelerate AI inferencing while reducing energy consumption and the need for additional accelerators. The Matrix Math Accelerator (MMA) engines built into the chip can enhance the speed and efficiency of inferencing processes without requiring any additional effort from AI DevOps teams. Furthermore, the ability to process reduced-precision data can further enhance the performance of the inferencing model without sacrificing accuracy. All of these factors make the IBM Power10 chip an ideal choice for clients seeking to extract insights from data in a cost-effective manner.

 


Here at CourseMonster, we know how hard it may be to find the right time and funds for training. We provide effective training programs that enable you to select the training option that best meets the demands of your company.

For more information, please get in touch with one of our course advisers today or contact us at training@coursemonster.com

Archives

  • September 2023
  • August 2023
  • July 2023
  • June 2023
  • May 2023
  • April 2023
  • March 2023
  • February 2023
  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • March 2021
  • February 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • August 2020
  • July 2020
  • June 2020
  • May 2020
  • March 2020
  • December 1969

Categories

  • Agile
  • APMG
  • Business
  • Change Management
  • Cisco
  • Citrix
  • Cloud Software
  • Collaborizza
  • Cybersecurity
  • Development
  • DevOps
  • Generic
  • IBM
  • ITIL 4
  • JavaScript
  • Lean Six Sigma
    • Lean
  • Linux
  • Marketing
  • Microsoft
  • Online Training
  • Oracle
  • Partnerships
  • Phyton
  • PRINCE2
  • Professional IT Development
  • Project Management
  • Red Hat
  • SAFe
  • Salesforce
  • SAP
  • Scrum
  • Selenium
  • SIP
  • Six Sigma
  • Tableau
  • Technology
  • TOGAF
  • Training Programmes
  • Uncategorized
  • VMware
  • Zero Trust

Meta

  • Log in
  • Entries feed
  • Comments feed
  • WordPress.org

home courses services managed learning about us enquire corporate responsibility privacy disclaimer

Our Clients

Our clients have included prestigious national organisations such as Oxford University Press, multi-national private corporations such as JP Morgan and HSBC, as well as public sector institutions such as the Department of Defence and the Department of Health.

Client Logo
Client Logo
Client Logo
Client Logo
Client Logo
Client Logo
Client Logo
Client Logo
  • Level 14, 380 St Kilda Road, St Kilda, Melbourne, Victoria Australia 3004
  • Level 4, 45 Queen Street, Auckland, 1010, New Zealand
  • International House. 142 Cromwell Road, London SW7 4EF. United Kingdom
  • Rooms 1318-20 Hollywood Plaza. 610 Nathan Road. Mongkok Kowloon, Hong Kong
  • © 2020 CourseMonster®
Log In Register Reset your possword
Lost Password?
Already have an account? Log In
Please enter your username or email address. You will receive a link to create a new password via email.
If you do not receive this email, please check your spam folder or contact us for assistance.