AI is powering breakthroughs across all domain. In this episode of Utilizing AI Podcast brought to you by Tech Field Day, part of The Futurum Group, David Kanter, Founder and Executive Director of MLCommons, joins hosts, Stephen Foskett and Frederic Van Haren, to talk about MLCommons’ role in driving valuable AI solutions, and helping organizations overcome the challenges around AI safety. MLCommons’ set of benchmarks provides transparent ratings and reviews of a wide range of products guiding buyers towards better purchase decisions.
A Seismic Wave of AI PCs, Renewed Focus on AI Safety, and More with David Kanter of MLCommons
Artificial intelligence has penetrated industries and market segments across the world, resulting in a market volume that is nothing like anything we saw before. Now the penetration is getting deeper than just organization-level.
A sharp uptick in the purchase of AI PCs shows the technology percolating through the consumer market. “In the last couple of years, AI has gone from something that was more of a technocratic concern to something that is tangible for everyone,” comments David Kanter, Founder, Executive Director, and Board Member of MLCommons.
As AI becomes pervasive, there are growing concerns about the safety of the technology. In this episode of Utilizing AI podcast, brought to you by Tech Field Day, part of The Futurum Group, Kanter joins hosts, Stephen Foskett and Frederic Van Haren, to talk about the rapid expansion of the AI PC market, and how MLCommons is working on spreading awareness and education about AI responsibility with open initiatives and collaborations with other members of the ecosystem.
A Means to Compare
MLCommons is an industry consortium for testing and evaluating AI systems. A one-of-a-kind community, it addresses a big pool of devices ranging from microcontrollers to IoT and more. Supported by many in the industry, it has a broad reach across constituencies, governments, industry academia and consumers.
MLCommons’ collective engineering effort is geared at uniting the makers and the consumers, and making artificial intelligence wholesome. “We’re focused on making AI better for everyone,” says Kanter.
MLCommons’ benchmark suites, MLPerf, serve as a gold standard guide for organizations looking to measure systems performance, and find viable alternatives. But it’s more than just a reference point. The benchmarks provide datapoints, a valuable means to measure and compare systems, and make apple-to-apple comparisons.
Typically, two groups of people benefit from this, told Kanter – solution providers who submit their results to track their systems’ performance levels transparently, and identify areas of improvement, and consumers that use those submissions as guides to closely examine their options, and make informed purchase decisions.
But there’s more. “Helping guide what is the right thing to buy is one aspect of it. Then after the purchase, helping to configure and operate is the other half. We really see it as providing best-known methods to the industry,” explains Kanter.
As noted, MLCommons draws from a variety of sources. Commercial vendors in cloud, software and hardware spaces make up the bigger part of the crowd. Submissions also come from universities and websites that send in their systems for evaluation.
MLPerf Suites release on a quarterly schedule. The next round of benchmarks is lined up for the end of this quarter, and its focus is AI inferencing. “We are always updating the benchmark suite, trying to stay abreast of all the developments, and one of the things that we heard from customers is that they love using MLPerf to make buying decisions, but they need large language models,” tells Kanter.
To cater to that, end of last year, MLCommons added GPT3 for training, and in 2024, they’re following it up with the Llama 270 billion model for inferencing. The focus is on two of the hottest areas – LLMs and image generation.
Kanter told Gestalt IT about two new, exciting developments at MLCommons. The first is the MLPerf Mobile Benchmark Suite with which the consortium makes inroads into the consumer mobile sector. The mobile benchmark suite aims to provide performance-accuracy benchmarks for AI inferencing in smartphones and laptops. The other project, MLPerf Client, provides ML benchmarks for consumer devices like desktops and laptops.
Inside the AI PC Boom
When ChatGPT went viral, it sparked an explosion of interest in AI-powered devices. Now all eyes are on AI PCs. As sales is peaking making big impacts on the overall PC market, some of the biggest names in the industry are betting on these systems.
These devices can run LLMs and AI workloads on-device making it possible for smaller LLMs south of 15 billion parameters to be trained natively on these systems. There is a host of use cases that could use their infinite power – input technologies like voice recognition, predictive text and video-to-text conversions, for example.
According to Kanter, the hottest use cases that could spread AI PCs to a broader market are conversing chatbots, real-time language translation, and summarization and synthesis. These applications, he says, will likely drive the real value of AI PCs in the coming days.
Safety Top of Mind
As a performance rating and review consortium, MLCommons is committed to helping organizations better understand artificial intelligence, and make the technology safe and accessible to all. “There is a much broader constituency for us. We are great at building and pulling our community together, but now the community is just so much larger,” says Kanter.
Last year, they formed a dedicated working group for AI safety. The idea behind it is to get people to work together, and put caution before performance when building solutions.
Lack of accuracy in AI models has raised big concerns among users as it spawns profound societal problems. It’s not just a debate of the ethicists. Risk-averse organizations are now being leery of the dangers of hasty development. Many are slowing down production to examine crossovers with ethical guidelines.
“Our feeling is that by building an open testing platform that can both support testing and benchmarks, we can really help improve the safety of AI overtime, and create trust, and guide responsible development in the same way that crash safety ratings help car manufacturers understand the scenarios they want to design against, like a rollover or crash,” says Kanter.
MLPerf benchmarks serve as a fabulous resource to study and understand those guardrails, and recognize where efforts can be put to move forward nimbly and responsibly.
Many noted industry players participate in MLCommons’ open consortium today. “We want to build the tooling and the infrastructure that allow you to ask the questions and get the answers in a robust, transparent, principled and open way,” Kanter says.
Be sure to watch the full episode, and for more interesting conversations on AI, keep watching the Utilizing AI Podcast.
Podcast Information
- Stephen Foskett is the Publisher of Gestalt IT and Organizer of Tech Field Day, now part of The Futurum Group. Find Stephen’s writing at Gestalt IT and connect with him on LinkedIn or on X/Twitter.
- Frederic Van Haren is the CTO and Founder at HighFens Inc., Consultancy & Services. Connect with Frederic on LinkedIn or on X/Twitter and check out the HighFens website.
- David Kanter is the Founder and Executive Director at MLCommons. You can connect with David on LinkedIn. Learn more about MLCommons on their website and follow them on X/Twitter and LinkedIn.
Thank you for listening to Utilizing AI, part of the Utilizing Tech podcast series. If you enjoyed this discussion, please subscribe in your favorite podcast application and consider leaving us a rating and a nice review on Apple Podcasts or Spotify. This podcast was brought to you by Tech Field Day, now part of The Futurum Group. For show notes and more episodes, head to our dedicated Utilizing Tech Website or find us on X/Twitter and Mastodon at Utilizing Tech.
