Inference is the process of getting answers from AI models. It represents the next great battleground in AI infrastructure. While training gets the headlines, inference is where the real-world value gets created. Every ChatGPT response, every AI-generated image, every automated decision requires inference compute. 

Launched October 2024, InferenceMAX by SemiAnalysis is an open-source, continuously updated benchmarking framework that measures how efficiently different hardware and software setups run large language models. It tracks throughput, latency, and cost per token across GPUs, inference frameworks, and precisions to show real-world performance trade-offs in LLM inference. 

Where Inference meets Inflection and Beyond 

When NVIDIA crushed the SemiAnalysis InferenceMAX benchmarks this week, beating AMD across every metric, the main story was on the hardware superiority. But the real story was over half of NVIDIA’s engineers work on software, not chips. NVIDIA isn’t a semiconductor company that happens to write code—it’s an integrated systems company that happens to make semiconductors. 

NVIDIA’s dominance here isn’t just about raw performance. As their senior director noted, “I don’t think most people fully appreciate how complex these inferencing workloads are when you’re deploying them at data center scale.” This complexity has the potential to create a moat that extends far beyond chip specifications. 

Reading the Signals: A Framework for Understanding Inflection 

Our CIO Mark Scalzo reminds us,“the question isn’t whether NVIDIA will maintain its lead forever—no company does. The question is how to remain properly equipped to identify the next set of inflection points hiding behind misunderstood competitive dynamics.”  

Consider how NVIDIA’s transformation unfolded through the lens of our inflection analysis: 

The Identification: While markets focused on cyclical GPU demand patterns, a deeper story was emerging. NVIDIA’s installed base of hardware plus its CUDA software ecosystem had quietly reached critical mass. This wasn’t about faster chips—it was about an entire platform achieving network effects that would prove nearly impossible to replicate. 

The Validation: When companies deliver revenue of 200%+ year-over-year growth, it’s not luck—it’s validation of a fundamental business transformation. But here’s what’s interesting: even after such dramatic outperformance, markets often remain anchored to old narratives. They see outliers where systematic analysis reveals sustainable competitive advantages. 

Further Inflection: As fundamentals evolve, so must positioning. When NVIDIA announced Blackwell and now discusses Vera Rubin for next year, they were not just iterating on hardware—they were extending a platform advantage that compounds with each generation. AMD’s acknowledgment that their competitive rack-scale systems won’t arrive until late 2026 tells you everything about the durability of this lead. 

Inference as Insight 

Just as AI models use inference to extract meaning from data, successful investing requires inferring future trajectories from current signals. The ability to see what others miss comes from: 

Looking beyond obvious metrics: Not just chip speeds but entire ecosystems. When a company has more software engineers than hardware engineers despite being known for hardware, that’s a signal worth investigating. 

Understanding time horizons: AMD’s frank admission about 2026 availability isn’t just a product delay—it’s a window into how long transformative advantages can persist. Markets often underestimate the time required to replicate complex, integrated systems. 

Recognizing compounding advantages: Network effects, switching costs, and ecosystem lock-in don’t show up in quarterly earnings models, but they determine long-term winners. NVIDIA’s software improvements deliver more performance on existing hardware—that’s customer value creation that extends far beyond selling new chips. 

The DeepSeek Moment When Narratives Collide with Reality 

The DeepSeek panic briefly questioned NVIDIA’s moat when Chinese researchers claimed breakthrough efficiency. Yet here we are months later, with NVIDIA extending its lead in real-world inference benchmarks. This pattern—narrative-driven volatility followed by fundamental reassertion—creates opportunities for those who can distinguish signal from noise. 

The lesson isn’t that NVIDIA is invincible. It’s that understanding the true sources of competitive advantage—software ecosystems, not just hardware specs—provides clarity when headlines create confusion. 

The Broader Pattern 

NVIDIA’s story exemplifies a broader pattern in technology investing: the most durable moats often exist in the least visible layers of the stack. While everyone debates nanometer processes and teraflops, the real competitive dynamics play out in software frameworks, developer ecosystems, and system integration. 

This same pattern appears across industries: 

  • Payment processors whose real value lies in fraud detection algorithms, not transaction routing 
  • Industrial companies whose software simulation capabilities matter more than their manufacturing prowess 
  • Healthcare firms where data integration capabilities eclipse device specifications 

An Important Leap Forward 

As AI shifts from training to inference, from labs to production, from experiments to revenue generation, new inflection points emerge. They don’t announce themselves with press releases. They reveal themselves through careful analysis of competitive dynamics, time horizons, and compounding advantages. 

The power of inference—both in AI and in investing—lies not in processing more data, but in extracting the right insights from the data available. Sometimes the most important signals aren’t the loudest ones. And sometimes, like NVIDIA’s software advantage, they’re hiding in plain sight. 

A new generation of benchmarking is an important leap forward into a brave new world. 

IMPORTANT DISCLOSURES

The Destra Multi-alternative fund that is sub advised by Validex invests in NVIDIA

Securities highlighted or discussed in this blog have been selected to illustrate Validex’s investment approach and/or market outlook and are not intended to represent any strategy or portfolio performance or be an indicator for how strategy or portfolio have performed or may perform in the future. Each security discussed in this blog has been selected solely for this purpose and has not been selected on the basis of performance or any performance-related criteria. The securities discussed herein do not represent an entire portfolio and, in aggregate, may only represent a small percentage of a strategy or portfolio holdings. The strategies and portfolios are actively managed, and securities discussed in this blog may or may not be held in such strategies or portfolios at any given time. These individual securities do not represent all the securities purchased, sold, or recommended and the reader should not assume that investments in the securities identified and discussed were or will be profitable. Nothing in this blog shall constitute a recommendation or endorsement to buy or sell any security or other financial instrument referenced in this letter. 

Validus Growth Investors, LLC , dba Validex Global Investing (Validex or VGI) seeks to invest in companies at every stage of their growth. From startups to publicly traded companies, our research identifies inflection points that have the potential to produce meaningful growth and income for the clients we serve. 

Investment Advisory Services are offered through Validex, an SEC Registered Investment Adviser. No offer is made to buy or sell any security or investment product. This is not a solicitation to invest in any security or any investment product of Validex. Validex does not provide tax or legal advice. Consult with your tax advisor or attorney regarding specific situations. Intended for educational purposes only and not intended as individualized advice or a guarantee that you will achieve a desired result. Opinions expressed are subject to change without notice. Investing involves risk, including the potential loss of principal. No investment can guarantee a profit or protect against loss in periods of declining value. All information is believed to be from reliable sources; however, we make no representation as to its completeness or accuracy. Opinions and projections are as of the date of their first inclusion herein and are subject to change without notice to the reader. As with any analysis of economic and market data, it is important to remember that past performance is no guarantee of future results. 

See social media content disclosure HERE