SAP Built the Enterprise AI Governance Layer Nobody Else Has. Here's the Blueprint.
The largest US bank by assets is running AI inference on hardware it controls. The SambaNova story reveals the enterprise AI bifurcation that cloud providers won't talk about.
JPMorgan Chase selected SambaNova Systems as its AI inference infrastructure partner in July 2026, deploying the company's SN40L and SN50 Reconfigurable Dataflow Units on-premise across its technology organization. The announcement landed the same week SambaNova closed the first tranche of a $1 billion Series F at an $11 billion valuation, led by General Atlantic with Intel Capital participating. Neither development made front-page tech news. Both matter more than the stories that did.
This is not a piece about SambaNova's funding round or JPMorgan's procurement cycle. It is about what the combination of those two events reveals about where enterprise AI infrastructure is actually heading — and why the cloud hyperscalers have a problem that their AI pitch decks are designed to obscure.
The Deal That Flew Under the Radar
JPMorgan Chase is the largest bank in the United States by assets, with $3.9 trillion on its balance sheet. It processes more financial transactions per day than many countries handle per year. When it makes a significant infrastructure decision — to move to cloud, to build a proprietary trading platform, to bet on a new chip architecture — the decision is not made lightly, and the reasoning is usually a leading indicator for how other large enterprises in regulated industries will make similar decisions over the next 24 months.
The SambaNova partnership is JPMorgan deploying AI inference hardware it controls, in facilities it controls, without sending inference queries through a third-party cloud API. JPMorgan is not abandoning cloud AI tools entirely. The bank uses OpenAI models through Azure for some workloads and runs internal experiments with Anthropic APIs. But for the production inference workloads that process sensitive data at scale, the bank made a deliberate choice to own the hardware.
SambaNova completed its Series F first close on July 8, 2026, with JPMorgan's deployment announced simultaneously. The sequencing matters: this is not a speculative hardware company raising money to find customers. This is a company raising money to fulfill a backlog that includes one of the most demanding enterprise technology buyers in the world.
What SambaNova's Reconfigurable Dataflow Unit Actually Is
Most enterprise buyers and most tech journalists treat GPU as a synonym for AI chip. It is not. Nvidia's H100, H200, and Blackwell B200 are general-purpose parallel processors that happen to be very good at matrix multiplication, which is the core mathematical operation in neural network inference and training. They are general-purpose by design because Nvidia's business model depends on a single chip architecture serving the entire AI market.
SambaNova built something different. The Reconfigurable Dataflow Unit is an application-specific processor designed from the ground up to run large language model inference efficiently. The key architectural difference is memory organization. Large language models require loading billions of parameters into memory for every inference call. GPUs handle this with a combination of on-chip SRAM and attached HBM memory, with data movement across the PCIe or NVLink bus representing a significant fraction of total inference latency and power consumption.
SambaNova's RDU architecture configures memory access patterns at the chip level to match the specific data access patterns of transformer model inference, reducing memory bottlenecks that are unavoidable on general-purpose GPU architectures. The practical result, according to SambaNova's published benchmarks and Tom's Hardware's independent analysis, is that the SN50 processes Llama 3.3 70B at roughly 895 tokens per second per user at FP8 precision — compared to 184 tokens per second per user on Nvidia's Blackwell B200. This is approximately a 5x throughput advantage on this specific workload.
The SN50 is sold as a rack-scale system rather than discrete cards. Enterprises receive a turnkey inference appliance with SambaNova's software stack pre-configured, reducing the deployment and operations burden that GPU clusters create. Intel provides Xeon CPUs for the surrounding compute in the system, which is why Intel Capital participated in the Series F — Intel has a financial and strategic interest in this deployment succeeding as a reference architecture for on-premise AI infrastructure.
The JPMorgan Decision: Why the World's Largest Bank Chose Hardware Over APIs
JPMorgan's technology leadership did not choose SambaNova because the benchmarks are better than Nvidia's. Hardware procurement at a $3.9 trillion balance sheet institution involves legal, compliance, risk, and vendor management layers that benchmark tables don't capture.
The decision rests on four factors that are structurally true for JPMorgan and structurally true for most large regulated enterprises:
Data sovereignty. JPMorgan processes customer account information, trading data, confidential advisory communications, and proprietary risk models. Every inference call sent to a cloud API transmits data to infrastructure the bank does not control, creating regulatory complexity under the Gramm-Leach-Bliley Act, FINRA regulations, OCC guidance on third-party risk, and JPMorgan's own internal data classification policies. On-premise inference keeps every query within the bank's own network boundary, eliminating the regulatory overhead of treating every AI interaction as a third-party data transmission.
Latency for agentic workloads. JPMorgan is building AI agents that handle multi-step research, document processing, and customer communication workflows. These agents require dozens to hundreds of model calls per task. At a cloud API round-trip latency of 80-300 milliseconds per call, a 50-step agent workflow takes 4-15 seconds of pure network overhead — before model inference time. On-premise inference running in the same data center as the application removes network latency from the critical path, reducing 50-step agent runtimes from minutes to seconds.
Total cost of ownership at volume. JPMorgan's AI inference volume is in the tens of billions of tokens per day and growing rapidly as agent workflows expand from pilot to production. At enterprise API pricing of $1-15 per million tokens depending on model and provider, this volume translates to $10-150 million per month in API costs at steady state. Own-and-operate hardware eliminates per-token pricing and replaces it with a fixed capital and operations cost that is dramatically lower at JPMorgan's scale.
Vendor concentration risk. An enterprise that depends on a single cloud provider's AI APIs for production-critical workflows has created a single point of failure and a single point of leverage in commercial negotiations. JPMorgan's technology strategy explicitly avoids this kind of concentration. On-premise inference hardware provides an alternative path that does not depend on any model provider's uptime, pricing decisions, or API terms of service changes.
The Performance Numbers Worth Knowing
The benchmark comparison that matters most for enterprise procurement is not peak throughput but throughput at scale, cost per million tokens at production volume, and latency at the 95th and 99th percentile:
| Metric | SambaNova SN50 | Nvidia B200 | Nvidia H200 |
|---|---|---|---|
| Llama 3.3 70B tokens/sec/user (FP8) | ~895 | ~184 | ~120 |
| Sustained throughput advantage vs H200 | 7x | 1.5x | 1x |
| Memory bandwidth per chip | 2.4 TB/s | 8 TB/s | 3.35 TB/s |
| On-premise deployment | Yes (rack appliance) | Yes (GPU cluster) | Yes (GPU cluster) |
| Per-token cloud API pricing | Not applicable | ~$1-3/M (via hyperscalers) | ~$1-3/M (via hyperscalers) |
| Supported model sizes | Up to Llama 3.1 405B | Up to frontier models | Up to frontier models |
The key caveat: these numbers are SambaNova's published benchmark results, and real-world production performance depends heavily on workload mix, model size, context length, and batch configuration. Independent benchmarks from Artificial Analysis show narrower gaps on some workloads. Enterprises considering SambaNova should run their specific production workload on evaluation hardware before committing to a deployment.
The Enterprise AI Infrastructure Bifurcation
The JPMorgan decision is not an anomaly. It is an early data point in a pattern that will reshape enterprise AI infrastructure over the next 24 months. Enterprise AI is bifurcating into two distinct architectures, not converging on a single cloud-API model.
| Category | Cloud API Model | On-Premise Inference |
|---|---|---|
| Primary buyers | Startups, SMBs, cost-variable enterprises | Large regulated enterprises, government, defense |
| Data governance | Third-party custody | Enterprise custody |
| Latency profile | 50-300ms network overhead | Sub-10ms local inference |
| Pricing model | Per-token variable | Fixed CapEx + OpEx |
| Break-even volume | Below ~100M tokens/day | Above ~100M tokens/day |
| Model selection | Provider's model catalog | Open weights + licensed models |
| Vendor dependency | High (API terms, pricing, uptime) | Low (hardware owned) |
This bifurcation is not a replay of the cloud-versus-on-premise debate from 2010. The cloud-versus-on-premise debate in the SaaS era was about total cost of ownership and operational flexibility. The AI inference version of this debate is about data control, latency, and unit economics at scale. These are different variables with different implications for which enterprises choose which path.
The enterprises that will run on-premise AI inference by 2028 are not the ones that failed to embrace cloud — they are the ones whose data governance requirements, inference volume, and latency sensitivity make cloud APIs structurally unsuitable regardless of pricing.
The $1B Series F and What $11B Valuation Means in 2026
CNBC's coverage of the SambaNova Series F focused on the headline numbers: $1 billion, $11 billion valuation, General Atlantic leading. The more interesting signal is the investor composition and timing.
General Atlantic is a growth equity firm. It does not typically lead speculative early-stage rounds — it leads rounds where revenue is real, growth is documented, and the primary use of capital is scaling production capacity and enterprise sales. The $1 billion raise is financing hardware production and deployment capacity, not R&D. This implies SambaNova has customers with committed purchase orders substantial enough to justify $1 billion in manufacturing-oriented capital.
Intel Capital's participation is strategic: Intel makes the Xeon CPUs that power the surrounding compute in SambaNova's rack appliance. Intel has a direct financial interest in the on-premise AI inference market growing, because every SambaNova deployment is also an Intel CPU deployment. Intel Capital is not investing in SambaNova's AI chip; it is investing in a market where Intel's chips participate in every sale.
The $11 billion valuation implies a significant ARR multiple. SambaNova does not disclose revenue, but at a typical infrastructure company growth-equity multiple of 10-20x ARR, the valuation implies revenues in the $500M-$1.1B range. For a company still shipping SN50 hardware in H2 2026, this is either aggressive or reflects very substantial forward bookings from enterprise customers like JPMorgan and SoftBank.
The Hyperscaler Problem: What AWS, Azure, and GCP Should Fear
Amazon Web Services, Microsoft Azure, and Google Cloud have built the dominant AI infrastructure businesses by selling API access to frontier models at per-token pricing. The JPMorgan-SambaNova dynamic creates a specific type of competitive risk that the hyperscalers' current pricing models are not well-positioned to address.
The hyperscaler business model for AI inference requires two things: that enterprises prefer per-token variable pricing over fixed infrastructure CapEx, and that enterprises trust third-party cloud infrastructure with production AI workloads. JPMorgan's decision is evidence that for large regulated enterprises at scale, both of these assumptions fail.
What the hyperscalers can offer in response is limited. They can lower per-token pricing — which several have done repeatedly since 2025, as the ongoing inference price war documents. But price alone does not solve the data governance objection, the latency objection, or the vendor concentration objection. Those require architecture, not discounts.
The hyperscalers' second option is to sell hardware under enterprise private cloud models — rack-level deployments in enterprise data centers, equivalent to what SambaNova is doing. AWS has Outposts, Azure has Stack, and Google has Distributed Cloud. None of these are purpose-built inference appliances optimized for LLM workloads. They are datacenter infrastructure adapted for hybrid cloud use cases. Against a purpose-built inference appliance from SambaNova, the hyperscalers' hybrid cloud offerings are running a different race.
The hyperscaler risk is not immediate. Cloud AI APIs will remain the dominant architecture for the majority of enterprise AI workloads through 2027, because the majority of enterprise AI deployments are at volumes where per-token pricing is cost-effective and data governance requirements are manageable. But for the subset of enterprises — large banks, insurers, pharma companies, defense contractors, government agencies — where volume is high and data governance is non-negotiable, the on-premise inference market is now commercially validated by a customer as credible as JPMorgan. That changes enterprise buyer conversations in ways that will take 12-18 months to show up in the hyperscalers' cloud revenue growth rates.
Compare this to what is happening simultaneously in the inference hardware space: Etched's transformer-specific ASIC at $10.3B valuation, Fireworks AI at $1B ARR on an inference optimization platform, and AMD's $5B bet on Anthropic compute — all converging on the same insight that the inference layer is a durable business opportunity that is not entirely captured by Nvidia or the cloud hyperscalers.
Five Infrastructure Questions Every Enterprise Team Should Answer Now
Before the next infrastructure procurement cycle, enterprise technology and AI teams should pressure-test their current architecture against five questions:
1. Map your data governance exposure. Which AI workloads in your production stack process data categories that create regulatory complexity if transmitted to third-party cloud APIs? Health records, financial data, legal communications, and HR records all have specific regulatory treatment that on-premise inference can simplify. If you have not explicitly categorized AI workloads by data sensitivity, start there.
2. Model your inference economics at 12-month projected volume. Take your current per-token API spend and project it forward at the growth rate your AI team is targeting. At what volume does the TCO of owned hardware cross below the TCO of cloud API pricing? Most enterprise teams discover this crossover is closer than they expected once agent workflows enter production. Do this calculation before you are already locked into cloud-only infrastructure.
3. Benchmark latency against your agentic task requirements. If your AI use cases are single-query assistants, cloud API latency is acceptable. If you are building multi-step agents that make 20-100 model calls per task, benchmark what cloud latency does to your end-to-end task completion time. The difference between 100ms per call and 5ms per call is imperceptible for a single interaction and the difference between a 10-second and sub-1-second experience for a 100-call agent workflow.
4. Assess your model portability. On-premise inference hardware like SambaNova's appliances run open-weight models (Llama, Mistral, and similar architectures) rather than proprietary models from OpenAI or Anthropic. Before committing to on-premise infrastructure, verify that your production use cases can be served adequately by open-weight models at the required quality level. For many enterprise tasks, they can. For tasks requiring frontier-tier capability that only closed models provide, cloud APIs remain the necessary path.
5. Build your vendor diversification map before you need it. The enterprise AI vendor landscape is consolidating rapidly. Companies that built single-vendor cloud API dependencies in 2024 and 2025 are discovering that pricing changes, service outages, and terms of service modifications create business continuity risk. A diversified infrastructure architecture — some workloads on cloud APIs, some on open-weight models via on-premise hardware — provides resilience that a pure-cloud model cannot.
Cerebras's IPO and the inference hardware market it validated created the opening for purpose-built inference hardware to be taken seriously in enterprise procurement. SambaNova and JPMorgan's partnership is the first production validation of that market at regulated-enterprise scale.
Takeaway: JPMorgan's choice to deploy SambaNova inference hardware on-premise is not a rejection of cloud AI — it is a sophisticated infrastructure decision made by a team that modeled the full TCO, latency profile, data governance requirements, and vendor risk of both options. The fact that the world's largest bank made this choice is a leading indicator for how large regulated enterprises will approach AI inference procurement over the next 24 months. SambaNova's $1B raise at $11B valuation is not a speculative bet on future technology — it is capital to fulfill a backlog created by customers who have already concluded that on-premise inference is the right architecture for their needs. The hyperscalers should be paying close attention.
Frequently Asked Questions
What is SambaNova Systems and what does the SN50 chip do?
SambaNova Systems is a Palo Alto-based AI hardware company that builds purpose-built AI accelerators called Reconfigurable Dataflow Units, or RDUs. Unlike Nvidia GPUs, which are general-purpose parallel processors adapted for AI workloads, RDUs are designed from the ground up for inference on large language models and agentic AI tasks. The SN50 is SambaNova's most recent RDU, announced in early 2026 with Intel as a deployment partner. SambaNova claims the SN50 processes Llama 3.3 70B at 895 tokens per second per user — versus 184 tokens per second per user on Nvidia's B200 GPU — representing roughly a 5x speed advantage for this specific workload profile. The SN50's primary architectural advantage is memory bandwidth: it uses high-bandwidth memory configured to maximize throughput for long-context inference, which is the dominant bottleneck for enterprise AI agents handling multi-step reasoning tasks. The chip is sold as a rack-scale system rather than as individual cards, which means enterprises buy a turnkey inference appliance rather than a collection of components to assemble. This full-stack delivery model is part of why JPMorgan chose SambaNova for its on-premise deployment: the enterprise can specify inference requirements and take delivery of a system that performs predictably, rather than managing driver updates, CUDA versions, and framework compatibility across a heterogeneous GPU cluster.
Why did JPMorgan choose on-premise AI inference over cloud providers?
JPMorgan's decision to deploy SambaNova inference hardware on-premise rather than using cloud AI APIs reflects four factors that are common to large regulated enterprises. First, data governance: JPMorgan processes customer financial data, proprietary trading information, and confidential client communications. Sending inference queries to a third-party cloud API means that data transits infrastructure the bank does not control, creating regulatory complexity under GLBA, FINRA, and internal audit requirements. On-premise inference keeps all data within the bank's own network boundary. Second, latency: cloud API round-trip times introduce 50-300 milliseconds of additional latency per inference call, which is acceptable for asynchronous tasks but becomes a compounding bottleneck in multi-step agentic workflows that require dozens of model calls to complete a task. On-premise inference eliminates the network round-trip. Third, total cost of ownership: at JPMorgan's inference volume, per-token cloud API pricing compounds into hundreds of millions of dollars annually. Own-and-operate inference hardware has higher upfront capital cost but dramatically lower marginal cost per token at scale. Fourth, vendor independence: relying exclusively on OpenAI, Anthropic, or Google for critical AI infrastructure creates concentration risk. Running inference on hardware the bank controls eliminates this dependency. All four of these factors become more acute as enterprise AI moves from pilot to production, which is why JPMorgan's choice is more likely to become common among large regulated enterprises than to remain exceptional.
How does the SambaNova SN50 compare to Nvidia's Blackwell B200?
According to SambaNova's internal benchmarks published in early 2026, the SN50 outperforms Nvidia's Blackwell B200 GPU on inference-specific workloads by significant margins: roughly 5x maximum throughput and more than 3x sustained throughput on Llama 3.3 70B at matched precision levels. On a per-user token-per-second basis with FP8 precision, SambaNova claims 895 tokens/second versus 184 tokens/second on the B200. Independent benchmarking by Artificial Analysis in July 2026, using a hybrid configuration of four Nvidia H200 GPUs and sixteen SambaNova SN50 RDUs, reached 763 tokens per second on MiniMax M2.7 at a 10,000-token context, several times faster than GPU-only configurations in the same benchmark. These advantages are specific to the SN50's architectural profile: high memory bandwidth optimized for large-model inference. The B200 retains significant advantages for training workloads, general-purpose compute, and workloads where Nvidia's CUDA ecosystem provides tooling benefits that the SambaNova platform cannot match. Enterprises evaluating SambaNova versus Nvidia should benchmark against their actual production workloads, not SambaNova's published numbers, because the performance gap varies substantially by model size, context length, and batch size.
What types of enterprise workloads benefit most from on-premise AI inference?
On-premise AI inference delivers the most value in four workload categories. First, regulated data processing: any enterprise that handles patient health information, customer financial data, or classified government information under regulatory frameworks that restrict data transmission benefits from keeping inference within a controlled network boundary. Banking, pharma, healthcare, defense, and government agencies are the primary sectors. Second, high-volume repetitive inference: enterprises running millions of AI queries per day for document processing, customer communication classification, or code review quickly find that per-token cloud API pricing exceeds the total cost of ownership for owned hardware. The break-even threshold varies by provider and negotiation, but enterprise buyers consistently report cloud inference becomes uneconomical somewhere between 100 million and 500 million tokens per day. Third, ultra-low latency applications: real-time trading systems, fraud detection, and interactive AI assistants with strict response-time SLAs cannot absorb cloud API network latency. On-premise inference running within the enterprise data center can achieve sub-10-millisecond inference latency on purpose-built hardware. Fourth, agentic AI pipelines: multi-step agent workflows that require dozens to hundreds of model calls to complete a task accumulate latency and cost with every API hop. On-premise inference dramatically improves the economics of agentic systems by removing per-call API pricing and network round-trips from the critical path.
What does SambaNova's $1B Series F valuation at $11B mean for the AI chip market?
SambaNova's $1B Series F closing at an $11B valuation in July 2026, led by General Atlantic with Intel Capital participating, is significant for three reasons. First, it validates the inference-specific hardware market as a durable investment category distinct from general-purpose GPU infrastructure. The funding follows Cerebras's IPO at a $8.1B valuation, Groq's reported $2.8B Series D, and Etched's $300M raise at $10.3B — all inference-focused hardware companies raising at premium valuations in the same 12-month window. Second, it signals that enterprise customers are committing to alternative inference infrastructure at a scale that justifies continued capital investment. SambaNova does not raise venture capital to fund speculative development — it raises to build hardware for customers with purchase commitments. General Atlantic and Intel Capital are not making a bet on technology; they are financing a backlog. Third, the valuation implies that SambaNova is generating revenue and bookings at a multiple consistent with high-growth infrastructure companies, not speculative pre-revenue startups. While SambaNova does not disclose revenue, an $11B valuation at this stage of the market implies ARR in the hundreds of millions, consistent with JPMorgan and SoftBank as anchor enterprise customers.