Google's HEIR Compiler Targets the Missing Layer in Private AI: Encrypted Inference Without a Cryptography PhD
Google's open-source HEIR compiler converts pre-trained models to homomorphic encrypted inference — with four demo apps and hardware accelerator partnerships.
Homomorphic encryption moves from cryptographer hobby to production compiler
On August 14, 2026, Google showcased HEIR (Homomorphic Encryption Intermediate Representation) — an open-source compiler toolchain that converts pre-trained AI models to run on encrypted inputs without decrypting user data on the server.
Jeremy Kun, Staff Software Engineer on Google's security team, announced the release in a blog post framing HEIR as the latest addition to Google's Private Computing Toolkit, alongside differential privacy, private information retrieval, and secure enclaves.
The privacy trade-off HEIR targets
Standard cloud AI inference requires plaintext data on the server — a non-starter for healthcare, finance, and regulated sectors. Homomorphic encryption allows computations directly on ciphertexts: servers process encrypted inputs and return encrypted results without seeing underlying information.
The catch has always been usability and cost. Manually converting models to homomorphic encryption historically required teams of cryptographers. HEIR aims to make the path closer to "one-click" encrypted inference for non-experts.
Google reports HEIR has accumulated four peer-reviewed publications, numerous citations, and collaborations with Georgia Tech, Carnegie Mellon, UC Santa Barbara, Purdue, the University of Edinburgh, Tsinghua University, and others.
Four production-shaped demos
Google shared four private inference applications compiled with HEIR, with latency on single-threaded CPU and source code on GitHub:
- Deep Learning Recommendation Model — private content recommendations (with Belfort Labs, LG, NYU)
- Credit card fraud detection — with Niobium and hardshell.ai
- Kitsune network anomaly detection — encrypted traffic analysis without revealing packet contents (with Niobium)
- Hotword detector — privacy-preserving audio trigger recognition (with Belfort Labs)
Google has partnered with homomorphic encryption hardware accelerators including Belfort, Niobium, Cornami, and Optalysys, with latency demonstrations planned.
Why this matters now
Three forces converge:
Regulatory pressure — GDPR, HIPAA, and sector-specific rules increasingly restrict raw data movement while still expecting AI-powered features.
Enterprise AI adoption — CIOs want model capabilities without expanding data breach surface area.
Cost curve — Google emphasizes homomorphic encryption costs are rapidly decreasing, shifting the privacy-capability trade-off toward a budget question rather than a feasibility question.
HEIR does not eliminate overhead. It lowers the expertise barrier and provides a shared compiler infrastructure for benchmarking optimizations — the same playbook LLVM used for general-purpose compilation.
Competitive landscape
Microsoft, IBM, and startups including Zama have pushed homomorphic and secure multiparty computation for years. Google's contribution is compiler-level tooling integrated with its broader Private Computing Toolkit and demonstrated on real model classes (recommendation, fraud, anomaly detection) rather than toy circuits alone.
For AI product teams, the near-term question is not whether homomorphic inference is free — it is whether HEIR makes encrypted inference plannable in a roadmap conversation with security and legal stakeholders.
### Sources
- Google — How Google is Making Private AI Practical with Homomorphic Encryption (August 14, 2026)
- Google GitHub — HEIR compiler repository (2026)