Afternoon briefing · 16:00
Memory and land: the bottlenecks of AI infrastructure
By · bombong
GPUs are no longer the only obstacle to scaling AI infrastructure. This afternoon's news showed both moves to broaden memory supply and difficulties around data center sites. Reports of growing local opposition on both sides of the Atlantic followed.
After GPUs, the next bottleneck is memory
If the morning piece looked at the conditions for bringing agents into real work, the afternoon news turns to the conditions of the infrastructure needed to run them. The most direct signal is memory. Citing two sources, The Information reported that TSMC has begun drawing up plans to produce next-generation 3D DRAM at the request of its largest customer, NVIDIA36. According to the report, the backdrop is a memory shortage and sharply higher DRAM prices driven by rising demand for AI chips.
The report matters because it shows that what governs GPU supply is shifting from the compute chip to the memory attached to it. If a foundry (a contract chip manufacturer) enters a market where memory makers are concentrated in a handful of companies, the way GPU supply chains are put together could change as well.
Efforts to broaden supply sources are also visible in China. Moore Threads said Kimi K3.1 Pro can be served on its MTT S5000 GPUs and MUSA software from launch day40. Huawei Ascend and Alibaba Cloud SuperPoD had earlier announced day-one support. The push to avoid relying on a single provider of compute is taking different forms in different regions.
Walls beyond the chip: sites and public opinion
Even with chips and memory in hand, you still need land and power to put them on, and the consent of local communities. According to TechCrunch, Crusoe handed its troubled Wyoming data center project over to Google37. The harder it becomes to develop large data centers, the more such assets may end up concentrated among operators with deep pockets.
Public opinion is also becoming a factor. In an interview with CNBC, OpenAI CFO Sarah Friar said the company will stick to its infrastructure expansion plans even after Democrats, who have opposed data center construction, won big in the midterm elections38. Friar said OpenAI will keep building relationships with local communities to secure compute. OpenAI's position is that AI demand makes the investment unavoidable.
In Europe, more than 100 civil society groups across 16 countries were reported to have launched joint action calling for a halt to data center construction39. On the 6th, a protest took place at a Microsoft data center construction site in Amsterdam. With opposition spreading beyond a single country to many, permitting timelines have more room to affect overseas expansion plans.
What remains unconfirmed, and the demand-side response
Still, there are gaps that make it too early to draw firm conclusions about this trend. TSMC's move into memory is only a report citing sources. Whether it will actually enter the business, and on what timeline and scale, is unknown, and neither company has officially confirmed it. The Crusoe deal also lacks a sale price and the cause of the setbacks, making it hard to judge the state of the whole industry from this one case.
There are also signs that expanding supply is not the only answer. According to a community summary, DoorDash said it replaced the web search built into LLMs with a separate search API gateway (a relay layer shared by multiple agents), keeping accuracy unchanged while cutting search fees by more than 98%45. This figure does not include model inference costs. Even so, it shows that as infrastructure constraints tighten, the value of restructuring systems to use the same resources more sparingly grows as well.
For practitioners, this means AI adoption plans don't end with securing models or GPUs. Memory prices, data center permitting, and the efficiency of call architectures all affect costs and timelines. Going forward, watch whether TSMC and NVIDIA officially confirm the 3D DRAM plans, and whether local opposition in the US and Europe actually delays expansion schedules. Another point to watch is the terms on which OpenAI's funding round, which assumes large-scale infrastructure investment, closes.
This piece was written with the help of AI and includes personal views.
Morning briefing · 08:00
Agents put to the test on permissions and measurement
By · bombong
Interest in AI agents is shifting from 'how smart are they' to 'can they be trusted in real operations.' Today, several companies made announcements along similar lines. They dealt with the permissions that determine whom an agent acts for, how results are measured, and which models to entrust with decisions.
The permission design that production-scale agents demand
As agents take on real work, the first challenge to surface is permissions. AWS released an implementation guide that combines Okta Cross App Access and Bedrock AgentCore to manage which agents can act on behalf of which users13. In the enterprise-scale setup, AgentCore Gateway serves as a single entry point and Cedar policies bind tools to agents. Allow and deny decisions are tracked in CloudWatch. Underlying this is the recognition that per-app OAuth or service accounts alone struggle to handle 'acting on behalf of a user.'
Production cases show why such mechanisms are needed. Greenlight Financial built Luna, a hierarchical multi-agent system on AgentCore, to handle security and risk cases. The company said it resolves more than 90% of up to 4,000 cases a month without human intervention14. Taxbit presented an assistant that brings together source citations, role- and region-specific guardrails, and quality measurement on production traffic20. Both cases, from the regulated fields of finance and tax, put control and traceability ahead of performance. AWS explained that while building Amazon Quick, the same building blocks kept being reimplemented under different names. So it organized them into a taxonomy of triggers, agents, capabilities and learned context21. This signals that agents are moving from individual features to managed platforms.
Measurement and judgment become separate layers too
If permissions are about 'what to let agents do,' other companies have started measuring 'how well they did' separately. OpenAI released BioBasicsBench, a benchmark measuring how well agents handle wet-lab work (experiments that involve handling reagents and samples directly). It has 100 tasks1. It focuses mainly on interpreting raw experimental data rather than knowledge Q&A, an example of the unit of evaluation shifting toward real work. GitHub added a field to the Copilot usage API showing how often review comments were applied. It also added a field flagging users with missing telemetry23. The direction is to look at whether results were actually used, not just whether a tool was adopted.
A move to use dedicated models for the judgment step emerged as well. Ollama v0.40.15 began supporting AutoTrust, a decision model. Given a question, it returns choices, yes/no probabilities and scores15. Nace AI published a technical report on its Drex family of decision models. Drex outputs a probability distribution over choices in a single pass, without decoding24. Separating routing and evaluation steps from generative LLMs makes it easier to record the basis for decisions as numbers.
Tuning to the execution environment fits the same pattern. Hugging Face introduced Harness Adapters, which train LoRA adapters to fit the harness and task in use9. Behind it is the recognition that the same model can produce different results depending on the framework it runs in.
Unverified figures and remaining gaps
Still, it is too early to say this trend is established. Greenlight's automated resolution rate is an operational figure stated by the company. For BioBasicsBench, the excerpt does not confirm per-model scores or how the dataset will be released. Nace AI's rankings are also the company's claims. More measurement tools and external verification of those measurements are separate matters.
The standards defining the scope of agent behavior are also still being adjusted. According to reports, Anthropic revised Claude Opus 5.5's guidelines after complaints that it blocked game development requests too aggressively. Directly cloning an existing game remains blocked, while creating a new game in the same genre is allowed19. Refusal lines are being redrawn through real-world use. There was also a report that security researchers found hundreds of servers hosting the penetration testing tool Artex publicly exposed on the internet16. Boson AI introduced Higgs Realtime Duplex, a full-duplex voice model that calls tools in the background even while speaking3. However, the API is at the waitlist stage and its performance has not been externally verified. As more agents converse and work at the same time, designing permissions and records could become more complex.
For companies considering agent adoption, the points to weigh are shifting from model leaderboards toward the design of delegated permissions, outcome measurement and decision records. Three things are worth watching: whether BioBasicsBench's per-model results and data are released, the actual latency of the Higgs Realtime Duplex API once it opens, and whether the automation rates cited in production case studies undergo external verification.
This piece was written with the help of AI and includes personal views.
Unconfirmed reports 3
Not yet confirmed by an official source. Check the original report.
- JD Logistics' 'Superbrain 3.0' (超脑 3.0): AI directs everything from warehouse to delivery while robot fleets collaborate (report) Original
- Claude Max '20x' labeling follow-up: "Judge finds it could mislead paying subscribers" (community report, unconfirmed) Original
- "Stanford researchers build AI memory system that recalls full conversation history with 16 memory tokens, cutting document token use up to 4,212x" (community report, details unconfirmed) Original
Community buzz 5
Topics widely discussed on Reddit, Hacker News, GeekNews and elsewhere. These are community reactions and may include unverified claims.
- AI accelerating AI research A post arguing that 'AI is greatly accelerating AI research' drew attention on both Reddit and Hacker News. On GeekNews, by contrast, someone shared an experience where AI missed important literature and repeated meaningless experiments, lacking research judgment, and a Reddit experiment reported that while a model can read and edit weights, 20 fine-tuning steps erase the edits.
- Golden age of local LLMs A post argued that local models have come close to the frontier thanks to Qwen 27B and Flash-Next plus Unsloth Dynamic v3.1 quantization. Posts about QFN 3.8 drawing a dog with the mouse in Paint via Open WebUI 0.13.0's virtual desktop control, and about a llama.cpp PR improving the MoE expert cache, also drew responses.
- Next-model release rumors A post citing The Information saying 'ChatGPT Astra 6.2 could launch today' spread, but other posts complained that GPT-6.2, Fable 5.5 and Qwen 4 all failed to arrive this week. Speculative posts about a 100-million-token context also circulated.
- Spreading AI skepticism Nilay Patel's remark quoted by Simon Willison (that the products aren't good enough to justify the cost) was widely discussed. An article about university libraries being hammered by AI scraping bots, threatening public access, and a GeekNews post saying ads and spam have all turned into AI scams were also shared.
- Making physical things with Claude Posts drew attention about Claude painting a watercolor tree with a robot arm over 4.5 hours while mixing missing colors itself, and about Claude writing its own MCP connector for SolidWorks and modeling a part in minutes.