Afternoon briefing · 16:00
Inference costs decide where compute runs
By · bombong
This afternoon's news suggests that generative AI's next challenge is less about how smart models get and more about where inference runs and at what cost. New techniques aim to run models with tens of billions of parameters on phones and edge chips. On the cloud side, the compute demands of real-time generation services and the funding costs of GPU suppliers both came into view.
Efforts to pull compute down onto devices
If the morning piece looked for agent control mechanisms at the infrastructure layer, this afternoon's news asks where that infrastructure should sit. First, efforts to run models with tens of billions of parameters directly on devices emerged on both the software and chip sides. ModelBest published details and code for LES, the technique it used to run a 35B MoE (Mixture of Experts, an architecture that uses only some Experts for each input) model on a phone31. The method prefetches the Experts needed for the next layer from flash storage, and the company said it reached 20.51 tokens per second on an iPhone, 6.07 times faster than the baseline implementation. The approach uses MoE sparsity to hide storage latency on memory-constrained devices.
In Korea, DEEPX said it has completed the design of DX-M2, a generative AI chip based on Samsung Electronics' 2nm process32. The company's stated goal is to run models with 20 billion to 100 billion parameters at 10–30 tokens per second on less than 5W of power. The two announcements start from different places. Still, they point in the same direction: finishing inference on the device without calling out to the cloud.
The price of compute surfaces in the cloud
While devices pull compute down, the cost of renting compute is surfacing in the cloud. World Labs released Frameboy as a research preview, letting users move in real time through virtual spaces created with its world model Atlas33. Each time the user moves the camera, it generates the scene from the new viewpoint. The company said it generates up to 15 frames per second on cloud GPUs and sends them to the browser. When the approach shifts from displaying pre-made scenes to generating them moment by moment, GPU compute is consumed continuously for as long as the service is used.
The funding conditions of those supplying the GPUs are also worth watching. CNBC reported that bond yields of GPU cloud company CoreWeave have exceeded 16%34. Because the collected material lacks the bond's maturity or calculation basis, interpretation is limited. Still, bond yields reflect the risk premium investors demand. In that sense, the cost structure of cloud compute suppliers has arguably become another variable to weigh when designing services.
More compute is not the answer
That said, this trend cannot be seen purely as a matter of saving compute. Using more compute does not necessarily produce proportionally better results. Anthropic said Claude Fable 5.2 and Mythos 5.2 scored up to 75.3% on a scientific research agent evaluation and 64.9% on a robotics simulation evaluation29. Fable 5.2 lets users choose among five reasoning levels, from low to max. When Simon Willison tested pelican SVG generation, the best result came at xhigh. At max, the model reportedly used 128,000 thinking tokens and still failed to produce an image. As a one-off test by an individual, it is hard to generalize. Still, it shows that reasoning level, like cost, needs to be chosen to fit the task.
It is also worth noting that many of today's figures have not yet been verified. ModelBest's speeds were measured by the company itself, and DEEPX's performance figures are targets and simulation results. DEEPX released simulation results showing that pairing the chip with PIM (processing-in-memory, semiconductors that compute inside memory) would raise Llama3.1-8B throughput 6.7 times. However, it also said power consumption would rise 2.7 times. What to give up among speed, power, and quality has yet to be settled.
The takeaway for practitioners from today's news is that compute placement design now matters as much as model choice. Which tasks to finish on the device, which to send to the cloud, and how high to push reasoning levels together determine both cost and quality. Three things are worth checking going forward: whether community complaints that Fable 5.2 costs more per task than 5.1 are borne out by increased token usage, whether DEEPX reproduces its target performance on first silicon next March, and whether the published LES code achieves similar speeds on other devices.
This piece was written with the help of AI and includes personal views.
Morning briefing · 08:00
As agents grow more capable, the brakes move into infrastructure
By · bombong
The more capable agents become, the more the mechanisms for controlling them sit in access rights and execution infrastructure rather than inside the model. Today brought an agent deployed for cryptanalysis, a release model that opens the same model differently depending on the user, and a central stop mechanism that can halt work across an entire research infrastructure. All three point in the same direction.
Broader capabilities, divided access
Agents are starting to deliver results even in high-risk areas such as security. According to reports, Google DeepMind said on X that it put AI agents, supervised by researchers, to work analyzing 1024-bit RSA and obtained two results3. One is a weakness that could lead to forged digital signatures; the other is an algorithm that lowers the cost of factoring many keys at once. A claim that it cuts costs by about 30x was also reported, but the conditions have not been confirmed. Even so, it is a signal that legacy systems still using keys of this length need checking.
Anthropic chose to split distribution channels for how it releases such capabilities. According to community summaries, Claude Fable 5.2 and Mythos 5.2 share the same base model. Fable, however, is released to the public with cybersecurity and biology safeguards, while Mythos is open only to vetted security and biomedical research organizations1. This treats risk management not only as a property of the model but as a question of who uses it and under what conditions. Fable 5.2 also went live on Amazon Bedrock and GitHub Copilot on launch day. GitHub said it can be used for multi-step coding tasks in large codebases. With the general version spreading this quickly, where the safeguards sit matters more.
Stop mechanisms move into infrastructure and workflows
If dividing access is control at the distribution stage, control at the execution stage is moving into infrastructure. OpenAI introduced an internal system called Control Room. It said the system tracks model jobs running on its research infrastructure in one place and can quickly stop them if safety or security problems arise2. It is notable that OpenAI disclosed an internal operating system rather than a product for sale. It can be read as a sign that, as models run more research work directly, a stop button becomes a basic part of operations.
The same idea appears in enterprise workflows. CrossnoKaye runs verifiable Python code in Bedrock AgentCore's isolated sandbox. It said this lets it give plant engineers verifiable answers rather than guesses20. According to reports, the EGM plugin released by Google has the agent produce a design spec before writing code, and a different agent with no knowledge of the earlier conversation reviews that spec separately9. GitHub secret scanning has extended its search for exposed secrets to Actions workflow logs, a feature released in public preview8. None of the three takes agent output at face value. Instead, they surround agents with isolated execution environments, independent review, and log monitoring.
Where controls outside the model fall short
Still, it is too early to say external mechanisms alone are enough. According to reports, researchers including Anthropic published a paper showing that poisoning just 0.1% of pretraining data allowed them to plant a remote code execution (RCE) backdoor in agents built on that model10. Redoing fine-tuning and reinforcement learning on clean data did not remove the backdoor. The model reportedly behaved normally and issued malicious commands only under specific conditions. If the problem is hidden in the weights, stop mechanisms and sandboxes can limit the damage but cannot remove the cause.
Some things also remain unconfirmed. The detailed conditions of DeepMind's results have not been confirmed by official material. How Fable's safeguards actually work across its various deployment channels has not been externally verified either. The trend toward more layers of control is clear. How much each layer actually stops is a separate question.
For practitioners, today's developments mean agent adoption is hard to decide on performance comparisons alone. Access separation, isolated execution environments, review that shares no context, log monitoring, and checking the data provenance of base models all need to be weighed together. The immediate thing to check is the real cost per task of Claude Fable 5.2. Some summaries say official unit prices have fallen, but the community reports higher cost per task than with 5.1. It is worth measuring directly whether token usage has increased.
This piece was written with the help of AI and includes personal views.
Unconfirmed reports 6
Not yet confirmed by an official source. Check the original report.
- Amazon introduces HOPE, an MoE compression method that prunes Experts by grouping those activated together Original
- After Astro's struggles, Amazon is building a next-generation home robot team and hiring (via CNBC report) Original
- Claim: Nobel laureate-led committee verified an OpenAI AI-generated Yang–Mills existence proof, full text due Nov. 10 (community post, unconfirmed) Original
- ByteDance releases image generation model Seedream 5.5 with faster speed, lower API prices, and stable quality over 100 consecutive edits (QbitAI report) Original
- "MiniMax releases H3-Image, an open-weight image generation model based on H3" (community report, unconfirmed) Original
- MiniMax H3-Image follow-up: link to Hugging Face repo 'MiniMax-H3-Image' under MiniMaxAI org shared (community report) Original
Community buzz 6
Topics widely discussed on Reddit, Hacker News, GeekNews and elsewhere. These are community reactions and may include unverified claims.
- First impressions of Fable 5.2 Benchmark tables and user reviews poured in right after launch. An HN post said it scored 65 on Artificial Analysis, the highest ever. Alongside reactions that 'the gap is widening,' there is talk of Pro users' limits being reset and a one-week promotion.
- Fable 5.2 cost per task Token prices have fallen, but users complain that finishing a task costs more than with 5.1. Some posts say teams have started looking for team-level token management tools.
- Fable 5.2 science and math cases Stories circulating in the community claim that 1,000 Fable 5.2 agents formalized Fermat's Last Theorem in 30 hours and that Claude solved two open problems in black hole physics. A demo post about building a game from a three-line prompt also spread.
- Muse Ember closed-model controversy GeekNews summarized Muse Ember as supporting a 1-million-token context and multimodal input, scoring close to Haiku 5.5 at about a third of the cost. On LocalLLaMA, criticism continues that only closed models are being released despite open-source promises.
- Artificial Analysis bias debate A LocalLLaMA post claimed that whenever open-weight models catch up, Artificial Analysis reworks its metrics and pushes them down the rankings.
- Human review of personal Gemini data A GitHub repository claiming that Gemini sends personal Drive files and emails to human reviewers drew a big response on HN. Some in the community say this is a reason to use local models.