AI Trend Research
EN

← AI glossary

AI glossary · Basic · Efficiency & infra · Language models · AI platforms & solutions

Flide

Flide is open-source inference infrastructure for running Jev-family "decision models" quickly. It uses an "exact state forking" approach: the same input is computed only once, and from that state the process branches into multiple questions, each producing its own answer.

About this term
Level
Basic · Technical terms you often meet in AI news
Added
2026-11-10
In plain words It is similar to a staff member who has carefully read a long case record once and then gets a series of questions such as "Should we refund?" and "Is this urgent?" Instead of rereading the record from the start for each question, they answer right away from what they already read. Unlike human memory, Flide makes sure each answer comes out exactly the same as if the question had been asked on its own.

What it is

Flide is inference infrastructure published on GitHub (OmniJev), built for running Jev-family decision models that pick one option from a fixed set of choices. Inference infrastructure here means the execution engine that sends requests to a trained model and returns its answers. According to the GeekNews introduction, it supports chat models such as Qwen3.5 and Gemma 4 as well as several decision models including OneJev, and depending on the model it also accepts images or video as input. It is licensed under Apache-2.0. Version 0.1.10 was released on November 9, 2026, and since it is still in the 0.1 range, it can be considered early-stage.

How it works

The core idea is "exact state forking." Input shared by several questions (for example, order details and a refund request) is processed once first to build the model's intermediate state. Then the process branches from this state for each question, producing each answer with a single forward pass (one computation through the model). Each answer is said to be the same as if that question alone had been appended to the same input and run separately. Users send one shared input and multiple questions together through the /v1/systemone API, and each question can have up to 256 options.

Shared input once, forked per questionShared input → Model state(Process); Model state → Question 1(Fork); Model state → Question 2(Fork); Question 1 → Answer 1; Question 2 → Answer 2Shared inputOrder and refundrequestModel stateComputed onceQuestion 1Refund, exchange, orescalate?Question 2Reply within 1 hour?Answer 1Answer 2ProcessForkFork
Shared input once, forked per question The common input on the left is computed once to build a state, and from that state the process branches for each question to produce separate answers. The questions are taken from the example in the introduction.

Why it is getting attention

Many tasks, such as email classification or customer inquiry handling, require several judgments on the same text, and existing approaches recomputed the same input for every question. The Flide team says it is up to 9.5x faster than general-purpose inference engines such as vLLM and SGLang. It also reports that applying 8 questions to each of 10,000 emails took 28 seconds with a single H200 GPU and OneJev 0.8B. It also includes operational features such as a session cache that keeps the state of earlier requests so an agent's next step only reads the newly added part, LoRA adapter serving, and FP8 quantization.

Things to know (limitations)

The performance figures come from the developers, and the introduction itself notes that the size of the gains varies widely by model and task. For some models and tasks, the results are barely different from existing engines. The benefit is larger for decision-style requests that pick from options than for chat that writes long free-form text. The --batch-invariant option, which makes results bit-for-bit identical no matter which requests they are batched with, sacrifices some speed. It is a good idea to check directly with flide verify whether forked results really match for your own models and requests.

Example

Suppose a customer service center receives the inquiry "It arrived with a cracked screen, please refund me." A system using Flide reads this inquiry once as shared input, then judges two questions separately from the same state: "Should we refund, exchange, or escalate to a human?" and "Is it urgent enough to require a reply within an hour?" The more inquiries there are, in the thousands, and the more questions there are, the more waste from rereading the same text is cut.

Related terms

Inference engine Decision model Jev KV cache Prompt caching Continuous batching

References