MLOps and LLMOps operations guide: keeping models running well after you build them

What MLOps is and why it is needed

The goal of every industrial machine learning project is to build an ML product and get it into production quickly. However, the MLOps overview study paper points out that many projects fall short of expectations because automating and operating them is hard. The study combined a literature review, a tool review and expert interviews to lay out the principles, components, roles and architecture MLOps requires, and to give a definition.

The Google Cloud documentation source says the real challenge is not building the model. What is hard is building an integrated ML system and keeping it running in production. In real systems, ML code is a small part, while surrounding elements such as configuration, data collection, data validation, resource management, serving infrastructure and monitoring are broad and complex. The CD4ML article source makes the same point, citing Google's 2015 paper 'Hidden Technical Debt in Machine Learning Systems.' ml-ops.org source sums up the purpose of MLOps as avoiding the 'technical debt' that builds up in machine learning applications.

How does it differ from ordinary software? The CD4ML article source explains that ML applications change along three axes: code, model and data. The Google Cloud documentation source notes that even when the code is fine, model performance can drop if the nature of incoming data keeps changing. So you need to track summary statistics of the data and the model's online performance, and send alerts or roll back to a previous version when they deviate from expectations. In short, MLOps is an engineering culture and practice that brings DevOps principles to ML systems, unifying development (Dev) and operations (Ops) source.

Components: what it is made of

First, look at the flow a model goes through to reach production. The Google Cloud documentation source describes it in this order: data extraction, data analysis, data preparation (cleaning, splitting into training, validation and test sets, feature engineering), model training, model evaluation, model validation (checking it beats the baseline), model serving and model monitoring. There are three serving options: online prediction through a microservice with a REST API, embedding the model on edge or mobile devices, or running it as part of a batch prediction system.

Automating this flow requires tools. The components listed by ml-ops.org source are: source control (versioning code, data and models), test and build services, deployment services, a model registry that stores trained models, a feature store that processes inputs the same way in training and serving, an ML metadata store that records model names, parameters, training/test data and metric results, and a pipeline orchestrator that automatically chains experiment steps.

The CD4ML article source uses a similar list: discoverable and accessible data, reproducible model training, model serving, testing and quality, experiment tracking, model deployment, continuous delivery orchestration, and model monitoring and observability. ml-ops.org source also explains why versioning is needed. You must be able to access every production model version when retraining with new data or new training methods, when performance degrades over time, when a model must be fixed after an attack, when you need to roll back quickly to a previous version, and when responding to audits by regulators or the company.

Automation maturity: from manual to fully automated

The Google Cloud documentation source and ml-ops.org source hold that how far these steps are automated determines the maturity of an ML process. The more mature it is, the faster models can be retrained with new data or new implementations. ml-ops.org source divides this into three levels. The first is a manual process, where people do everything from data preparation to training and testing by hand with tools such as Jupyter notebooks. The second is ML pipeline automation, where retraining starts automatically when new data arrives and data and model validation steps are included. The third is CI/CD pipeline automation, where data, model and training pipeline components are built, tested and deployed automatically.

Microsoft's Azure documentation source breaks this into five finer levels. At Level 0, 'No MLOps,' building, deployment, training and testing are all manual, and model performance is not tracked in one place. At Level 1, 'DevOps but no MLOps,' building and testing of application code are automated, but results are hard to track and reproduce. At Level 2, 'Automated training,' training is automated, training performance tracking is centralized and model management is established. Deployment is still manual but easy. At Level 3, 'Automated model deployment,' deployment is automatic, deployed models can be traced back to the original data, and A/B testing is part of the deployment process. At Level 4, 'Full MLOps automated operations,' signals of drift (data or performance gradually shifting) or performance degradation trigger retraining automatically.

The levels are not only about technology. According to the Microsoft documentation source, at Level 0 data scientists, data engineers and software engineers work separately, and the output is often a single model file handed over manually. From Level 2, data scientists and data engineers work together to turn experiment code into scripts that can be run repeatedly. The documentation notes that it is common for one organization to show traits of several levels at once, and recommends viewing the levels as a continuum rather than separate steps.

CI, CD and CT: how continuous integration changes in machine learning

The two pillars of DevOps are continuous integration (CI) and continuous delivery (CD). The Google Cloud documentation source explains that both take on broader meanings in ML. CI tests and validates not only code and components but also data, data schemas and models. CD does not ship a single software package; it deploys a system, the training pipeline, which in turn automatically deploys the prediction service. On top of this comes CT (Continuous Training), which automatically retrains and serves models and is unique to ML systems. ml-ops.org source also adds CM (Continuous Monitoring), meaning ongoing monitoring of production data and model performance metrics linked to business metrics.

Testing also expands. The Google Cloud documentation source says that in addition to ordinary unit and integration tests, data validation, quality evaluation of trained models and model validation are needed. As triggers for retraining, ml-ops.org source lists schedules, messages, monitoring events, and changes to data, training code or application code.

The CD4ML article source summarizes these principles as 'in small, safe increments, reproducibly, and always in a deployable state.' If software is always deployable, when to deploy becomes a business decision rather than a technical one. The article aims to shorten development cycles to days or hours instead of weeks or months, and to build a feedback loop that learns from model behavior in production to improve the model.

What changes in LLMOps

Microsoft documentation source defines GenAIOps (also called LLMOps) as the practices and strategies for managing large language models (LLMs) in production. Databricks source views LLMOps as an extension of MLOps for LLMs and explains that most MLOps operational requirements still apply. Microsoft's MLOps maturity document source also states that GenAIOps does not replace the MLOps maturity levels but complements them, adding capabilities such as prompt lifecycle, retrieval augmentation (RAG), output safety and token cost.

Databricks source describes in detail what changes. First, rather than training a model from scratch, teams often start from a foundation model and fine-tune it on new data. Second, specialized hardware such as GPUs becomes essential for both training and deployment, and inference costs make model compression and distillation techniques important. Hyperparameter tuning is also used to reduce cost, not only to improve accuracy. Third, prompt engineering, which refines prompt templates, becomes a core task. It is also used to reduce the risks of hallucination, prompt injection, sensitive data leakage and jailbreaks. Fourth, building pipelines that chain multiple LLM calls with tools such as LangChain or LlamaIndex and connect them to vector DBs or web search becomes more central to development than building new LLMs.

Evaluation and observability also change. Traditional ML uses metrics with clear calculations, such as accuracy, AUC and F1, but LLM evaluation needs different metrics such as BLEU or ROUGE source. LLM tasks often have no single correct answer, so end-user feedback is important for evaluation. Feeding this feedback into the pipeline makes evaluation easier and also provides data for later fine-tuning source. Microsoft documentation source recommends evaluation metrics such as groundedness, relevance and similarity, along with content safety. Databricks source recommends setting alerts in the monitoring pipeline for both model drift and malicious user behavior.

Microsoft documentation source divides GenAIOps maturity into four stages: initial, defined, managed and optimized. At the initial stage, teams learn various LLM APIs and start structured prompt design and basic evaluation metrics. At the defined stage, they move to CI/CD integration, advanced evaluation metrics and content safety. At the managed stage, they strengthen proactive monitoring, fine-tuning, versioning and rollback.

Step-by-step adoption

The direction the source materials commonly recommend can be arranged as a sequence for getting started. First, start with a single use case. ml-ops.org source recommends handling only one ML use case at a time, and says that in the design phase you should examine the data you need and define the model's functional and non-functional requirements so you can design the architecture, serving strategy and test suite in advance.

Second, set up versioning and experiment tracking. Version not only code but also data and models, and record in a metadata store which parameters and data produced which metrics sourcesource. In terms of Microsoft's maturity model source, this is preparation for moving from Level 1 to Level 2. Third, automate the training pipeline, add data validation and model validation, and then store trained models in a model registry sourcesource. Fourth, automate pipeline and model deployment with CI/CD and add A/B testing (Level 3 in w3). Fifth, connect production monitoring so that drift or performance degradation signals trigger retraining (Level 4 in w3).

For LLM applications, you can follow the sequence in Microsoft documentation source. First become familiar with LLM APIs and structured prompt design, introduce basic evaluation metrics, and then expand to CI/CD integration, advanced evaluation such as groundedness, and content safety. Microsoft documentation source recommends using the maturity model to assess your current state and set realistic success criteria, and to build capabilities gradually rather than taking on the complexity of a full implementation from the start.

Checklist, common mistakes and limitations

The items to actually check are as follows. Are code, data and models all versioned? Are experiment results and training performance recorded in one place? Are there data validation, model quality evaluation and model validation tests? Do you confirm that a new model beats the baseline before deploying it? Can you quickly roll back to a previous serving version? Do you monitor data summary statistics and online performance and get alerts when they deviate? If an audit request comes in, can you access every version of the models and data you ran in production sourcesourcesource? For LLM applications, add: Do you manage prompt versions and lifecycle? Do you evaluate with metrics such as groundedness? Do you watch token cost? Are alerts set for drift and malicious use sourcesourcesource?

There are also common mistakes. The CD4ML article source names as the first problem an organizational structure in which data engineers, data scientists and ML engineers each "throw their part over the wall" to the next team without agreeing on expectations. It is also common, as in Microsoft's Level 0 source, to write scoring scripts by hand after experiments and not version them. Treating the work as done based only on offline test set performance is also risky, because performance drops in production when the data changes source.

It is also worth knowing the limitations. The MLOps overview study paper points out that MLOps is still a vague term and that its meaning for researchers and practitioners is unclear. Google Cloud documentation source mainly covers predictive AI systems, and Microsoft's maturity document source also centers on predictive, tabular and traditional ML. So these are hard to apply directly to LLMs. Keep in mind too that tools change quickly. Microsoft documentation source announced that its Prompt flow will be retired on April 20, 2027. It is safer to learn the principles before specific tools.

Further study

If you are new, start with the Google Cloud documentation source. It explains the difference between DevOps and MLOps, the stages of ML development, CI, CD and CT, and the maturity levels in a single flow. Then review the component table, continuous X (CI, CD, CT, CM) and the reasons for versioning in the ml-ops.org principles article source to get the practical vocabulary down.

For a feel of real implementation, the CD4ML article source is a good fit. Using a sales forecasting example application, it walks end to end through data, training, serving, experiment tracking and monitoring. To diagnose where your organization stands, look at the people, process and technology tables for each level in Microsoft's MLOps maturity document source. The MLOps overview study paper summarizes the academic definition and the open challenges.

On the LLM side, we recommend first reading the Databricks article source to see how it differs from MLOps, then checking Microsoft's GenAIOps maturity document source to see what to expand at each stage.