CUSTOSKUS-tos
AI Stewardship
Custos is Latin for “keeper.” AI Stewardship keeps your production AI measured, current, and improving.
Production AI that keeps improving.
Is this for you?
- Businesses running AI without their own engineering team
- Systems that should improve as models improve
- Owners who want one accountable keeper
The situation
Production AI needs a keeper.
Your AI system is live and part of daily work. The models underneath it change, costs move, and the work it handles shifts over time.
Many businesses running AI have no engineering team of their own to watch it. Someone still needs to notice when quality moves, judge whether a new model is worth adopting, and keep a safe way back.
You want one accountable keeper, with every change measured and every release approved by your own team.
Our approach
Measured against the baseline you accepted.
Stewardship starts from the accepted baseline: the quality, cost, and outcomes your system delivered when you accepted it. We monitor the signals you agree and evaluate current performance against that baseline.
When a new model or technique looks worthwhile, we prepare it as a candidate. It is adopted only after it passes technical, security, licensing, and economic checks.
Reviewed corrections can inform the next candidate, and your named approver reviews the evidence before any release. Rollback stays in place, and reporting covers the measures agreed for your plan.
Use cases by industry
Where this service fits.
Typical applications across industries. They show where the service applies, not past client work or results.
- Insurance agency
Keeping a renewal preparation agent current
An agent prepares renewal packets from the agency management system, and an account manager approves each packet before it reaches a client. Stewardship tracks packets accepted as prepared, exceptions routed to staff, and cost per accepted packet against the accepted baseline. When a carrier changes its forms, we prepare and test an update, and the operations lead approves its release.
- Distributor
Order intake that keeps pace with catalog changes
An agent reads emailed purchase orders and drafts sales orders in the ERP, and a customer service lead releases each draft to the warehouse. As products, units, and customer part numbers change, stewardship measures line accuracy and exceptions routed against the accepted baseline. Reviewed corrections feed the next candidate update, which the operations manager approves before its release.
- Logistics
Watching a shipment exception workflow
An AI workflow sorts carrier status messages and drafts customer notices for delayed shipments, and a dispatcher approves each notice before it is sent. Stewardship monitors classification accuracy, notices sent without edits, and running cost per shipment handled. When a newer model looks suitable, we test it on the company’s own message history before the dispatch manager decides.
- Property management
Maintenance request triage kept measured
An agent reads tenant requests, sets a priority, and drafts a work order in the property management system for a manager to approve. Stewardship compares suggested priorities with each reviewer’s final decision, and investigates any shift from the accepted baseline as buildings and vendors change. Updates are prepared as candidates, approved by the portfolio manager, and released with the previous version kept ready for rollback.
- Staffing agency
Candidate summaries under steady review
An AI workflow summarizes applications against each job order in the applicant tracking system, and a recruiter decides every next step. Stewardship checks summary accuracy and reviewer edits against the accepted baseline, and watches for shifts as clients, roles, and job markets change. Any update to the model or its instructions goes to the agency’s named approver with its evaluation evidence.
- E-commerce
Product listing drafts that stay accurate
Agents draft product descriptions and attribute data from supplier sheets, and a merchandiser approves each listing before it goes live. Stewardship measures drafts accepted without edits, attribute accuracy, and cost per accepted listing. When a lower-cost model meets the baseline on the same test set, the merchandising lead reviews the evidence and decides whether to adopt it.
- Clinic group
Referral intake that handles sensitive records
An AI workflow reads incoming referrals, extracts patient and payer details, and drafts an intake record for the scheduling team. A staff member confirms each record, and access to sensitive records stays within the systems agreed in scope. Stewardship checks field accuracy and exceptions routed to staff, and the practice manager approves every change before release.
- Field services
Service report drafting kept current
Technicians dictate job notes, and an agent drafts the service report and parts list in the field service system. The service coordinator approves each report before it reaches the customer or the invoice. Stewardship tracks reports approved as drafted and parts lines corrected, and prepares updates for review when equipment models, service codes, or price lists change.
What you receive
Evaluation and monitoring
Quality, cost, and outcomes reviewed within the agreed coverage.
Model upgrades
New models and techniques adopted only after they pass technical, security, licensing, and economic checks, with rollback preserved.
The learning loop
Reviewed corrections can inform candidate improvements. People approve changes before release.
Reporting by agreement
Agree which operating measures and recommendations belong in your reporting scope.
How it works
- MonitorReview agreed quality, cost, and operating signals.
- EvaluateCompare current performance with the accepted baseline.
- Prepare improvementsScope updates or retraining when the evidence supports them.
- ApproveYour named approver reviews the evaluation and release plan.
- Release and reviewDeploy the approved version and retain the agreed rollback route.
How success is measured
The measures your approver signs.
Each measure goes into the acceptance criteria with its test data, threshold, and the person who checks it.
- Quality against the accepted baseline
- Current outputs are scored with the same rules and test set used at acceptance. Any movement is reported with its likely cause.
- Exceptions routed correctly
- Measures whether cases the system should not handle reach a person. It is checked by sampling routed and unrouted work against reviewer decisions.
- Cost per accepted outcome
- Running cost divided by the outcomes your reviewers accept, tracked from usage records. It shows whether a model or volume change affects the economics.
- Reviewer edits and effort
- How often reviewers change a draft, and how much they change. A rising edit rate is an early signal that the work has shifted.
- Release and rollback readiness
- Each release has an evaluation, an approval record, and a tested route back to the previous version. It is checked at every release review.
Where care is needed
What we watch, and how it is handled.
- Model change underneath the system
- Hosted models are updated and retired on their provider’s schedule. We track those changes for the models you use, and test any replacement on your own work before your approver decides.
- Shifts in the work itself
- New customers, rules, and record types change what the system sees. We ask your owner to share business changes, and compare live work with the test set so the baseline stays representative.
- Access and credentials
- Stewardship needs access to the environments the system runs in. We work with the least access the scope needs, record what is used, and leave credential control with your team.
- Scope drift
- Ongoing care can grow into new projects without a clear decision. We keep the plan’s coverage in writing, and raise new work as a separate proposal for your approval.
- Ownership and exit
- Your business owns the system, the baseline, and the evaluation records. If you bring care in-house, we hand over the records, the release history, and the rollback routes.
Who does what
Your team decides. We engineer.
Your team
- Name the approver for updates and releases
- Agree the signals, coverage, and reporting for your plan
- Keep access to the systems in scope current
- Share business changes that affect the work, such as new rules or new record types
- Decide which responsibilities your team retains
Sophrono
- Monitor the agreed quality, cost, and operating signals
- Evaluate current performance against the accepted baseline
- Prepare model updates and improvements with evaluation evidence
- Check each candidate for technical, security, licensing, and economic fit
- Release only approved versions, with rollback kept in place
At the end
The decisions you make next.
The service ends with evidence and a choice. Each option is yours, and none is assumed.
Continue the plan
Keep the current coverage and review the measures at the next agreed point. Your approver keeps deciding each release.
Change the plan
Move between Foundation, Continuity, Operations, or a Custom plan as your operating requirements change. Scope for the new plan is agreed before it starts.
Fund one targeted improvement
Where the evidence points to one clear gain, Iteration Sprints target one agreed metric at a fixed price. The result is then measured within stewardship.
Hand it to your team
When your team is ready to keep the system, we hand over the baseline, evaluation records, and release history. Your team then owns every release decision.
Before we start
What to have ready.
- The system in scope, and its accepted baseline if one exists
- Documentation and access for the environments it runs in
- The measures that matter most to your business
- A named approver for changes
Related services
Stewardship options
Pick the option closest to the coverage your system needs. Scope, capacity, and reporting are agreed for your system, and people approve every release.
What “accepted” means
Measured against criteria you agree to in advance.
- The report covers the agreed operating signals.
- Each proposed update includes evaluation evidence.
- Human approval is recorded before release.
- Coverage matches the agreed stewardship scope.
Full engagement terms are finalized in a Master Services Agreement.
Talk about stewardship
Tell us about the system to look after.
We reply with a short assessment conversation. Scope and capacity are agreed for your system.
- A senior engineer reads every request
- A reply by email with the next step
- No obligation until scope and price are agreed
Not ready to scope this? Ask an engineer first: a free 15-minute call that names the agentic systems that could fit.
The Canon rule behind this service
AI Stewardship answers to Canon VII.
Where this fits
- Learn Research
- Explore free AI Workload Evaluation
- Try one workflow Diagnostic
- Plan and validate System Blueprint
- Build Custom Agentic Systems
- Improve and operate AI Stewardship
Not ready yet? Iteration Sprints. After this: Agent Assurance or Research-to-Production.
Questions
What is included?
Scope, capacity, and reporting are agreed for the selected tier or custom proposal.
Who approves changes?
Your named approver reviews changes before deployment.
Can our team operate the system?
Yes. We can agree the responsibilities your team retains and the coverage Sophrono provides.
What are the plans called?
Foundation, Continuity, Operations, and Custom. Scope is agreed for the plan you choose, and a Custom plan requires a scoping deposit.
Can you look after a system we did not build?
Often, yes. We start by agreeing a baseline on the system as it runs today, so every later change is measured against it.
How do you decide a new model is worth adopting?
It must meet the accepted baseline on your own work and pass technical, security, licensing, and economic checks. The evidence then goes to your approver, who decides.
Does stewardship cover retraining?
Retraining can be scoped as an improvement when the evidence supports it. Like any update, it is evaluated and approved before release.
How are fees set?
Fees are set with the scope agreed for your operating requirements. Full engagement terms are finalized in a Master Services Agreement.
What happens when something goes wrong in production?
The agreed monitoring signals raise it, and the previous approved version stays available. How issues are handled and reported is set in the scope you agree.
Do we need our own engineers?
No. Stewardship suits businesses without an engineering team. Your side names an owner who approves releases and decides priorities.
How does stewardship start?
With a handover review of the system, its records, and its acceptance criteria. Your owner and our engineers then agree the plan, the measures, and who approves each change.
Which plan is right for us?
That depends on your operating requirements, which we agree with you before a plan starts. Scope, coverage, and reporting are then set for the plan you choose.
Do you change our system without our approval?
No. Every update is prepared as a candidate, evaluated, and released only after your named approver agrees. The previous approved version stays available for rollback.
What is the learning loop?
Corrections your reviewers make are recorded and reviewed. They can inform the next candidate improvement, which is evaluated against the baseline and approved by a person before release.
Can Research-to-Production run inside stewardship?
Yes. Research-to-Production is the dedicated pipeline for evaluating new research, models, and tools, and it can run inside a stewardship plan. Each candidate still needs your approver’s decision.
Who owns the records stewardship produces?
The run records, the baseline, and the evaluation history are kept for your business, with access limited to what the agreed scope needs. Ownership terms are finalized in the Master Services Agreement.
Can one plan cover more than one system?
Yes, where the plan is scoped for it. Each system keeps its own baseline, its own measures, and a named approver for its releases.
What reporting do we receive?
Reporting covers the operating measures and recommendations agreed for your plan. It sets current performance against the accepted baseline, so any change is clear to your owner.
How does stewardship treat running cost?
Cost per accepted outcome is one of the signals we can agree to watch. When a lower-cost model or configuration meets the baseline, it is prepared as a candidate for your approver.
How do you handle a model its provider retires?
We track announced changes for the models your system uses. A replacement is prepared as a candidate, tested on your work against the baseline, and released only after your approver agrees.
Does stewardship replace our internal owner?
No. Your owner keeps the business decisions: priorities, which responsibilities stay in-house, and approval of every release. Stewardship supplies the engineering, the evidence, and the release work.
How do we raise a change in our business rules?
Share it with us as it happens, such as a new rule, product, or record type. We assess its effect on the system and prepare any update as a candidate for your approver.
What does our approver need to know?
The business outcome, not the engineering. Evidence is presented in terms of the agreed measures, with the risks and the rollback route stated plainly.