Operating Agentic Apps: A ValuScope Workflow and Measurement Guide
An operations guide for running approval-gated AI agents from deal intake to IC, with KPIs measuring quality, speed, safety, and adoption.
An operations guide for running approval-gated AI agents from deal intake to IC, with KPIs measuring quality, speed, safety, and adoption.
Operating Agentic Apps: A ValuScope Workflow and Measurement Guide
Version note: This article reflects the
valuscope-nextscreens, manual, and code as of July 12, 2026. No public production statistics establish a ValuScope labor-reduction rate or investment-performance effect. I therefore separate effects visible in the implementation from KPIs that still need measurement.
Part one covered design philosophy, and part two covered implementation. The final part is about operation. When I helped standardize review quality in an investment team, I learned that a capable tool is not enough. An operating model must say who reviews what, in which order, against which acceptance standard.
TL;DR: What You'll Learn
The usage and measurement conclusions are below.
| Perspective | Practical point |
|---|---|
| Starting point | Prepare the Deal and Fund Profile, then begin with public Web Research |
| Documents | Store only authorized material and pass it to Deck Analysis and later agents |
| Human decision | Review the artifact, proposed stage, and next agent in the inbox |
| Improvement | Record rejection reasons and use them to change input or prompts on rerun |
| Observable effects | Standardized work, visible state, artifact lineage, explicit decision points |
| Effects to measure | Labor, approval rate, reruns, quality, stage dwell time, and cost effectiveness |
Measure whether decision-relevant evidence arrives faster, more completely, and with a clearer audit trail—not how many words the model generated.
Prepare Three Foundations Before Running an Agent
Launching an agent without context tends to produce generic analysis. ValuScope grounds work in three sources: Deal, Fund Profile, and Document.
| Foundation | Minimum information | Effect on output |
|---|---|---|
| Deal | Company, website, industry, stage, description | Web research and entity accuracy |
| Fund Profile | Fund size, sectors, stage, geography, ticket | Fund fit and issue prioritization |
| Document | Deck, financials, meeting notes, references | Company-specific hypotheses, numbers, gaps |
Fund Profile is easy to overlook. The same startup should be evaluated differently by a small seed fund and a large late-stage vehicle. The June 4 change that added fund data to agent context moved ValuScope from general company analysis toward analysis for this investor.
More documents are not automatically better. An old deck, new financial model, and undated meeting note may contain conflicting numbers. Keep filenames, document types, dates, and sharing rights clean enough that a reviewer can identify primary evidence.
Step 1: Begin Screening With Public Information
Web Research is the natural first agent. It uses public-safe information such as company name, website, industry, and stage to inspect official claims, news, and third-party evidence.
Before execution, verify four things:
- The company is not operating in stealth.
- The name and description may be sent to web search and the OpenAI API.
- The input excludes nonpublic revenue, valuation, and personal information.
- The company website identifies the correct entity.
ValuScope displays a dedicated confirmation for web-search runs. The extra click matters because it reframes the action from “run AI” to “send these fields to external services.”
When reviewing output, look past fluency and check:
- Are source URLs and dates present?
- Are company claims separated from third-party verification?
- Do market figures share a definition, geography, and date?
- Is missing information explicitly marked unknown?
- Does the output identify claims that must be verified from documents?
The useful acceptance criterion is not simply pass or fail. It is whether the research produced the right next verification tasks. Completing an investment view from public research alone would systematically favor companies with stronger public communication.
Step 2: Make the Thesis Company-Specific With Document Analysis
After Web Research, use Deck Analysis when a pitch deck exists. If no source document is available, the workflow can move to Meeting Agenda instead, avoiding a complete stop while documents are pending.
Review Deck Analysis against explicit criteria.
| Dimension | Strong output | Warning sign |
|---|---|---|
| Numbers | Unit, period, currency, and source page align | Revenue and ARR or monthly and annual figures mix |
| Fact vs estimate | Estimates include assumptions and confidence | Market share absent from the deck is stated as fact |
| Investment thesis | Conditions and kill criteria appear together | Only strengths are listed |
| Moat | Durability is explained through a framework such as 7 Powers | Analysis stops at “unique technology” |
| Missing information | Owner, timing, and evidence request are specific | “Further review needed” ends the section |
When I investigated a yield issue that had been attributed to equipment, I avoided locking onto one story and asked what observation would falsify each hypothesis. Agent review needs the same discipline. The goal is not a plausible narrative but a falsifiable set of questions.
ValuScope can save Deck Analysis as a DOCX Document, allowing later agents to reuse the artifact. That introduces a feedback risk: an AI-generated document can be mistaken for primary evidence. The implementation excludes generated files with the AI- prefix when deciding whether source documents exist. It is a small but important guard.
Step 3: Read What Approval Will Do in the Inbox
When a run finishes, an AWAITING_REVIEW card appears in the action inbox. The card shows the deal, agent, model, summary, proposed stage, and next agents.
Reviewing in three layers makes the process repeatable.
| Layer | What to inspect | Example decision |
|---|---|---|
| Evidence | Sources, numbers, dates, document consistency | Reject on a factual error |
| Analysis | Thesis, falsifiers, risks, missing data | Rerun with a specific missing angle |
| Action | Proposed stage and next agent | Do not approve if the deal should not advance |
“Well written” and “ready to advance” are separate judgments. The card presents them together, but the operating procedure should keep them distinct.
Approval moves the workflow forward only and launches the next step. Web Research proposes SCREENING; Deck Analysis or Meeting Agenda proposes REVIEW; Question List proposes IC. IC Proposal does not automatically move a deal to CLOSED. The investment or pass decision remains manual.
When rejecting, record a reason that can change the next run. “Low quality” is not actionable. “Two of three comparable companies are private,” “ARR and revenue were conflated,” or “a 2024 deck was treated as current” gives the team a concrete correction.
Step 4: Operate Meeting, Review, and IC as One Chain
The agents become easier to evaluate when their artifacts form one chain instead of six isolated tools.
Meeting Agenda
Define the meeting objective as distinguishing advance from pass, not collecting as much information as possible. Every priority question should state how its answer changes the thesis. A killer question protects scarce meeting time.
Question List
A 20–30 question follow-up list should include the question, intent, expected evidence, and decision threshold. A document alone does not create progress. Adding owner, due date, answer, evidence link, and unresolved status turns the list into review operations.
IC Proposal
Review TAM/SAM/SOM, base/bull/bear cases, MOIC, pre-mortem, and missing evidence. Polished prose can hide unverified assumptions. Start with what must be true for the recommendation to hold.
The chain is not intended to replace judgment. It organizes the entrance to each meeting and leaves explicit verification tasks at the exit.
Step 5: Separate Chat Queries From Change Proposals
ValuScope chat can answer from portfolio and deal context or construct a data-change proposal. “List companies with short runway” is a query. “Move Company A to REVIEW” is a change request.
For changes, the interface compares current and new values and lets the user select fields. If the database changed after proposal generation, conflicting fields are skipped. In a team environment, that conflict is not a failure. It protects a newer human edit from an older AI proposal.
A useful operating rule is:
- Use chat broadly for search, aggregation, summarization, and comparison.
- Receive database mutations as proposals and inspect the diff.
- Confirm count and targets before any multi-record change.
- Link financial or stage changes to the supporting document or meeting history.
Chat can be the entry point, while forms, types, history, and approval remain the data-quality boundary.
Effect 1: What the Implementation Already Demonstrates
Even without public production statistics, the code and specification demonstrate several functional effects. These are features that exist, not speculative productivity claims.
| Effect | Implementation evidence | Meaning |
|---|---|---|
| Work standardization | Six shared personas, skills, and output formats | Reduce variation in review dimensions |
| State visibility | Run, ActionItem, and inbox grouping | Separate running, review, failed, and completed work |
| Explicit judgment | Approval, rejection, rerun, and decision note | Separate generated output from human decision |
| Artifact reuse | Document, DOCX/XLSX, and sourceDocIds | Reuse work in later agents and meetings |
| Forward-only safety | isForwardStage | Prevent accidental backward transitions during chaining |
| Concurrent-edit safety | currentValue conflict detection | Avoid overwriting newer data with stale proposals |
The most important effect may not be labor reduction. It may be visibility into where work is waiting. A review often stalls on missing documents, unclear ownership, or pending review rather than analysis itself. The inbox is valuable as a decision queue, not merely an AI output folder.
Effect 2: KPIs to Measure After Adoption
Do not announce “50% faster” before measurement. Capture four weeks of baseline, then compare a four-to-eight-week adoption period. Deal difficulty varies, so stratify by stage, industry, and document completeness where possible.
| KPI | Definition | Desired direction | Caveat |
|---|---|---|---|
| Time to first review | Run start to AWAITING_REVIEW | Lower | Separate model speed from document volume |
| Human review time | Review start to approve/reject | Lower | Superficial review can look fast |
| First-pass approval rate | Approved without rerun | Higher | Stratify hard cases |
| Rerun rate | Same agent type rerun | Lower | Intentional iteration is not necessarily bad |
| Rejection reason coverage | Rejections with a concrete reason | Higher | Audit meaningless free text |
| Stage dwell time | Days spent in each stage | Lower | A well-reasoned pass is not delay |
| Evidence defect rate | Missing source, wrong number, wrong date | Lower | Requires sample audits |
| Proposal conflict rate | Updates skipped because DB changed | Contextual | A high rate may indicate stale proposals |
| Cost per approved artifact | API and worker cost per approval | Lower | Low cost is useless with low quality |
| Downstream reuse rate | Artifact referenced in a later step or meeting | Higher | Do not count download alone as use |
You do not need a large analytics platform to begin. Existing Run and ActionItem timestamps support derived metrics like these.
type AgentMetric = {
runId: string
agentType: string
queuedAt: string
completedAt: string | null
decidedAt: string | null
decision: 'APPROVED' | 'REJECTED' | 'DISMISSED' | null
rerunOf: string | null
tokensUsed: number
evidenceDefects: number
}
const minutesBetween = (from: string, to: string | null) =>
to ? (Date.parse(to) - Date.parse(from)) / 60_000 : null
const summarize = (m: AgentMetric) => ({
generationMinutes: minutesBetween(m.queuedAt, m.completedAt),
decisionMinutes: m.completedAt ? minutesBetween(m.completedAt, m.decidedAt) : null,
firstPassApproved: m.decision === 'APPROVED' && m.rerunOf === null,
costProxy: m.tokensUsed,
evidenceDefects: m.evidenceDefects,
})
Keep definitions identical across teams and languages. If “completed,” “approved,” and “used” mean different things to different groups, a polished dashboard will not produce comparable evidence.
A Four-Week Adoption Plan
Roll out the lowest-risk agents first instead of asking everyone to use all six immediately.
| Week | Scope | Exit criterion |
|---|---|---|
| 1 | Web Research on three to five deals | Data confirmation and source review become routine |
| 2 | Deck Analysis and Meeting Agenda | Review standards for numbers and gaps are aligned |
| 3 | Question List and inbox approval | Reasons, reruns, owners, and due dates are operational |
| 4 | IC Proposal and chat change proposals | The team confirms the boundary around final decisions |
Each week, manually compare at least two samples with source documents. Approval rate alone rewards a team for weak review, so pair it with sampled evidence-defect audits.
Define excluded cases upfront. Stealth companies, high litigation or regulatory sensitivity, special-category personal data, and documents that contractually cannot reach an external API should remain outside the workflow. Exclusion is not failure; it is part of the operating design.
Gotchas That Destroy the Effect
Several operating habits can erase the value of the architecture.
- Rubber-stamp approval: if reviewers do not read outputs, human-in-the-loop becomes decorative audit logging.
- Artifact overproduction: similar generated documents make the current version unclear. Preserve lineage and versioning.
- Single-metric optimization: minimizing time can lower quality, while maximizing approval can avoid hard cases.
- Confirmation fatigue: repetitive warnings become invisible. Show concrete fields and distinguish higher-risk actions visually.
- Prompt-only improvement: missing input, stale documents, and incorrect workflow state cannot be fixed by prompt wording.
- Self-reinforcing artifacts: if generated documents are treated as primary evidence, errors gain apparent authority.
During early adoption, I would accept slower reviews in exchange for specific decision notes. Without that evidence, the team cannot later determine whether to change the model, prompt, input, or workflow rule.
Conclusion: Measure Decision Quality, Not Automation Rate
Operating ValuScope is not a sequence of agent buttons. It is a process that separates public information, internal documents, and human judgment, then passes each artifact to the next verification step.
The implementation demonstrates standardization, state visibility, artifact lineage, approval boundaries, and conflict prevention. Labor reduction, investment outcomes, and quality improvement still require measurement. Preserving that distinction is part of earning trust in an AI product.
The target state is not an AI that selects investments. It is an AI that surfaces missing evidence and falsifying conditions sooner, enabling people to make more explainable decisions. Even with a modest automation rate, higher decision reproducibility would be a meaningful effect.
- Part 1: Designing Agentic Apps With Human-Controlled Automation
- Part 2: How to Architect an Agentic App With Next.js, Amplify, and Lambda
- Follow-up: Handing a Solo VC Workflow to AI Agents (Work OS)
References
- OpenAI API Developer Quickstart
- OpenAI API Data Controls
- OpenAI Enterprise Privacy
- AWS Amplify Gen 2: Set Up Storage
- AWS Lambda: Capturing Asynchronous Invocation Records
- TanStack Query v5: Polling
Next Issue Ideas
- Idea 1: Implement an agent KPI dashboard — Calculate approval, rerun, dwell time, token cost, and evidence-defect metrics from Run and ActionItem data.
- Idea 2: Design a sample audit for AI output — Turn source comparison, severity grading, and reviewer calibration into a repeatable operating process.
- Idea 3: Define a staged path to auto-approval — Select low-risk cases using historical defect rate and reversibility rather than model confidence alone.
This article is for informational purposes only and does not recommend any security, product, investment, or securities action. Generative AI assisted the writing, and the content was checked against source code and official documentation. See the disclaimer for details.
