Quick answer
How do you evaluate an AI consulting proposal? Score the proposal against eight questions: architecture fit, evaluation framework, named builders, failure mode coverage, maintenance plan, weekly deliverables, contract protections (IP, exit rights, performance guarantees), and price decomposition. A proposal that scores 12 or above out of 16 across those eight dimensions is worth proceeding with. Below 8 is a signal to go back to the vendor before committing budget.
Section 01 · Context
Why most AI consulting proposals look the same
The AI consulting market tripled between 2023 and 2025. Supply of experienced practitioners did not keep pace. The gap was filled by vendors who learned to write compelling proposals without having shipped many production systems.
Most AI consulting proposals are structurally identical. They open with a market positioning statement, describe a phased methodology, name a few technologies, list case studies, and close with a price and a timeline. The format is designed to look thorough — not to be evaluated rigorously. Most proposals survive a casual read. Very few survive eight pointed questions.
These are those eight questions. They come from having written proposals, evaluated proposals for clients, and been on the receiving end of AI systems built by vendors who could not answer them.
Before you schedule the follow up call, score the proposal you have received against each one.
Section 02 · Question 1
Does the proposed architecture match your problem?
Read the technical section and ask one thing: is this architecture designed for your specific situation, or could it have been copy pasted into any AI project?
A generic architecture description looks like this: "We will use a large language model with a vector database for retrieval and an API layer for integration with your existing systems." That sentence applies to roughly 80 percent of enterprise AI projects. It tells you nothing about whether the vendor has thought about your data shape, your query volume, your latency requirements, or your existing infrastructure.
A specific architecture description names trade offs. It explains why a vector retrieval approach was chosen over traditional BM25 search for your use case — or argues the opposite and explains why. It addresses your specific data format: structured tables, unstructured PDFs, real time event streams, or something else. It addresses latency: is sub-200ms required, and if so, how is that handled? It mentions the failure case: what happens when retrieval returns no relevant results?
What to look for
An architecture diagram tailored to your system, not a generic AI stack diagram. Specific references to your data sources and query patterns. At least one acknowledged trade off where an alternative was considered and rejected.
Red flag
"We use state of the art LLMs and vector databases to power intelligent retrieval." This is a sentence that means nothing specific. Ask the vendor to replace it with specifics before you proceed.
Section 03 · Question 2
Is there an evaluation framework?
Every AI system performs well in a vendor demo. The question is whether the proposal defines what performing well means in measurable terms before go live.
An evaluation framework answers three questions: What are the specific acceptance criteria? How will those criteria be measured? What is the threshold below which the system should not go to production?
For a document extraction system, that might look like: 94 percent field extraction accuracy on a 300-document hold out test set, measured against human annotations, before production deployment. For a customer service agent, it might look like: task completion rate above 78 percent on a representative sample of 200 real user queries, with a human review mechanism for queries below 60 percent confidence.
The numbers matter less than the methodology. A proposal that includes no measurement methodology before launch is not describing a production system. It is describing a demo with a go live date attached.
What to look for
A section titled 'Definition of done,' 'Acceptance criteria,' or 'Evaluation methodology' with specific, measurable thresholds. The methodology for testing those thresholds before launch. The explicit condition under which the system should not go to production.
Red flag
'We will work with your team to validate the system before launch.' This is a process description with no measurement commitment. Ask the vendor what the acceptance criteria are and what happens if the system does not meet them.
Section 04 · Question 3
Who builds it, specifically?
This is the question most buyers skip because it feels uncomfortable to ask. It is also the question that predicts delivery quality more reliably than any other.
Most AI consulting firms sell on the strength of their senior people and deliver on the strength of their junior people. The senior ML researcher in the sales call becomes an oversight resource while two contractors who joined the firm last quarter do the actual build. The guide to hiring an agentic AI consultant covers a fuller framework for evaluating team depth. A proposal that refers to "our team of AI engineers" without naming individuals has not committed to staffing the project with specific people.
Ask for names and profiles
Request the names and LinkedIn profiles of the specific engineers who will work on your project. Ask each of them — directly, in a call — about a production AI system they built, a specific failure they caught in production, and how they resolved it.
Verify the sales-to-delivery match
Ask whether the people in the sales call will be involved in the build. Ask what percentage of their time will be allocated to your project specifically.
Require a staffing clause
Ask what happens to your project if a named engineer leaves the firm mid engagement. The answer should be in the contract, not just a verbal assurance.
Section 05 · Question 4
What are the failure modes?
An AI system that works on clean test data does not automatically work on production data. This seems obvious. Most proposals do not address it.
Failure modes in AI systems are not hypothetical. They are predictable. A retrieval system fails when the knowledge base contains documents with distribution properties different from the test set. A customer service agent fails when users ask questions outside the intended scope. A data extraction pipeline fails when document formats change. A reasoning agent fails when given ambiguous inputs and no step limit on its reasoning loop.
The three failure mode categories every AI consulting proposal should address:
Model output failures
What happens when the model is wrong, low confidence, or contradicts a previous output? What confidence threshold triggers a human review pathway? What fallback content is shown when confidence is below threshold?
Infrastructure failures
What happens when the underlying API goes down, latency spikes above acceptable thresholds, or a tool call times out? Is there a cached fallback, a graceful degradation mode, or a circuit breaker?
Input failures
What happens when user input is outside the system's designed scope — adversarial, ambiguous, or simply unexpected? A proposal that only describes the happy path is not planning for production.
Section 06 · Question 5
What does the maintenance plan look like?
An AI system that is handed over without a maintenance plan is a system that will degrade silently. Model behavior changes when API providers update the underlying model. User query distributions shift over time. Prompts become brittle.
None of this is exotic. It is predictable decay. A vendor who does not address it in the proposal is either unaware of it — which is a technical concern — or is planning to sell you a separate maintenance contract after the dependency is established, which is a commercial concern.
What a strong maintenance section looks like: monitoring responsibilities named by role, performance review checkpoints at 30, 60, and 90 days post launch, defined triggers for prompt revision or architecture changes, and a clear answer to the question of what happens when the underlying model is deprecated by the API provider.
The question most buyers forget to ask
Ask the vendor to describe one case where a system they built degraded after launch and what they did about it. Practitioners who have shipped production AI systems have this story. Vendors who have not cannot tell it convincingly.
Section 07 · Question 6
How is progress measured week to week?
An AI project with a three month timeline and no intermediate accountability structure is a three month window in which things can go wrong without you knowing.
Delivery risk in AI consulting concentrates at two points: the transition from discovery to build (scope shifts), and the transition from build to production (performance gaps surface). Both are visible in advance if you have weekly deliverables and a defined review cadence. They are invisible if progress is reported narratively on a monthly basis.
A weekly deliverable is not a status update. It is a specific, reviewable artifact: a completed architecture document, a working retrieval prototype on a 50-document subset, a completed evaluation dataset with 100 annotated cases, a performance report against acceptance criteria on the staging environment. Something you can open, read, and evaluate.
Payment gates vs. deliverable gates
A milestone structure that only shows payment gates — where the vendor shows you a 'completed phase' every four to six weeks — is not the same as weekly deliverable gates. Payment gates mean you pay when told the phase is done. Deliverable gates mean you evaluate a specific artifact before payment releases.
Section 08 · Question 7
What are the contract protections?
Three contract clauses determine whether you are protected if the engagement goes wrong. Most proposals omit at least one of them.
IP ownership
Who owns the code, the models, the fine tuned weights, and the prompt templates after the engagement ends? The contract should explicitly state that all code, prompts, evaluation datasets, and trained artifacts produced during the engagement are owned by the client. The vendor retains no license to use them in other engagements.
Exit rights
What do you receive if you terminate the engagement early? The answer should be: all work product produced to date in a specified format, a handover document covering architecture decisions and known issues, and access to all relevant infrastructure in a transferable state.
Performance guarantees
If the system does not meet the acceptance criteria defined in the proposal, what happens? Options are: rework at the vendor's cost until criteria are met, a partial refund, or a credit against future work. Ask the vendor explicitly what happens if the system does not pass the acceptance test on the first attempt.
The nine red flags to check before you sign any AI consulting proposal covers the warning signs that appear in proposals and discovery calls before you reach the contract stage. These three clauses are the minimum floor once you have decided to proceed.
Section 09 · Question 8
Does the price reflect the actual scope?
A price is not a number. It is a claim about how much work the vendor believes the scope requires. If you cannot decompose that claim, you cannot evaluate it.
The decomposition is straightforward for an AI consulting project: total engineering hours by role, multiplied by blended rate, plus model API costs at scale, plus infrastructure costs. For a three month engagement at $150,000 to $200,000, that decomposes to roughly 500 to 700 engineering hours at $250 to $300 per hour, plus $5,000 to $15,000 in model API and infrastructure costs depending on usage volume. If the numbers do not add up, ask why.
Model API costs at scale deserve a separate conversation. A system processing 10,000 queries per month at an average of 2,000 tokens per query costs roughly $400 to $600 per month in API fees at current GPT-4o pricing. A system processing 500,000 queries per month costs $20,000 to $30,000 per month. Whether that is in scope or out of scope for the engagement, it should be in the proposal.
What to look for in the price section
A staffing model that decomposes hours by role and rate. An API cost estimate at your expected usage volume. A clear statement of what is and is not included in the price, and the assumptions that the estimate depends on. Ask the vendor to show you the staffing model behind the number.
Section 10 · Rubric
The scoring rubric
Score each of the eight questions 0, 1, or 2. A total of 12 or above is a strong proposal. A total below 8 is a signal to re engage the vendor before proceeding.
| Question | Good (2 pts) | Weak (1 pt) | Red flag (0 pts) |
|---|---|---|---|
| Architecture fit | Tailored to your data shape, latency, and infrastructure with specific trade offs | General description with some specifics | Generic 'LLMs + vector database' with no trade offs |
| Evaluation framework | Measurable acceptance criteria and test methodology before go live | Mentions validation but no specific thresholds | No acceptance criteria in the proposal |
| Named builders | Specific engineers named with production references | Team structure described but no individual names | 'Our team of experts' with no specifics |
| Failure modes | Error handling, confidence thresholds, and fallback behavior documented | Mentions some failure cases abstractly | Only the happy path described |
| Maintenance plan | Monitoring cadence, 30/60/90 day checkpoints, and model update plan | Some mention of post launch support | Proposal ends at handover |
| Weekly deliverables | Specific reviewable artifacts per phase and weekly review cadence | Phase milestones but no intermediate deliverables | Payment gates only, no deliverable structure |
| IP ownership clause | Client owns all code, prompts, datasets, and trained artifacts | Shared ownership or license arrangement | Not addressed in the proposal |
| Price decomposition | Staffing model with hours, roles, rates, and API costs at volume | Total price with phase breakdown but no staffing model | Single number with no decomposition |
Section 11 · FAQ
Frequently asked questions
The questions buyers ask most when they have an AI consulting proposal in front of them and need to decide whether to proceed.
How do you evaluate an AI consulting proposal?
Score the proposal against eight questions: architecture fit, evaluation framework, named builders, failure mode coverage, maintenance plan, weekly deliverables, contract protections (IP, exit rights, performance guarantees), and price decomposition. A proposal that scores 12 or above out of 16 across those eight dimensions is worth proceeding with. Below 8 is a signal to go back to the vendor before committing budget.
What should an AI consulting proposal include?
A strong proposal includes a phase breakdown with specific deliverables and acceptance criteria per phase, architecture decisions explained with the trade offs relevant to your situation, an evaluation methodology with measurable success thresholds before launch, named engineers with production references, a post launch maintenance and monitoring plan, and a price that decomposes into staffing hours and API costs at scale.
What contract clauses should I require in an AI consulting engagement?
Require three specific clauses: IP ownership (all code, prompts, datasets, and trained artifacts belong to you, not licensed to you), exit rights (the deliverables you receive if you terminate early, in a transferable format), and a performance guarantee (what happens if the system does not meet the acceptance criteria — rework at vendor cost is the minimum acceptable answer).
How do I sanity check the price of an AI consulting proposal?
Ask the vendor for the staffing model behind the price: hours by role, blended rate, and assumptions. For a three month engagement at $150,000 to $200,000, a reasonable decomposition is 500 to 700 engineering hours at $250 to $350 per hour plus infrastructure and API costs. Also ask for an API cost estimate at your expected query volume — a high volume production deployment can add $10,000 to $30,000 per month in ongoing API fees that should be scoped separately.
How is evaluating an AI consulting proposal different from evaluating a software development proposal?
The core difference is the role of evaluation and failure modes. In software development, acceptance criteria are usually functional and deterministic: the feature works or it does not. In AI consulting, acceptance criteria must be statistical (the model achieves X accuracy on Y test set) and must cover failure modes explicitly (what happens when the model is wrong). An AI consulting proposal that skips acceptance criteria is proposing a system with no defined production bar.
What is the most common thing missing from AI consulting proposals?
The evaluation framework — the definition of done with measurable thresholds before go live. Most proposals describe what will be built, but very few define the specific acceptance criteria that must be met before the system goes to production. This omission means the vendor controls the definition of done, and disputes about delivery quality become subjective rather than measurable.
Most AI consulting proposals look competent until you ask pointed questions. The eight questions in this framework are designed to surface the gaps that generic proposals hide: no named engineers, no acceptance criteria, no failure mode coverage, no maintenance plan, and no defensible price decomposition.
A proposal that scores cleanly on seven of eight is genuinely strong. A proposal that fails four of eight is telling you something worth hearing before you sign.
If you want a technical perspective on an AI consulting proposal you have received — or want to understand what a well structured engagement should look like for your specific use case — the agentic AI consulting service is where that conversation starts. The first call is a scoping conversation, not a sales pitch.