📚 首页 / mock-exams / mock-04.md

Mock Exam 04 — Mixed Closing Exam · Tools / MCP / API / Evals(英文题干 · 卷四)

原创声明:本卷为课程原创教学模拟题,不是、也不冒充官方真题;未使用任何 brain dump 或泄露材料。 定位:CCDV-F / CCAR-F 混合视角收口卷,加重 tools / MCP / API mechanics / evals。题型为 MC + MR(多选标注数量)+ 4 道 scenario,每题标注目标认证域(官方域名原文)。注意:本卷的分桶配比为课程设计(收口练习用途),不是官方 CCDV-F 权重(CCDV-F 官方八域权重见 ../14-certification-prep/exam-domain-map.md,VERIFIED 截至 2026-08-30)。 参考形式(官方,VERIFIED 截至 2026-08-30):CCDV-F 53 题/120 分钟/720 及格;CCAR-F 60 题/120 分钟/720 及格。本卷 40 题为聚焦练习。

Part I · Tool Use Loop & Design(Q1–Q8)

Q1. What is the correct order of one complete tool-use round trip? (CCDV-F · Tools and MCPs) A. User asks → model executes the tool itself → model answers B. User asks → app executes tool → model sees nothing → app answers C. User asks → model returns a tool_use block → your app executes the tool → your app sends back a tool_result → model produces the final answer D. User asks → model answers → app runs a tool afterwards for logging

Q2. You want the model to be free to either answer directly or call a tool, whichever it judges appropriate. The tool_choice setting is: (CCDV-F · Tools and MCPs) A. auto B. any (forced tool use) C. tool (a specific named tool) D. none

Q3. Two tools on your agent — search_tickets and find_tickets — do nearly the same thing with different names. The likely effect and fix: (CCDV-F · Tools and MCPs) A. No effect; redundancy is free B. Better accuracy; more choice helps C. No effect as long as descriptions differ D. Degraded selection accuracy from ambiguity — consolidate into one tool or clearly differentiate their scopes

Q4. A good tool description states: (CCDV-F · Tools and MCPs) A. Only the tool's function name B. What it does, when to use it, when NOT to use it, parameter meanings, and constraints C. Only the parameter types D. Nothing — the model figures tools out

Q5. For write operations that might be retried after a timeout, the safe design is: (CCDV-F · Tools and MCPs) A. Idempotent operations (e.g., client-supplied idempotency keys) so a retry cannot double-execute B. Hope the retry lands on a different server C. Retry blindly; duplicates are the database's problem D. Disable retries for all writes, even legitimate ones

Q6. An agent drafts a refund approval; the money movement itself should be: (CCAR-F · Tool Design & MCP Integration) A. Executed by the model directly for speed B. Executed by the model after it double-checks its own work C. Validated and executed by deterministic application code (policy checks, authorization), with the model's output treated as input, not authorization D. Skipped; refunds are impossible

Q7.(Select TWO) Which TWO belong in a tool-design checklist? (CCDV-F · Tools and MCPs) A. Keep tools focused — one tool, one purpose B. Return errors to the model as structured, actionable results C. Merge everything into one mega-tool for simplicity D. Hide failures from the model so it stays confident

Q8.【Scenario: The over-privileged admin assistant】 (CCAR-F · Tool Design & MCP Integration) Scenario: An internal assistant manages calendars: read events, create events, delete events, and send emails on behalf of executives. A red-team exercise shows the model can be talked into deleting a board meeting via an injected email and exfiltrating the attendee list. Constraints: executives want automation for routine scheduling; destructive actions and external sends must never happen unattended; audit trail required. Options: A. Remove all tools; assistants type manually B. Split tool privileges: read-only calendar tools run freely; destructive tools (delete) and external send require an explicit human approval gate in application code, with scoped credentials (no bulk export) and full audit logging C. Keep the tools but add "be careful" to the system prompt D. Give the assistant full admin rights so it can fix its own mistakes


Part II · MCP(Q9–Q16)

Q9. In MCP, the component that maintains a 1:1 connection to a single server (inside the host) is the: (CCDV-F · Tools and MCPs) A. Server B. Transport C. Client D. Registry

Q10. You want to expose your database's schema documentation and reference tables to applications without letting the model invoke them as actions. The right MCP primitive: (CCDV-F · Tools and MCPs) A. Tools — everything is a tool B. Prompts C. Sampling D. Resources — application-driven read-only context

Q11. An agent called your MCP tool with a malformed ID, received a structured error result saying "ID must match pattern TKT-12345; you sent TKT12345", and its next call was correct. What design enabled this recovery? (CCDV-F · Tools and MCPs) A. The tool silently corrected the ID B. The server returned a business error as a tool result (isError: true) with an actionable message — the model self-corrected C. The session crashed and restarted D. A human fixed it mid-flight

Q12. A team hosts an MCP server on their own machine for local development (no remote clients). The natural transport is: (CCDV-F · Tools and MCPs) A. stdio — local process communication, no network exposure B. Streamable HTTP on a public IP C. WebSocket over the WAN D. Email

Q13. Why is token passthrough (forwarding the user's token verbatim to downstream services) forbidden in MCP? (CCDV-F · Tools and MCPs) A. It is slower than generating new tokens B. Tokens expire too quickly C. It bypasses audience control and breaks the audit chain — downstream services can't tell what they're authorizing for whom D. It only works with HTTP transports

Q14. You want an MCP server available to all your personal projects, not just one repo. The right scope for its config is: (CCDV-F · Tools and MCPs) A. Project scope (.mcp.json in one repo) B. Session-only ephemeral scope C. A sticky note D. User scope (your personal config, available across your projects)

Q15.(Select TWO) Which TWO statements about MCP primitives are correct? (CCDV-F · Tools and MCPs) A. Tools are invoked under the model's control B. Resources can mutate system state C. Prompts are user-controlled templates (e.g., slash commands) D. All three primitives are automatically loaded into every request

Q16.【Scenario: Vetting a third-party MCP server】 (CCAR-F · Tool Design & MCP Integration) Scenario: Marketing wants to connect your production agent to a third-party MCP marketplace server that posts to social media and reads a CRM. The vendor is reputable; the tool list is long and changes weekly. Constraints: production data (CRM) is in scope; unattended posts are unacceptable; the vendor's server code is not auditable by you. Options: A. Trust the reputable vendor; connect with full credentials immediately B. Connect read-only first with narrowly scoped credentials (a dedicated integration identity), review the actual tool surface and its changes, gate any write/post action behind human approval, and monitor egress — trust is verified, not assumed C. Connect with the CEO's personal tokens for maximum permissions D. Reject all third-party servers forever, build everything in-house


Part III · Claude API Mechanics(Q17–Q27)

Q17. What does streaming primarily improve? (CCDV-F · Applications and Integration) A. Total generation time — it makes the model faster B. Perceived latency — time-to-first-token drops and users watch progress; total time is roughly unchanged C. Output quality D. Price per token

Q18. Your request fails with HTTP 400 invalid_request_error. The correct response: (CCDV-F · Applications and Integration) A. Retry immediately with exponential backoff B. Retry with double concurrency C. Retry until it succeeds D. Fix the request itself (bad parameter/schema) — a 400 will fail identically on every retry

Q19. Calls fail with HTTP 401/403 authentication errors. The correct response: (CCDV-F · Applications and Integration) A. Check/rotate credentials and headers — retrying with the same bad auth cannot succeed B. Retry harder C. Switch models D. Increase max_tokens

Q20. You receive HTTP 429 and the response includes a retry-after header. You should: (CCDV-F · Applications and Integration) A. Fire five parallel retries to win the race B. Retry immediately at the same rate C. Honor the header: wait as instructed, then retry with exponential backoff and jitter D. Drop the request permanently

Q21. You receive HTTP 429 with no retry-after, and the error details indicate an organization spend limit. You should: (CCDV-F · Applications and Integration) A. Wait 60 seconds and retry forever — it will clear B. Read the error code, then address the actual cause: budget/cap configuration or quota increase — retrying cannot clear a spend-limit 429 C. Switch to a different model to dodge the limit D. Disable error handling

Q22. Sustained HTTP 529 (overloaded) responses indicate: (CCDV-F · Applications and Integration) A. Your request is malformed B. Your authentication expired C. You exceeded your personal rate limit D. Platform-side overload — back off exponentially, shed load (circuit breaker), and retry later; hammering makes it worse

Q23. To meter per-request token consumption, you read: (CCDV-F · Applications and Integration) A. The usage fields in the API response (input/output/cache tokens) B. Your intuition C. The HTTP status code D. The model's display name

Q24. Traffic is growing and you hit rate limits during daily peaks. A sound response combines: (CCDV-F · Applications and Integration) A. Buying more servers for your own cluster B. Ignoring peaks; users understand C. Smoothing load (caching, batching non-interactive work off-peak, backoff on 429) and, if sustained, requesting a limit increase with data D. Disabling retries so errors surface faster

Q25.(Select TWO) Which TWO errors are typically retryable with backoff? (CCDV-F · Applications and Integration) A. HTTP 400 invalid_request_error B. HTTP 429 with a retry-after header C. HTTP 401 authentication error D. HTTP 529 overloaded_error

Q26.(Select TWO) Which TWO practices legitimately reduce perceived or actual latency? (CCDV-F · Applications and Integration) A. Streaming responses (earlier first token) B. Prompt caching for stable prefixes (less reprocessing) C. Raising temperature D. Sending the entire knowledge base with every request

Q27.【Scenario: The launch-day 429 storm】 (CCDV-F · Applications and Integration) Scenario: A product launch drives 10× traffic. Current design: one mid-tier model, full 80k-token knowledge context per request, no caching, aggressive client retries. You see a 429 storm and rising p95. Constraints: budget is 2× (not 10×); p95 SLO 12 s; a classifier already distinguishes simple vs complex questions; nightly jobs exist. Options: A. Switch everything to the flagship model and demand 10× quota B. Combination: prompt caching over the stable knowledge prefix; route simple questions to a fast tier and complex ones to the strong tier; exponential backoff with jitter plus a circuit breaker on 429/529; move non-interactive jobs to the Message Batches API overnight C. Rate-limit users to one question per day D. Disable the knowledge features during peak hours; ship a bare chatbot


Part IV · Evals & Observability(Q28–Q36)

Q28. A golden dataset should be: (CCDV-F · Eval, Testing, and Debugging) A. Arbitrary prompts typed by the dev team last night B. Representative of real traffic, human-labeled (or carefully verified), versioned, and split so it isn't also the tuning set C. Every log you have, unlabeled D. Demo cases that always pass

Q29. The purpose of a regression gate in CI is to: (CCDV-F · Eval, Testing, and Debugging) A. Block merges/releases when prompt, tool, or model changes measurably degrade the golden-set score B. Slow developers down C. Replace code review D. Generate dashboards nobody reads

Q30. A sound LLM-as-judge setup includes: (CCDV-F · Eval, Testing, and Debugging) A. Any cheap model with a one-line prompt B. A strong model with no calibration, trusted blindly C. Binary yes/no verdicts only D. A concrete rubric, an "Unknown/insufficient information" escape hatch, and periodic calibration against human-labeled samples

Q31. Retrieval was "improved" by raising K from 5 to 50; recall@k rose, but end answers got worse. The most likely explanation: (CCDV-F · Eval, Testing, and Debugging) A. Recall is a broken metric and should be banned B. The vector database rebooted C. Precision collapsed — 50 chunks flood the context with noise (context rot); read recall and precision (or nDCG) as a pair D. Users became pickier

Q32. The most reliable signal that a support agent actually resolved a ticket: (CCDV-F · Eval, Testing, and Debugging) A. The agent's final message says "resolved" B. The user stopped replying C. Check the environment's end state (ticket status actually changed in the system of record) D. The conversation was long

Q33. Your eval set has scored 96% for three straight months. The right move: (CCDV-F · Eval, Testing, and Debugging) A. Congratulate the team; the set is perfect B. Suspect eval saturation: graduate old items to a regression suite, add harder/newer items, and keep a held-out set for honest reporting C. Delete the eval set D. Lower the pass bar

Q34.(Select THREE) A minimal-but-honest metric set for a RAG system includes: (CCDV-F · Eval, Testing, and Debugging) A. recall@k (retrieval coverage) B. precision@k or nDCG (retrieval noise/ranking) C. groundedness of generated answers (claims traceable to sources) D. Thumbs-up counts as the sole quality metric

Q35.(Select TWO) Which TWO belong in production observability for a Claude app? (CCDV-F · Eval, Testing, and Debugging) A. Per-request traces with latency, token usage, cost, and tool-call outcomes (PII-safe) B. Error rates segmented by error type (429/529/4xx/tool failures) C. Aggregate QPS alone D. Storing every raw prompt unredacted forever

Q36.【Scenario: The upgrade that wasn't】 (CCDV-F · Eval, Testing, and Debugging) Scenario: A new model snapshot scores 118/150 on your golden set vs 121/150 for the pinned production snapshot. The diff: 11 newly-passing hard items, 14 newly-failing items clustered in tool-call formatting. Constraints: quality regressions are unacceptable; release window is 10 days; you have a 1% canary lane. Options: A. Ship the new snapshot — newer is better, and 118≈121 B. Analyze the 14 failures; if they trace to fixable tool-schema/prompt drift, fix and re-eval to ≥ parity plus the 11 wins, then canary 1%→25%→100% with rollback ready; if unfixable, defer and keep the pin C. Declare failure and ban upgrades for a year D. Flip a coin; the averages are basically equal


Part V · Agents, Workflows & Model Choice(Q37–Q40)

Q37. A nightly job always runs the same five steps (fetch → clean → classify → enrich → load) with an LLM only at "classify". The right architecture: (CCDV-F · Agents and Workflows) A. An autonomous agent that decides each night what to do B. A multi-agent system with five subagents C. A human doing it manually D. A deterministic workflow (orchestrated in code) with the LLM called only at the classify step

Q38. Which of these is NOT a valid stopping condition for an agent loop? (CCDV-F · Agents and Workflows) A. A maximum iteration count enforced by code B. A token/spend budget cap enforced by code C. A wall-clock timeout enforced by code D. The agent reporting that it "feels done"

Q39. You must classify 5 million short support messages per day into fixed categories. Latency budget per message is tight; quality bar is moderate. The sound default: (CCDV-F · Model Selection and Optimization) A. The flagship model for every message B. A fast, inexpensive model tier validated by per-task evals, with sampling-based quality monitoring and escalation for low-confidence cases C. The flagship model twice per message for confidence D. A random model per message for diversity

Q40. Overnight, an alias auto-update changed model behavior and broke two pipelines. The practice that prevents this class of incident: (CCDV-F · Model Selection and Optimization) A. Pinning exact model snapshots; upgrades only through eval gates and staged rollout B. Trusting aliases; they never change C. Disabling monitoring so nobody notices breakage D. Retrying failed calls until the alias changes back


(Mock 04 完) · 对答案:mock-04-answers.md · 回看:mock-03.md · 索引:README.md

来源:CCDV-F/CCAR-F 形式与域名为官方 Exam Guide(VERIFIED,截至 2026-08-30,见 ../00-certification-current-status/certification-status.md);本卷分桶配比为课程设计(收口练习),非官方权重。题目技术内容 = 本课程模块 M01–M13 原创。
📚 目录