Databricks Certified Generative AI Engineer Associate: Certified Generative AI Engineer Associate

Incorrect. A context-augmented prompt cannot be constructed before retrieval because the “context” comes from retrieved documents/chunks. Also, vector search requires a query embedding, so placing the embedding model after vector search breaks the dependency order. This sequence reverses key prerequisites in the RAG pipeline.

Incorrect. Putting the response-generating LLM first implies generation happens before retrieval and prompt augmentation, which is not the typical RAG pattern. Additionally, embeddings are required before vector search, and prompt augmentation must occur before the LLM generates the final response. This ordering is fundamentally inconsistent with standard RAG.

Incorrect. This places the response-generating LLM first and vector search later, which contradicts the core idea of RAG (retrieve context before generating). It also places context-augmented prompt before vector search, but the prompt can only be augmented after retrieval. The dependencies are out of order.

Incorrect. This prompt ties the label “In Stock” to the customer asking for a product, not to whether the product is actually available. In availability inquiries, the customer will almost always ask about a product, so this would systematically mislabel out-of-stock cases. It also fails to define the decision rule based on inventory status.

Incorrect. While JSON formatting is often useful for application integration, it violates the stated constraint that the LLM is designed to output only the term “In Stock” or only the term “Out of Stock.” Adding JSON introduces extra tokens and structure, likely breaking downstream expectations and evaluation that rely on exact label matching.

Incorrect. Like option A, it incorrectly maps the label to the act of asking for a product rather than the product’s availability. This would misclassify in-stock items as out-of-stock. It also provides no conditional logic about availability, so it cannot support correct classification behavior.

Contacting the data curators before using the trained model is generally a more appropriate step than waiting until after use has begun. While simply notifying them is not the same as obtaining formal permission, early outreach can help clarify licensing terms, request authorization, and document due diligence before legal exposure occurs. This makes it a risk-reduction action rather than the clearly inappropriate choice. On exams, actions taken before use are usually safer than actions taken after use.

Using data that you personally created and that is completely original is an appropriate way to reduce licensing risk. If you truly own the content and it does not incorporate third-party protected material, you control the rights and can decide how it may be used or licensed. This is one of the safest sources of training data from a legal perspective. The main caveat is ensuring the material is genuinely original and not derived from copyrighted or restricted sources.

Using only data that is explicitly labeled with an open license and then following the license terms is a standard best practice for avoiding legal risk. Openly licensed data provides clear permission boundaries, and compliance with requirements such as attribution, redistribution limits, or commercial-use restrictions helps maintain lawful use. This approach supports governance, auditability, and defensible model development practices. It is therefore an appropriate action, not the one being asked for.

Safety guardrails are designed to prevent the model from generating harmful or unsafe content (e.g., hate, harassment, self-harm instructions, explicit sexual content, violence). While politics can be sensitive, the requirement here is not about preventing harm categories; it is about refusing an out-of-domain topic and keeping the assistant focused on insurance. That aligns better with contextual/topic guardrails than safety guardrails.

Security guardrails focus on protecting systems and data: preventing prompt injection, blocking requests for secrets/credentials, stopping data exfiltration, and enforcing access controls. The prompt does not mention sensitive data, system prompts, or unauthorized actions. The issue is topical scope control (no politics), which is not primarily a security concern, so this option is not the best fit.

Compliance guardrails address legal and regulatory requirements such as handling PII, retention, auditability, and industry regulations (e.g., HIPAA, GDPR). Although the question mentions “company policy,” the policy described is a scope restriction (no politics), not a regulatory compliance requirement. Therefore, a contextual guardrail is more appropriate than a compliance guardrail.

call_cust_history is designed for internal customer usage and chargeback consistency. Its primary keys and purpose indicate it tracks who used the HelpDesk and how often, not what the issue was or how it was resolved. It may help with reporting, prioritization, or cost allocation, but it provides little direct semantic or labeled information for root cause and resolution retrieval.

maintenance_schedule lists outages and planned maintenance windows. This can provide situational context (e.g., many tickets during an outage), but it typically does not contain per-ticket troubleshooting steps or resolutions. It is better used as a filter or enrichment feature (“was there an outage at that time?”) rather than a primary knowledge source for identifying root cause and resolution.

call_rep_history is maintained to calculate representative call resolution performance using call_duration and call_start_time. This is operational KPI data about agents, not the content of incidents. It lacks the descriptive problem statements and fix details needed for a chatbot to find related tickets and recommend resolutions, so it is not a strong RAG knowledge source.

Option B provides far more context than needed for a single 512-token chunk and therefore does not align with the stated priority of minimizing cost and latency. Its 11GB smallest model size is substantially larger than A, which implies higher serving cost, greater memory requirements, and slower inference. The much larger embedding dimension of 2560 also increases vector storage and retrieval computation. This option would only make sense if the application needed to include many chunks or much longer prompts.

Option C is massively oversized for the described use case, with a 32768-token context window that is unnecessary for 512-token chunks. The associated 14GB model and 4096-dimensional embeddings would significantly increase infrastructure cost, latency, and index size. Nothing in the prompt suggests a need for very long-context reasoning or many retrieved passages in one request. As a result, this option is the least aligned with the requirement to prioritize efficiency over quality.

Option D appears attractive because its context length matches the chunk size exactly and it has the smallest model and embedding dimension. However, a 512-token context window cannot practically fit a 512-token chunk plus the user question, system prompt, delimiters, and any other prompt formatting. That means it does not fully satisfy the functional requirement of running the application with retrieved context. The smallest option that actually leaves room for minimal overhead is A, so D is too tight despite its lower cost profile.

Incorrect ordering: it places “Evaluate model” before “LLM generates a response.” In RAG, evaluation typically measures the quality of generated answers (and their grounding in retrieved context), so you need the generation step before you can evaluate end-to-end performance. The rest of the flow (ingest → index → query → retrieve → generate → deploy) is mostly correct, but the evaluation placement makes this option wrong.

Incomplete: it only covers ingestion, indexing, evaluation, and deployment, but omits the core online RAG steps (user query, retrieval, and generation). A RAG application must include query-time retrieval from Vector Search and LLM response generation. You also cannot meaningfully evaluate a RAG QA system without running the retrieval+generation pipeline on test questions.

Incorrect sequence: it starts with user queries before ingestion and indexing. In a real RAG system, the knowledge base must be ingested, embedded, and indexed ahead of time so retrieval can work at inference. While documents can be updated incrementally, the baseline pipeline requires the vector index to exist before users can query and retrieve relevant context.

Energy usage per query can be relevant for sustainability reporting or cost optimization, but it is not the most essential metric for monitoring a customer service LLM in production. Operational health typically prioritizes user-facing performance (latency, throughput, error rates) and quality/safety outcomes. Energy is also harder to measure accurately per request in shared infrastructure.

Perplexity is an offline training/evaluation metric that measures how well a language model predicts tokens on a dataset. It does not reliably correlate with task success for customer service, especially in RAG or tool-using applications. Production monitoring should focus on runtime behavior and user outcomes rather than training-time perplexity scores.

HuggingFace leaderboard values are generic benchmark results for base models and are useful during model selection, not production monitoring. They do not capture your specific prompts, retrieval data, guardrails, or customer service domain constraints. Production issues (latency spikes, failures, hallucinations) won’t be detected by referencing static leaderboard scores.

This inverts the retrieval problem: it embeds project scopes and queries with employee profiles. The task is to find employees for a new project, so the natural index is employee profiles and the query is the project scope. Also, availability filtering is not integrated into the retrieval step here, and embedding “all project scopes” is unnecessary when projects are transient compared to a stable employee corpus.

Keyword extraction plus keyword matching is brittle for unstructured text. It misses semantic similarity (synonyms, related concepts, implicit requirements) and tends to underperform on real-world profiles and scopes. It also scales poorly if you iterate through many profiles. Modern Databricks GenAI design favors embeddings/vector search for semantic retrieval rather than LLM-extracted keywords and string matching.

A custom similarity tool that scores each employee profile against the project scope can work conceptually, but iterating through a very large team is inefficient and expensive. It ignores the main advantage of vector databases: approximate nearest neighbor search to avoid brute-force comparisons. This option lacks a vector index/retrieval step, making it a poor architecture for scale and latency.

Vector Search is used to index embeddings and retrieve semantically similar content for the retrieval step of a RAG workflow. It helps supply relevant context to the model, but it does not log serving endpoint requests or responses. There is no built-in request/response observability function in Vector Search that replaces a logging microservice. Therefore, it is not the correct feature for monitoring endpoint traffic.

Lakeview is Databricks' dashboarding and visualization capability for presenting data and metrics. It can be used to build dashboards on top of inference logs after those logs have been collected, but it does not itself capture model serving traffic. The question asks for the feature that performs the logging task, not the feature that visualizes the results. As a result, Lakeview is not the right answer.

DBSQL is the SQL analytics layer used to query and analyze data stored in Databricks. It is useful for examining inference logs once they have been written to tables, but it does not natively intercept or persist requests and responses from model serving endpoints. In other words, DBSQL is an analysis tool rather than a traffic-capture mechanism. That makes it unsuitable as a replacement for the described microservice.

A finance-specific guardrail framework can improve relevance by constraining outputs to acceptable financial-domain behavior and policies. For example, it can require the model to stay within finance-related topics, avoid unsupported claims, and enforce use of approved financial sources or response patterns. Although guardrails are often associated with safety and compliance, domain-tailored guardrails also help keep responses aligned to the intended subject matter. That makes this a helpful control rather than the best answer.

Increasing compute can support more sophisticated relevance-improving techniques, even if the option mentions speed. Additional compute may allow better retrieval pipelines, stronger reranking models, larger context windows, or more frequent refresh of the latest stock news corpus. While compute alone does not guarantee relevance, it can enable systems that perform deeper relevance analysis and process more context. Because it can indirectly help relevance, it is not the clearest choice for what will NOT help.

Manual review is a direct quality-control mechanism that can catch irrelevant, misleading, or non-financial outputs before they reach users. In a high-stakes domain like finance, human reviewers can verify that answers are grounded in the latest stock news and correct problematic responses. This clearly improves the relevance and reliability of outputs, even though it may reduce scalability. As a result, it is not the correct answer.

Incorrect. Frequently retraining or fine-tuning an LLM to incorporate new documents is expensive, slow, and operationally heavy. It also risks catastrophic forgetting and requires repeated evaluation. For rapidly changing knowledge, best practice is to keep the knowledge external and use retrieval at inference time (RAG) rather than continual model updates.

Incorrect. Prompt engineering plus an LLM alone cannot reliably answer questions about newly published documents if that information is not in the model’s training data. Without retrieval (or another external knowledge mechanism), the model will guess or hallucinate. This may be low effort initially but fails the “most up-to-date answers” requirement.

Incorrect for the stated constraints. Adding an agent and a fine-tuned LLM increases complexity, cost, and maintenance. Agents are valuable when the system must choose among tools, perform multi-step reasoning, or execute actions. For straightforward document Q&A with frequently updated content, a simple retriever + prompt + LLM is the lowest-cost, lowest-effort approach.

vsc.get_index() retrieves a handle/metadata for an index that already exists. It’s useful after creation (or when reusing an existing index) to perform operations like querying. It is not the next logical step immediately after creating a new endpoint when the goal is to set up a new vector store, because no index has been created yet.

`vsc.create_direct_access_index()` is not the best answer here because it represents a different ingestion model. With direct access, you create the index and then write records directly into it, which is more appropriate when your application manages ingestion explicitly rather than syncing from a Delta table. The question asks for the next logical call when setting up a managed vector store, and the standard Databricks workflow for that is a Delta Sync index. While direct access can support managed embeddings in some configurations, it is not the most natural choice implied by this prompt.

vsc.similarity_search() is a query-time operation used after an index exists and has data/embeddings available. Immediately after creating only the endpoint, there is nothing to search yet. Similarity search is part of application development/inference, not the setup step for creating the managed vector store.

Grouping chat logs by users and summarizing interactions is primarily an analytics or support-handoff task. It can help staff understand a customer’s history, but it does not directly drive an automated booking experience in the moment. Summaries don’t capture required structured booking fields (date/time/party size) and won’t inherently reduce escalations or phone calls as effectively as guided self-serve flows.

Customer reviews as input with sentiment classification as output is valuable for measuring satisfaction and improving service, but it does not address the operational goal of handling booking inquiries automatically. It’s not a conversational workflow and won’t help capture booking parameters or complete reservations, so it won’t meaningfully reduce phone calls or human intervention for bookings.

Online chat logs with cancellation options as output addresses only one narrow scenario (cancellations). The problem statement is broader: “common customer inquiries” and improving booking experience end-to-end. A cancellation-only output doesn’t support booking creation/modification, answering FAQs, or collecting required details, so it won’t minimize escalations across typical booking interactions.

Incorrect. Directly modifying an LLM’s internal architecture is not a practical or supported approach for most enterprise use cases, especially when using hosted foundation models (e.g., via Databricks Model Serving or external APIs). Preprocessing is typically done outside the model through prompt templates, retrieval pipelines, or application code, not by changing the LLM weights/architecture.

Incorrect. Custom preprocessing is common and often necessary (templating, safety instructions, context injection, normalization). LLMs are designed to be steered via prompts; they do not require training examples of your exact preprocessed format to work. The key is to keep preprocessing consistent, deterministic where possible, and evaluated for quality and safety.

Incorrect. Postprocessing can help (format validation, JSON parsing, guardrails), but it does not replace preprocessing. Many outcomes depend on how the prompt is constructed (system instructions, retrieved context, constraints). Effective GenAI systems typically use both preprocessing (to guide generation) and postprocessing (to validate/shape outputs).

Incorrect. Reducing interaction time is a rate-limiting or UX constraint, not a content-safety control. Malicious prompts can be issued immediately, so time limits do not reliably prevent prompt injection, toxic content, or attempts to exfiltrate sensitive information. Rate limiting can be a supporting control, but it is not the most effective safeguard for harmful inputs.

Incorrect. Telling the user the input is malicious but continuing the conversation is risky because it still allows the model to generate harmful or policy-violating content. Attackers can iterate and refine prompts, and the model may comply despite warnings. Best practice is to refuse and stop or redirect to safe alternatives, not to continue engaging with malicious requests.

Incorrect. Increasing compute improves latency/throughput but does not improve safety. Faster processing can even amplify harm by enabling higher-volume abuse. Safety requires governance and guardrails (moderation, policy checks, least-privilege tool access, monitoring), not more hardware resources.

The ability to generate responses in code is not a safety indicator for a translation use case. Code generation is a capability metric relevant to developer or programming assistants, not to whether a translation is faithful, non-harmful, or policy-compliant. A model could generate code well and still produce unsafe or inaccurate translations.

Similarity to the previous language is vague and can be misleading. Translation quality is about semantic equivalence and intent preservation, not superficial similarity to the source language. A safe translation can differ significantly in wording or structure while remaining accurate. This option does not directly capture hallucinations, harmful additions, or meaning drift.

Latency and output length are important for user experience and cost/performance monitoring, but they do not measure safety. A fast response can be unsafe, and a slow response can be safe. Similarly, longer outputs might correlate with verbosity or hallucination risk, but length alone is not a reliable qualitative safety indicator for translation.

Incorrect. Suggesting articles is appropriate only when the question is clearly not urgent and after the chatbot has gathered enough context to provide relevant, pre-approved resources. With severe headache and dizziness, routing to reading materials risks delaying care and violates the prompt’s requirement to escalate urgent cases to emergency services.

Incorrect. While empathetic language can be part of a good user experience, it does not meet the functional requirement for urgent cases. It provides no actionable safety guidance and could be interpreted as reassurance, which is risky in healthcare contexts and contrary to best practices for high-stakes domains.

Incorrect. Asking for more details is the correct behavior only for non-emergency situations where the chatbot is collecting information to pass to a doctor’s office. In a potentially urgent scenario, follow-up questions can delay escalation. The safer design is to route immediately to emergency services guidance.

Incorrect. Using a smaller embedding model affects how documents and queries are represented in vector space, such as embedding dimensionality, retrieval quality, latency, or cost. It does not reduce the number of text tokens included in the prompt sent to the response-generating LLM. The context window error is about prompt length, not embedding size or embedding model complexity. Therefore, changing the embedding model would not directly solve the token overflow problem.

Incorrect. Reducing the maximum output tokens can be useful when a serving system enforces a combined input-plus-output budget, but the error here explicitly states that the prompt token count alone is 4595 and exceeds the 4096-token limit. That means the request is already invalid before any output generation is considered. Lowering output tokens does not reduce the size of the input prompt itself. For this specific error wording, the engineer must reduce prompt content rather than generation length.

Incorrect. Retraining the response-generating model using ALiBi would constitute changing the model, which the question explicitly forbids. It is also a major model-development effort rather than a practical pipeline adjustment. Even if a positional encoding strategy could help with longer contexts in some architectures, it does not address the immediate operational requirement to keep using the current model unchanged. The question asks for implementation changes in the RAG pipeline, not model retraining.