Practice Test #1

Skipping or commenting out components can reduce runtime, but it is a manual, error-prone workflow that changes the pipeline graph and may bypass important validations. It also doesn’t leverage Vertex AI Pipelines’ built-in orchestration best practices. In team settings, this harms reproducibility and makes it harder to compare runs because different runs execute different subsets of steps.

Dataflow can be excellent for large-scale ETL, but migrating feature processing to Dataflow is a redesign that adds operational overhead and may increase costs (Dataflow job charges, potential always-on resources, and additional integration). It also doesn’t inherently solve iteration speed if the pipeline still retriggers expensive processing; caching would still be needed.

Adding a T4 GPU increases cost and often provides little to no benefit for scikit-learn training, which is typically CPU-bound and not GPU-accelerated. Even if training sped up, the overall 80-minute runtime likely includes significant data ingestion/cleaning time, so a GPU would not address the main bottleneck and violates the “without increasing costs” requirement.

Natural Language API provides pretrained NLP capabilities (sentiment, entity extraction, and a limited content classification taxonomy). It is fast to integrate but offers minimal control over model architecture, training, and deployment. It also may not support custom 8-category warranty triage labels. This conflicts with the requirement to use TensorFlow and maintain full control over code, serving, and deployment, making it unsuitable here.

AutoML Natural Language can train a custom text classifier quickly from labeled data and is often a strong choice for rapid delivery. However, it is a managed training and serving solution where you do not maintain full control over the TensorFlow model code and deployment mechanics. While it can be orchestrated in pipelines, it violates the explicit requirement for full TensorFlow control, so it is not the best answer.

Using an established text classification model “as-is” is unlikely to work because the target labels are specific (8 warranty triage categories) and won’t match the pretrained model’s original label set or taxonomy. Even if the model outputs generic categories, it won’t map cleanly to your business classes without adaptation. The requirement emphasizes leveraging existing resources, but still implies customization; transfer learning is needed.

Dataproc/Spark can preprocess large datasets, but exporting transformed Parquet to Cloud Storage introduces extra data movement, storage management, and pipeline operations. It also risks training/serving skew because weekly inference is required to run directly in BigQuery; you would need to re-implement the same feature logic in BigQuery or continuously export new partitions. This is typically less cost-effective and less consistent than doing feature engineering in BigQuery.

Loading 2.6 TB (900M rows) into a local pandas DataFrame is infeasible due to memory and compute constraints and would be extremely slow and costly. It also breaks scalability and operational best practices for production ML. This option ignores distributed processing and BigQuery’s strengths, and it would not support ongoing weekly inference in BigQuery without duplicating preprocessing logic elsewhere.

Dataflow/Beam is strong for streaming and ETL, but writing preprocessed data as CSV is inefficient (large size, slow parsing, poor typing) and increases storage and pipeline overhead. Like option A, it also complicates training/serving consistency because inference must run in BigQuery; you would still need equivalent SQL feature logic or repeated exports for new partitions, increasing cost and operational complexity.

This is not an appropriate pattern because Cloud Functions do not typically deploy a new Cloud Composer DAG in response to storage events; DAGs are normally pre-authored and scheduled within an existing Composer environment. More importantly, Cloud Composer is a long-lived Airflow environment with ongoing baseline cost, which conflicts with the requirement to minimize always-on orchestration compute. A per-file trigger would also react to each object arrival rather than naturally reasoning about all files since the last successful run. For an ML workflow on Google Cloud, Vertex AI Pipelines is the more suitable orchestrator.

A Cloud Storage-triggered Cloud Function would fire on every object creation event, so with 40–60 files per hour and bursty arrivals, this could launch many separate pipeline runs. That does not inherently satisfy the requirement to run only when new files have arrived since the last successful run as a single coordinated batch unless additional debouncing, locking, and aggregation logic is built. The option does not mention such coordination, so it is incomplete and operationally risky. It may also create duplicate or overlapping runs during bursts.

Cloud Composer can orchestrate the workflow, but it requires an always-on Airflow environment with persistent scheduler and worker resources. That directly conflicts with the requirement to minimize always-on compute costs for orchestration. Using a GCSObjectUpdateSensor also implies a heavier polling/sensor-based pattern than necessary for this use case. While technically feasible, it is less cost-effective and less aligned with Google Cloud's managed ML orchestration approach than Vertex AI Pipelines plus a lightweight trigger.

Cloud Bigtable can deliver very low-latency key/value reads and can handle high write throughput, so it may appear suitable for online feature retrieval. However, it lacks native feature-store capabilities: point-in-time consistent historical retrieval, feature definitions/metadata, and managed feature versioning workflows. You would need to design row-key schemas, maintain multiple versions, manage retention, and build offline training extracts yourself, increasing operational effort and risk of leakage.

Vertex AI Datasets help organize and version training datasets, but they are not an online feature serving system. They do not provide sub-50 ms key-based retrieval of the latest features for real-time inference, nor do they provide feature-store semantics like point-in-time feature retrieval across entities. You would still need a separate online store and custom pipelines for feature computation, serving, and historical reconstruction.

BigQuery timestamp-partitioned tables are strong for offline analytics and can store 200M rows/day efficiently, but BigQuery is not intended for ultra-low-latency online feature serving. The BigQuery Storage Read API is optimized for high-throughput batch reads (e.g., training pipelines), not per-request millisecond lookups for live inference. Achieving sub-50 ms consistently would typically require caching or a dedicated online store, increasing complexity.

Partially addresses governance by using Vertex AI-managed resources, but it’s not the strongest/most explicit guarantee of automatic end-to-end lineage across both training and weekly batch prediction. “Vertex AI training pipeline” is ambiguous (could be interpreted as a training job rather than Vertex AI Pipelines). Without explicitly using Vertex AI Pipelines/Metadata-integrated components, you may not get a complete lineage graph linking data snapshot, model artifact, and prediction job executions automatically.

BigQuery can help with data governance, but this option relies on custom SDK prediction routines and a standalone custom training job, which typically requires manual logging to achieve end-to-end lineage. You might capture some metadata in BigQuery or logs, but it won’t automatically create a unified lineage graph linking the exact training snapshot, the registered model artifact, and each batch prediction job in a consistent, audit-friendly way.

Vertex AI Experiments is useful for tracking metrics, parameters, and comparisons during model development, and Model Registry helps manage model versions. However, Experiments does not automatically capture full lineage across batch prediction jobs and their outputs. You would still need additional orchestration/metadata wiring to link the exact training data snapshot to the model and to each weekly batch prediction execution in an auditable lineage graph.

This option relies on the default Compute Engine service account, which is generally discouraged for least-privilege designs because it is commonly reused across workloads. Even if it can technically allow notebook code to call Vertex AI APIs, it increases blast radius and makes access harder to isolate for a short-lived contractor prototype. The question explicitly emphasizes least-privilege, so a dedicated service account is the better pattern.

This option is flawed because it grants Notebook Viewer to the contractors, which is a read-only role and does not provide the level of access needed to open and run a shared notebook environment for experimentation. It also grants Notebook Viewer to the contractors rather than a role that actually enables active notebook use. Although using a dedicated service account with Vertex AI User is a good idea, the human-access portion is insufficient, so the overall configuration is not correct.

This option is incorrect because a user-managed notebook instance should not run under an individual contractor's personal identity. Tying the runtime to one lead contractor creates operational risk, poor continuity, and weak separation between human and workload identities. It also does not properly grant the rest of the contractors the permissions needed to use the notebook environment or ensure a clean least-privilege design.

Cloud Functions can react to Cloud Storage object finalize events and could trigger Vertex AI training. However, the requirement is to retrain only when code changes are pushed to the main branch and to keep full version control of code and build artifacts. A bucket-based workflow is weaker for Git branch semantics, commit traceability, and artifact provenance, and can trigger on non-meaningful object updates.

Manually running gcloud to submit Vertex AI training jobs does not meet the requirement to minimize manual steps and ensure automatic retraining on code pushes. It also reduces consistency and auditability because humans may forget to retrain, use the wrong parameters, or submit from an untracked local state, undermining reproducibility and governance.

Cloud Composer (managed Airflow) can poll for changes and launch training, but it is an always-on orchestration service with continuous environment costs. A daily polling DAG also violates the “only when code changes are pushed to main” intent because it is schedule-based and can introduce delay. Composer is better for complex, multi-stage pipelines, not simple Git-triggered retraining.

Vertex AI Workbench managed notebooks provide a strong web-based notebook experience, but they do not inherently provide a distributed Spark runtime. To run PySpark at scale, you typically need to connect to a Dataproc cluster or configure Spark yourself, plus manage kernels/connectors. That adds setup steps and potential friction for a 24-hour sprint, violating the “distributed Spark out of the box” and “minimal setup” requirements.

Colab Enterprise is convenient for managed notebooks, but distributed Spark on 40 TB is not its primary, turnkey use case. You would still need a Spark backend (commonly Dataproc) and additional configuration to ensure stable cluster resources, networking, and consistent dependencies for 12 users. It can work, but it’s not the fastest, most robust path compared to Dataproc’s native Spark + notebook components.

A single Compute Engine VM with manually installed Spark and Jupyter is the slowest and riskiest approach for a time-boxed sprint. It requires manual installation, dependency management, and operational work (security, user access, scaling). It also does not provide robust distributed Spark execution unless you build and manage a multi-node Spark cluster yourself, which contradicts the “minimal setup” requirement.