Practice Test #1

Incorrect. Type hints are not the defining advantage of pandas UDFs. Spark requires explicit return types for UDFs (and pandas UDFs) via Spark SQL types, and while Python type hints may be used in code style, they are not the core benefit tested. The key differentiator is vectorized execution with Arrow, not typing support.

Partially true but not the best answer. pandas UDFs do allow you to write logic using pandas/NumPy operations inside the function because inputs arrive as pandas objects. However, the exam question asks for a benefit compared to standard UDFs; the primary, canonical benefit is vectorized batch processing (and Arrow-based transfer), not merely that pandas APIs can be used.

Incorrect. Both standard PySpark UDFs and pandas UDFs operate on distributed Spark DataFrames; distribution is a property of Spark, not a unique advantage of pandas UDFs. pandas UDFs still run per partition/executor like other Spark transformations, so this does not distinguish them from standard UDFs.

Incorrect. pandas UDFs do not inherently guarantee in-memory processing or prevent spilling to disk. Spilling is determined by Spark’s execution plan, shuffle operations, partition sizes, and memory configuration. While Arrow can improve transfer efficiency, it does not change Spark’s fundamental memory management or eliminate disk spill behavior.

spark_df.describe() is a Spark DataFrame method that returns a new DataFrame containing basic descriptive statistics (count, mean, stddev, min, max) for numeric columns (and count/min/max for non-numeric). It does not generate visualizations such as histograms. It’s useful for quick numeric summaries but does not meet the requirement for “visual histograms.”

dbutils.data(spark_df).summarize() is not the correct API shape. In Databricks, dbutils.data is a utility namespace and summarize is called as a function (dbutils.data.summarize(df)), not by calling dbutils.data(df) as if it were a constructor returning an object with methods. This option is therefore syntactically/semantically incorrect for the intended summarize feature.

This task can be accomplished in a single line in Databricks notebooks using the built-in summarize helper. While in pure Spark (outside Databricks) you would typically need multiple steps (compute bins, then plot), Databricks provides a one-line EDA summary that includes histograms. So it is incorrect to claim it cannot be done in one line.

spark_df.summary() is another Spark DataFrame method that extends describe() by allowing additional statistics (e.g., percentiles) depending on parameters. Like describe(), it returns a DataFrame of summary statistics and does not automatically render histograms or other visual distribution plots. It helps with numeric profiling but not with visual histogram output.

Incorrect. Supplying all features in a Spark DataFrame is not what enables fs.score_batch to work with only customer_id. If you already have all feature columns, you could score with standard MLflow/Spark methods. fs.score_batch’s key value is automatic feature retrieval, which requires Feature Store model logging metadata.

Incorrect as the best answer. If the model truly uses only customer_id as its sole input feature, then the provided batch_df would indeed contain the required input column and scoring could work. However, this is a narrow special case and not the Feature Store-specific condition being tested by fs.score_batch. The generally correct condition is that the model was logged with Feature Store feature metadata, which allows Databricks to automatically look up additional features using customer_id.

Incorrect. The code can perform the desired inference in a valid and common situation: when the model is Feature Store–logged with feature lookups. In that case, fs.score_batch will enrich batch_df with the required features and return predictions.

Incorrect. Features do not need to be in a single Feature Store table. Feature Store supports training sets built from multiple feature tables via multiple lookups. What matters is that the model was logged with the feature lookup metadata so Feature Store knows which tables/features to retrieve and how to join them.

17.0 is the RMSE from the worst-performing validation fold only. Cross-validation is designed to summarize performance across all folds, not to report only the maximum error. While the worst fold can be useful for diagnosing instability, it is not the overall CV RMSE.

12.0 is the RMSE from one specific fold (and also the median of the three values). Although median can be used as a robust summary in some analyses, the conventional and expected definition of overall cross-validation RMSE for exam questions is the mean across folds, not the median or a single fold value.

39.0 is the sum of the fold RMSE values (10 + 12 + 17). Summing metrics across folds is not a standard way to report cross-validation performance because it scales with the number of folds and is not directly interpretable as an error measure for the model.

10.0 is the RMSE from the best-performing validation fold only. Selecting the minimum fold metric would overstate expected generalization performance and defeats the purpose of cross-validation. The overall CV RMSE should reflect performance across all folds, typically via the mean.

This is not why the code fails. While it is best practice to fit the imputer on the training set and then transform both training and test sets (to prevent leakage and ensure consistent medians), the immediate issue is that the code cannot impute even a single DataFrame because it never fits. The question asks why the block is not accomplishing imputation at all, not about workflow hygiene across splits.

inputCols and outputCols do not need to be exactly the same. Spark ML allows outputCols to be different so you can preserve original columns and create imputed versions (e.g., col -> col_imputed). You may choose to overwrite by setting outputCols equal to inputCols, but it’s optional. A mismatch between these lists only matters if their lengths differ or names are invalid, not because they must match exactly.

Calling fit() is required, but not “instead of transform” in the overall process. The correct sequence is fit() to create an ImputerModel, then transform() using that model. So transform is still needed to apply the learned medians to the DataFrame. The option is misleading because it implies transform should not be used, when in fact it must be used after fitting.