Full Databricks-Certified-Data-Engineer-Associate Practice Test and 234 Unique Questions, Get it Now!
The Best Databricks-Certified-Data-Engineer-Associate Exam Study Material Premium Files and Preparation Tool
GAQM Databricks-Certified-Data-Engineer-Associate is a certification exam designed to test the skills and knowledge of individuals in the field of data engineering. Databricks-Certified-Data-Engineer-Associate exam is created by GAQM, a leading global provider of certification programs for professionals in various industries. Databricks Certified Data Engineer Associate Exam certification is intended for those who work with Databricks and want to demonstrate their expertise in designing, building, and maintaining data pipelines using this technology.
NEW QUESTION # 34
A data engineer wants to create a data entity from a couple of tables. The data entity must be used by other data engineers in other sessions. It also must be saved to a physical location.
Which of the following data entities should the data engineer create?
- A. Temporary view
- B. Function
- C. View
- D. Table
- E. Database
Answer: D
Explanation:
A table is a data entity that is stored in a physical location and can be accessed by other data engineers in other sessions. A table can be created from one or more tables using the CREATE TABLE or CREATE TABLE AS SELECT commands. A table can also be registered from an existing DataFrame using the spark.catalog.createTable method. A table can be queried using SQL or DataFrame APIs. A table can also be updated, deleted, or appended using the MERGE INTO command or the DeltaTable API. Reference:
Create a table
Create a table from a query result
Register a table from a DataFrame
[Query a table]
[Update, delete, or merge into a table]
NEW QUESTION # 35
Which two conditions are applicable for governance in Databricks Unity Catalog? (Choose two.)
- A. Both catalog and schema must have a managed location in Unity Catalog provided metastore is not associated with a location
- B. You can have more than 1 metastore within a databricks account console but only 1 per region.
- C. If metastore is not associated with location, it's mandatory to associate catalog with managed locations
- D. If catalog is not associated with location, it's mandatory to associate schema with managed locations
- E. You can have multiple catalogs within metastore and 1 catalog can be associated with multiple metastore
Answer: B,C
Explanation:
You can have more than 1 metastore within a Databricks account console but only 1 per region:
This is correct. In Databricks Unity Catalog, you can create multiple metastores, but you can only have one metastore per region within an account.
If metastore is not associated with location, it's mandatory to associate catalog with managed locations: This is also correct. If a metastore does not have a default storage location, then the catalog must be associated with a managed location to ensure that data is stored properly within Unity Catalog.
NEW QUESTION # 36
A Databricks single-task workflow fails at the last task due to an error in a notebook. The data engineer fixes the mistake in the notebook. What should the data engineer do to rerun the workflow?
- A. Restart the cluster
- B. Rerun the pipeline
- C. Switch the cluster
- D. Repair the task
Answer: D
Explanation:
For a single-task workflow, the correct action is to repair the task, which reruns only the failed task after fixing the error, avoiding unnecessary re-execution of the whole workflow.
NEW QUESTION # 37
Which of the following describes when to use the CREATE STREAMING LIVE TABLE (formerly CREATE INCREMENTAL LIVE TABLE) syntax over the CREATE LIVE TABLE syntax when creating Delta Live Tables (DLT) tables using SQL?
- A. CREATE STREAMING LIVE TABLE should be used when data needs to be processed incrementally.
- B. CREATE STREAMING LIVE TABLE is redundant for DLT and it does not need to be used.
- C. CREATE STREAMING LIVE TABLE should be used when the subsequent step in the DLT pipeline is static.
- D. CREATE STREAMING LIVE TABLE should be used when the previous step in the DLT pipeline is static.
- E. CREATE STREAMING LIVE TABLE should be used when data needs to be processed through complicated aggregations.
Answer: A
Explanation:
Explanation
The CREATE STREAMING LIVE TABLE syntax is used when you want to create Delta Live Tables (DLT) tables that are designed for processing data incrementally. This is typically used when your data pipeline involves streaming or incremental data updates, and you want the table to stay up to date as new data arrives.
It allows you to define tables that can handle data changes incrementally without the need for full table refreshes.
NEW QUESTION # 38
A data engineer is processing ingested streaming tables and needs to filter out NULL values in the order_datetime column from the raw streaming table orders_raw and store the results in a new table orders_valid using DLT.
Which code snippet should the data engineer use?
- A.
- B.
- C.
- D.
Answer: C
NEW QUESTION # 39
A data engineer ingests semi-structured JSON logs into a Delta table using Auto Loader with schema evolution enabled. A new string field named userAgent appears in the JSON source data.
What happens to the new userAgent field?
- A. The userAgent field is added as a new nullable column in the Delta table and populated for new data.
- B. All existing data is backfilled with a default value for userAgent, and the column is made non-nullable.
- C. The userAgent field is silently ignored, and its data is dropped from all new files.
- D. Ingestion fails with a schema-mismatch error until the table is manually altered.
Answer: A
NEW QUESTION # 40
A platform team is creating a standardized template for Databricks Asset Bundles to support CI/CD. The template must specify defaults for artifacts, workspace root paths, and a run identity, while allowing a "dev" target to be the default and override specific paths.
How should the team use databricks.yml to satisfy these requirements?
- A. Use bundle, artifacts, workspace, run_as, and targets at the top level; set one target with default: true and override workspace paths or artifacts under that target.
- B. Use deployment, builds, context, identity, and environments; set dev as default environment and override paths under builds.
- C. Use project, packages, environment, identity, and stages; set dev as default stage and override workspace under environment.
- D. Use roots, modules, profiles, actor, and targets; where profiles contain workspace and artifacts defaults and actor sets run identity.
Answer: A
Explanation:
In Databricks Asset Bundles, the databricks.yml file defines all top-level configuration keys, including bundle, artifacts, workspace, run_as, and targets. The targets section defines specific deployment contexts (for example, dev, test, prod). Setting default: true for a target marks it as the default environment. Overrides for workspace paths and artifact configurations can be defined inside each target while keeping defaults at the top level.
Reference Source: Databricks Asset Bundle Configuration Guide - "Structure of databricks.yml and target overrides."
NEW QUESTION # 41
A data engineer needs to apply custom logic to string column city in table stores for a specific use case. In order to apply this custom logic at scale, the data engineer wants to create a SQL user-defined function (UDF).
Which of the following code blocks creates this SQL UDF?
- A.
- B.
- C.
- D.
- E.
Answer: E
Explanation:
https://www.databricks.com/blog/2021/10/20/introducing-sql-user-defined-functions.html
NEW QUESTION # 42
A data engineer is writing a script that is meant to ingest new data from cloud storage. In the event of the Schema change, the ingestion should fail. It should fail until the changes downstream source can be found and verified as intended changes. Which command will meet the requirements?
- A. addNewColumns
- B. failOnNewColumns
- C. none
- D. rescue
Answer: B
Explanation:
The failOnNewColumns option enforces strict schema control during ingestion. If new columns appear in the source, the ingestion job fails immediately, ensuring downstream changes are reviewed and validated before processing continues.
NEW QUESTION # 43
A new data engineering team team has been assigned to an ELT project. The new data engineering team will need full privileges on the table sales to fully manage the project.
Which of the following commands can be used to grant full permissions on the database to the new data engineering team?
- A. GRANT SELECT CREATE MODIFY ON TABLE sales TO team;
- B. GRANT SELECT ON TABLE sales TO team;
- C. GRANT USAGE ON TABLE sales TO team;
- D. GRANT ALL PRIVILEGES ON TABLE team TO sales;
- E. GRANT ALL PRIVILEGES ON TABLE sales TO team;
Answer: E
NEW QUESTION # 44
A data engineering team has two tables. The first table march_transactions is a collection of all retail transactions in the month of March. The second table april_transactions is a collection of all retail transactions in the month of April. There are no duplicate records between the tables.
Which of the following commands should be run to create a new table all_transactions that contains all records from march_transactions and april_transactions without duplicate records?
- A. CREATE TABLE all_transactions ASSELECT * FROM march_transactionsUNION SELECT * FROM april_transactions;
- B. CREATE TABLE all_transactions ASSELECT * FROM march_transactionsMERGE SELECT * FROM april_transactions;
- C. CREATE TABLE all_transactions ASSELECT * FROM march_transactionsINNER JOIN SELECT * FROM april_transactions;
- D. CREATE TABLE all_transactions ASSELECT * FROM march_transactionsINTERSECT SELECT * from april_transactions;
- E. CREATE TABLE all_transactions ASSELECT * FROM march_transactionsOUTER JOIN SELECT * FROM april_transactions;
Answer: A
Explanation:
The correct command to create a new table that contains all records from two tables without duplicate records is to use the UNION operator. The UNION operator combines the results of two queries and removes any duplicate rows. The INNER JOIN, OUTER JOIN, and MERGE operators do not remove duplicate rows, and the INTERSECT operator only returns the rows that are common to both tables. Therefore, option B is the only correct answer. References: Databricks SQL Reference - UNION, Databricks SQL Reference - JOIN, Databricks SQL Reference - MERGE, [Databricks SQL Reference - INTERSECT]
NEW QUESTION # 45
Which of the following can be used to simplify and unify siloed data architectures that are specialized for specific use cases?
- A. None of these
- B. Data lakehouse
- C. Data lake
- D. All of these
- E. Data warehouse
Answer: B
NEW QUESTION # 46
A data engineer wants to create a relational object by pulling data from two tables. The relational object does not need to be used by other data engineers in other sessions. In order to save on storage costs, the data engineer wants to avoid copying and storing physical data.
Which of the following relational objects should the data engineer create?
- A. Delta Table
- B. Temporary view
- C. Spark SQL Table
- D. View
- E. Database
Answer: B
Explanation:
A temporary view is a relational object that is defined in the metastore and points to an existing DataFrame. It does not copy or store any physical data, but only saves the query that defines the view. The lifetime of a temporary view is tied to the SparkSession that was used to create it, so it does not persist across different sessions or applications. A temporary view is useful for accessing the same data multiple times within the same notebook or session, without incurring additional storage costs. The other options are either materialized (A, E), persistent (B, C), or not relational objects ©. References: Databricks Documentation - Temporary View, Databricks Community - How do temp views actually work?, Databricks Community - What's the difference between a Global view and a Temp view?, Big Data Programmers - Temporary View in Databricks.
NEW QUESTION # 47
A data engineer has configured a Structured Streaming job to read from a table, manipulate the data, and then perform a streaming write into a new table.
The code block used by the data engineer is below:
If the data engineer only wants the query to process all of the available data in as many batches as required, which of the following lines of code should the data engineer use to fill in the blank?
- A. trigger(parallelBatch=True)
- B. trigger(processingTime="once")
- C. trigger(continuous="once")
- D. trigger(availableNow=True)
- E. processingTime(1)
Answer: D
Explanation:
https://spark.apache.org/docs/latest/api/python/reference/pyspark.ss/api/pyspark.sql.streaming.DataStreamWriter
NEW QUESTION # 48
In order for Structured Streaming to reliably track the exact progress of the processing so that it can handle any kind of failure by restarting and/or reprocessing, which of the following two approaches is used by Spark to record the offset range of the data being processed in each trigger?
- A. Checkpointing and Idempotent Sinks
- B. Replayable Sources and Idempotent Sinks
- C. Write-ahead Logs and Idempotent Sinks
- D. Structured Streaming cannot record the offset range of the data being processed in each trigger.
- E. Checkpointing and Write-ahead Logs
Answer: E
Explanation:
Structured Streaming uses checkpointing and write-ahead logs to record the offset range of the data being processed in each trigger. This ensures that the engine can reliably track the exact progress of the processing and handle any kind of failure by restarting and/or reprocessing. Checkpointing is the mechanism of saving the state of a streaming query to fault-tolerant storage (such as HDFS) so that it can be recovered after a failure. Write-ahead logs are files that record the offset range of the data being processed in each trigger and are written to the checkpoint location before the processing starts. These logs are used to recover the query state and resume processing from the last processed offset range in case of a failure. References: Structured Streaming Programming Guide, Fault Tolerance Semantics
NEW QUESTION # 49
A data engineer needs access to a table new_table, but they do not have the correct permissions. They can ask the table owner for permission, but they do not know who the table owner is.
Which of the following approaches can be used to identify the owner of new_table?
- A. Review the Permissions tab in the table's page in Data Explorer
- B. Review the Owner field in the table's page in Data Explorer
- C. There is no way to identify the owner of the table
- D. Review the Owner field in the table's page in the cloud storage solution
- E. All of these options can be used to identify the owner of the table
Answer: B
Explanation:
he approach that can be used to identify the owner of new_table is to review the Owner field in the table's page in Data Explorer. Data Explorer is a web-based interface that allows users to browse, create, and manage data objects such as tables, views, and functions in Databricks1. The table's page in Data Explorer provides various information about the table, such as its schema, partitions, statistics, history, and permissions2. The Owner field shows the name and email address of the user who created or owns the table3. The data engineer can use this information to contact the table owner and request for permission to access the table.
The other options are not correct or reliable for identifying the owner of new_table. Reviewing the Permissions tab in the table's page in Data Explorer can show the users and groups who have access to the table, but not necessarily the owner4. Reviewing the Owner field in the table's page in the cloud storage solution can be misleading, as the owner of the data files may not be the same as the owner of the table5. There is a way to identify the owner of the table, as explained above, so option E is false.
:
1: Data Explorer | Databricks on AWS
2: Table details | Databricks on AWS
3: Set owner when creating a view in databricks sql - Databricks - 9978
4: Table access control | Databricks on AWS
5: External tables | Databricks on AWS
NEW QUESTION # 50
A data engineer wants to create a new table containing the names of customers who live in France.
They have written the following command:
CREATE TABLE customersInFrance
_____ AS
SELECT id,
firstName,
lastName
FROM customerLocations
WHERE country = 'FRANCE';
A senior data engineer mentions that it is organization policy to include a table property indicating that the new table includes personally identifiable information (Pll).
Which line of code fills in the above blank to successfully complete the task?
- A. "COMMENT PII"
- B. COMMENT "Contains PIT
- C. TBLPROPERTIES PII
- D. 0
Answer: C
Explanation:
To include a property indicating that a table contains personally identifiable information (PII), the TBLPROPERTIES keyword is used in SQL to add metadata to a table. The correct syntax to define a table property for PII is as follows:
CREATE TABLE customersInFrance
USING DELTA
TBLPROPERTIES ('PII' = 'true')
AS
SELECT id,
firstName,
lastName
FROM customerLocations
WHERE country = 'FRANCE';
The TBLPROPERTIES ('PII' = 'true') line correctly sets a table property that tags the table as containing personally identifiable information. This is in accordance with organizational policies for handling sensitive information.
NEW QUESTION # 51
A data engineer needs to create a table in Databricks using data from a CSV file at location /path/to/csv.
They run the following command:
Which of the following lines of code fills in the above blank to successfully complete the task?
- A. None of these lines of code are needed to successfully complete the task
- B. USING CSV
- C. FROM CSV
- D. FROM "path/to/csv"
- E. USING DELTA
Answer: D
Explanation:
A data lakehouse is a new paradigm that can be used to simplify and unify siloed data architectures that are specialized for specific use cases. A data lakehouse combines the best of both data lakes and data warehouses, providing a single platform that supports diverse data types, open standards, low-cost storage, high-performance queries, ACID transactions, schema enforcement, and governance. A data lakehouse enables data engineers to build reliable and scalable data pipelines that can serve various downstream applications and users, such as data science, machine learning, analytics, and reporting. A data lakehouse leverages the power of Delta Lake, a storage layer that brings reliability and performance to data lakes. Reference: What is a data lakehouse?, Delta Lake, Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics
NEW QUESTION # 52
A data engineer is designing an ETL pipeline to process both streaming and batch data from multiple sources. The pipeline must ensure data quality, handle schema evolution, and provide easy maintenance. The team is considering using Delta Live Tables (DLT) in Databricks to achieve these goals. They want to understand the key features and benefits of DLT that make it suitable for this use case. Why is Delta Live Tables (DLT) an appropriate choice?
- A. Automatic data quality checks, built-in support for schema evolution, and declarative pipeline development
- B. Requires custom code for data quality checks, no support for streaming data, and complex pipeline maintenance
- C. Supports only batch processing, no data versioning, and high infrastructure costs
- D. Manual schema enforcement, high operational overhead, and limited scalability
Answer: A
Explanation:
Delta Live Tables (DLT) is appropriate because it provides automatic data quality checks through expectations, built-in support for schema evolution, and a declarative approach to pipeline development, making it easier to build and maintain both streaming and batch ETL pipelines.
NEW QUESTION # 53
A data engineer has a Python variable table_name that they would like to use in a SQL query. They want to construct a Python code block that will run the query using table_name.
They have the following incomplete code block:
____(f"SELECT customer_id, spend FROM {table_name}")
Which of the following can be used to fill in the blank to successfully complete the task?
- A. spark.sql
- B. spark.delta.table
- C. dbutils.sql
- D. spark.table
- E. spark.delta.sql
Answer: A
Explanation:
The spark.sql method can be used to execute SQL queries programmatically and return the result as a DataFrame. The spark.sql method accepts a string argument that contains a valid SQL statement. The data engineer can use a formatted string literal (f-string) to insert the Python variable table_name into the SQL query. The other methods are either invalid or not suitable for running SQL queries. References: Running SQL Queries Programmatically, Formatted string literals, spark.sql
NEW QUESTION # 54
A data engineer wants to analyze historical versions of a Delta table to debug a data quality issue. The engineer needs to query the table as it existed two days ago. Which Delta Lake feature allows querying older snapshots of a table?
- A. VACUUM
- B. Z-Ordering
- C. Time Travel
- D. OPTIMIZE
Answer: C
NEW QUESTION # 55
A data engineer needs access to a table new_table, but they do not have the correct permissions. They can ask the table owner for permission, but they do not know who the table owner is.
Which of the following approaches can be used to identify the owner of new_table?
- A. Review the Permissions tab in the table's page in Data Explorer
- B. Review the Owner field in the table's page in Data Explorer
- C. There is no way to identify the owner of the table
- D. Review the Owner field in the table's page in the cloud storage solution
- E. All of these options can be used to identify the owner of the table
Answer: B
Explanation:
he approach that can be used to identify the owner of new_table is to review the Owner field in the table's page in Data Explorer. Data Explorer is a web-based interface that allows users to browse, create, and manage data objects such as tables, views, and functions in Databricks1. The table's page in Data Explorer provides various information about the table, such as its schema, partitions, statistics, history, and permissions2. The Owner field shows the name and email address of the user who created or owns the table3. The data engineer can use this information to contact the table owner and request for permission to access the table.
The other options are not correct or reliable for identifying the owner of new_table. Reviewing the Permissions tab in the table's page in Data Explorer can show the users and groups who have access to the table, but not necessarily the owner4. Reviewing the Owner field in the table's page in the cloud storage solution can be misleading, as the owner of the data files may not be the same as the owner of the table5. There is a way to identify the owner of the table, as explained above, so option E is false.
Reference:
1: Data Explorer | Databricks on AWS
2: Table details | Databricks on AWS
3: Set owner when creating a view in databricks sql - Databricks - 9978
4: Table access control | Databricks on AWS
5: External tables | Databricks on AWS
NEW QUESTION # 56
Identify a scenario to use an external table.
A Data Engineer needs to create a parquet bronze table and wants to ensure that it gets stored in a specific path in an external location.
Which table can be created in this scenario?
- A. An external table where the schema has managed location pointing to specific path in external location.
- B. An external table where the location is pointing to specific path in external location.
- C. A managed table where the location is pointing to specific path in external location.
- D. A managed table where the catalog has managed location pointing to specific path in external location.
Answer: B
NEW QUESTION # 57
A data engineer is designing a data pipeline. The source system generates files in a shared directory that is also used by other processes. As a result, the files should be kept as is and will accumulate in the directory. The data engineer needs to identify which files are new since the previous run in the pipeline, and set up the pipeline to only ingest those new files with each run.
Which of the following tools can the data engineer use to solve this problem?
- A. Data Explorer
- B. Auto Loader
- C. Delta Lake
- D. Databricks SQL
- E. Unity Catalog
Answer: B
Explanation:
Auto Loader is a tool that can incrementally and efficiently process new data files as they arrive in cloud storage without any additional setup. Auto Loader provides a Structured Streaming source called cloudFiles, which automatically detects and processes new files in a given input directory path on the cloud file storage.
Auto Loader also tracks the ingestion progress and ensures exactly-once semantics when writing data into Delta Lake. Auto Loader can ingest various file formats, such as JSON, CSV, XML, PARQUET, AVRO, ORC, TEXT, and BINARYFILE. Auto Loader has support for both Python and SQL in Delta Live Tables, which are a declarative way to build production-quality data pipelines with Databricks. References: What is Auto Loader?, Get started with Databricks Auto Loader, Auto Loader in Delta Live Tables
NEW QUESTION # 58
A data engineer converts an external Delta table to a Unity Catalog managed table. A Structured Streaming job that reads from the table continues running during the conversion. After the conversion completes, the streaming job stops processing new records.
How should the data engineer resolve the issue?
- A. Run REFRESH TABLE on the converted table to update the streaming checkpoint.
- B. Restart the streaming job so that it uses the new managed-table location.
- C. Grant the streaming job additional permissions on the new managed-storage location.
- D. Delete the streaming checkpoint directory and reprocess the complete source from the beginning.
Answer: B
Explanation:
Databricks intentionally stops existing streaming reads and writes after an external Delta table is converted to a Unity Catalog managed table. This protects data consistency because the table's managed storage location and access path have changed. The documented recovery action is to restart the stream with the same configuration. The existing checkpoint can then resume processing from the last committed offset, and supported path-based access is redirected to the converted managed table. REFRESH TABLE invalidates cached metadata but does not repair a stopped Structured Streaming query or modify its checkpoint. Deleting the checkpoint would discard progress information and could cause unnecessary reprocessing or duplicate output. Additional permissions are not the indicated solution when the failure is specifically caused by managed-table conversion. Therefore, option A is correct.
NEW QUESTION # 59
......
Get Instant Access to Databricks-Certified-Data-Engineer-Associate Practice Exam Questions: https://examtorrent.dumpsreview.com/Databricks-Certified-Data-Engineer-Associate-exam-dumps-review.html
