At ValidExamDumps, we consistently monitor updates to the Databricks-Certified-Associate-Developer-for-Apache-Spark-3.0 exam questions by Databricks. Whenever our team identifies changes in the exam questions,exam objectives, exam focus areas or in exam requirements, We immediately update our exam questions for both PDF and online practice exams. This commitment ensures our customers always have access to the most current and accurate questions. By preparing with these actual questions, our customers can successfully pass the Databricks Certified Associate Developer for Apache Spark 3.0 exam on their first attempt without needing additional materials or study guides.
Other certification materials providers often include outdated or removed questions by Databricks in their Databricks-Certified-Associate-Developer-for-Apache-Spark-3.0 exam. These outdated questions lead to customers failing their Databricks Certified Associate Developer for Apache Spark 3.0 exam. In contrast, we ensure our questions bank includes only precise and up-to-date questions, guaranteeing their presence in your actual exam. Our main priority is your success in the Databricks-Certified-Associate-Developer-for-Apache-Spark-3.0 exam, not profiting from selling obsolete exam questions in PDF or Online Practice Test.
The code block displayed below contains an error. The code block should return a copy of DataFrame transactionsDf where the name of column transactionId has been changed to
transactionNumber. Find the error.
Code block:
transactionsDf.withColumn("transactionNumber", "transactionId")
Correct code block:
transactionsDf.withColumnRenamed('transactionId', 'transactionNumber')
Note that in Spark, a copy is returned by default. So, there is no need to append copy() to the code block.
More info: pyspark.sql.DataFrame.withColumnRenamed --- PySpark 3.1.2 documentation
Static notebook | Dynamic notebook: See test 2, Question: 26 (Databricks import instructions)
The code block shown below should return a DataFrame with only columns from DataFrame transactionsDf for which there is a corresponding transactionId in DataFrame itemsDf. DataFrame
itemsDf is very small and much smaller than DataFrame transactionsDf. The query should be executed in an optimized way. Choose the answer that correctly fills the blanks in the code block to
accomplish this.
__1__.__2__(__3__, __4__, __5__)
Correct code block:
transactionsDf.join(broadcast(itemsDf), 'transactionId', 'left_semi')
This Question: is extremely difficult and exceeds the difficulty of questions in the exam by far.
A first indication of what is asked from you here is the remark that 'the query should be executed in an optimized way'. You also have qualitative information about the size of itemsDf and
transactionsDf. Given that itemsDf is 'very small' and that the execution should be optimized, you should consider instructing Spark to perform a broadcast join, broadcasting the 'very small'
DataFrame itemsDf to all executors. You can explicitly suggest this to Spark via wrapping itemsDf into a broadcast() operator. One answer option does not include this operator, so you can disregard
it. Another answer option wraps the broadcast() operator around transactionsDf - the bigger of the two DataFrames. This answer option does not make sense in the optimization context and can
likewise be disregarded.
When thinking about the broadcast() operator, you may also remember that it is a method of pyspark.sql.functions. One answer option, however, resolves to itemsDf.broadcast([...]). The DataFrame
class has no broadcast() method, so this answer option can be eliminated as well.
All two remaining answer options resolve to transactionsDf.join([...]) in the first 2 gaps, so you will have to figure out the details of the join now. You can pick between an outer and a left semi join. An
outer join would include columns from both DataFrames, where a left semi join only includes columns from the 'left' table, here transactionsDf, just as asked for by the question. So, the correct
answer is the one that uses the left_semi join.
The code block displayed below contains an error. The code block is intended to join DataFrame itemsDf with the larger DataFrame transactionsDf on column itemId. Find the error.
Code block:
transactionsDf.join(itemsDf, "itemId", how="broadcast")
broadcast is not a valid join type.
Correct! The code block should read transactionsDf.join(broadcast(itemsDf), 'itemId'). This would imply an inner join (this is the default in DataFrame.join()), but since the join type is not given in the
question, this would be a valid choice.
The larger DataFrame transactionsDf is being broadcasted, rather than the smaller DataFrame itemsDf.
This option does not apply here, since the syntax around broadcasting is incorrect.
Spark will only perform the broadcast operation if this behavior has been enabled on the Spark cluster.
No, it is enabled by default, since the spark.sql.autoBroadcastJoinThreshold property is set to 10 MB by default. If that property would be set to -1, then broadcast joining would be disabled.
More info: Performance Tuning - Spark 3.1.1 Documentation (https://bit.ly/3gCz34r)
The join method should be replaced by the broadcast method.
No, DataFrame has no broadcast() method.
The syntax is wrong, how= should be removed from the code block.
No, having the keyword argument how= is totally acceptable.
Which of the following code blocks returns a copy of DataFrame transactionsDf in which column productId has been renamed to productNumber?
More info: pyspark.sql.DataFrame.withColumnRenamed --- PySpark 3.1.2 documentation
Static notebook | Dynamic notebook: See test 2, Question: 35 (Databricks import instructions)
The code block shown below should return the number of columns in the CSV file stored at location filePath. From the CSV file, only lines should be read that do not start with a # character. Choose
the answer that correctly fills the blanks in the code block to accomplish this.
Code block:
__1__(__2__.__3__.csv(filePath, __4__).__5__)
Correct code block:
len(spark.read.csv(filePath, comment='#').columns)
This is a challenging Question: with difficulties in an unusual context: The boundary between DataFrame and the DataFrameReader. It is unlikely that a Question: of this difficulty level
appears in the
exam. However, solving it helps you get more comfortable with the DataFrameReader, a subject you will likely have to deal with in the exam.
Before dealing with the inner parentheses, it is easier to figure out the outer parentheses, gaps 1 and 5. Given the code block, the object in gap 5 would have to be evaluated by the object in gap 1,
returning the number of columns in the read-in CSV. One answer option includes DataFrame in gap 1 and shape[0] in gap 2. Since DataFrame cannot be used to evaluate shape[0], we can discard
this answer option.
Other answer options include size in gap 1. size() is not a built-in Python command, so if we use it, it would have to come from somewhere else. pyspark.sql.functions includes a size() method, but
this method only returns the length of an array or map stored within a column (documentation linked below). So, using a size() method is not an option here. This leaves us with two potentially valid
answers.
We have to pick between gaps 2 and 3 being spark.read or pyspark.DataFrameReader. Looking at the documentation (linked below), the DataFrameReader is actually a child class of pyspark.sql,
which means that we cannot import it using pyspark.DataFrameReader. Moreover, spark.read makes sense because on Databricks, spark references current Spark session
(pyspark.sql.SparkSession) and spark.read therefore returns a DataFrameReader (also see documentation below). Finally, there is only one correct answer option remaining.
More info:
- pyspark.sql.functions.size --- PySpark 3.1.2 documentation
- pyspark.sql.DataFrameReader.csv --- PySpark 3.1.2 documentation
- pyspark.sql.SparkSession.read --- PySpark 3.1.2 documentation
Static notebook | Dynamic notebook: See test 3, Question: 50 (Databricks import instructions)