General Code Snippets of Databricks

 To calculate the time taken by code snippets or notebook to run in Databricks. Even one can find the elapsed time of particular code or bunch of code. Code in Scala : To check the how much time a SQL query is taking to run in Databricks. val startTime = System.nanoTime() val result = spark.sql("SELECT * FROM my_table") result.show() /we can remove this if one doesn't want to table result val endTime = System.nanoTime() val duration = (endTime - startTime) / 1e9 println(s"Query took $duration seconds") In above code output the start time is in timestamp format, to convert it into date. For that link is given below :  link to convert timestamp into date Code in Python/Pyspark : To check the how much time a code snippet is taking to run. Or even one can try it in whole notebook of Databricks. import time start_time = time.time() <put the code here> end_time = time.time() elapsed_time = end_time - start_time print(f'Elapsed time: {elapsed_time}') li...