What is the default size threshold for broadcasting a table in Spark?
💡 Model Answer
In Spark, the default threshold for automatically broadcasting a table is 10 megabytes. This value is controlled by the configuration property spark.sql.autoBroadcastJoinThreshold. If a table’s size is below this threshold, Spark will broadcast it to all executors during a join, eliminating the need for a shuffle. The threshold can be increased or decreased by setting the property to a larger or smaller value, or set to a negative number to disable automatic broadcasting entirely. For example, spark.conf.set("spark.sql.autoBroadcastJoinThreshold", "20MB") will allow tables up to 20 MB to be broadcasted.
This answer was generated by AI for study purposes. Use it as a starting point — personalize it with your own experience.
🎤 Get questions like this answered in real-time
Assisting AI listens to your interview, captures questions live, and gives you instant AI-powered answers on a discreet on-screen overlay.
Get Assisting AI — Starts at ₹500