When I tested the features of the recently released Databricks on the Google Cloud platform, I checked out the BigQuery integration. Databricks is using a fork of the open-source Google Spark Connector for BigQuery. So I wondered how to check if a certain predicate of a query is indeed pushed down to BigQuery (or not). It turns out it is easy!
https://medium.com/geekculture/predicate-pushdown-for-apache-spark-with-google-bigquery-2ad4f9e81e6
Speak Your Mind