An open-source Python library for simplifying local testing of Databricks workflows using PySpark and Delta tables. This library enables seamless testing of PySpark processing logic outside Databricks ...

Introduced in Spark 4.x, Python Data Source API allows you to create PySpark Data Sources leveraging long standing python libraries for handling unique file types or specialized interfaces with spark ...