We see that PDT doesn't scale well with the the amount of data complexity you usually have to deal with in the warehouse.
On top of that, the results of PDTs are only accessible to Looker. As your data team matures, there will be many other users and applications that need to take advantage of your transformed data in the warehouse. (e.g. modelling in Python)
Dataform also brings other features not available in PDTs. For example:
- Data validations tests (assertions) to ensure data quality in all your data applications (including Looker)
- Incremental tables: You can define tables that update incrementally instead of being rebuilt from scratch, hence delivering better performance and saving costs
- Reusable code. You can define reusable code snippets across your entire project
I much agree with this response. The term that I've heard to summarize all of the above is "semantic layer". That layer will run in the warehouse, vs. some external dashboard tool.
We see that PDT doesn't scale well with the the amount of data complexity you usually have to deal with in the warehouse.
On top of that, the results of PDTs are only accessible to Looker. As your data team matures, there will be many other users and applications that need to take advantage of your transformed data in the warehouse. (e.g. modelling in Python)
Dataform also brings other features not available in PDTs. For example:
- Data validations tests (assertions) to ensure data quality in all your data applications (including Looker)
- Incremental tables: You can define tables that update incrementally instead of being rebuilt from scratch, hence delivering better performance and saving costs
- Reusable code. You can define reusable code snippets across your entire project