Hi I can find easily courses books with technicalities of spark and technologies .
What I cannot find are good courses for designing meaning for choosing partition keys, folder structure, pipeline stages (do you want a cleanup phase and when not etc).
Here is a good random post I found https://www.google.com/amp/s/hub.packtpub.com/common-big-data-design-patterns/amp/
https://www.researchgate.net/figure/Typical-four-layered-big-data-architecture-ingestion-processing-storage-and_fig1_329368513
Is there something better than such random posts like books courses? All I find is courses that are either too low level with technicalities or top high levels.
Yeah I know there is the data intensive applications I read it and it's not dealing with these specific design decisions its more of 40 k feel look.
