Processing Multiple Models vs. One Big Model?
I use a number of Models, each with a specific job to do. A few models pull data into my warehouse from various sources, and a few models abstract different cuts of the raw data suitable for various different kinds of reporting.
I wind up having to open a list of models and executing them each one by one in a specific order. How should a really be doing this?
What seems intuitive to me is to be able to include models in the data flow of other models, so that I could have one model that understands the processing order and can orchestrate the whole batch process.
I don't see that that is an option. Perhaps the only option is to try to combine all my modular models into one big model, or just keep processing each by hand?
What's the best practice here? Thanks.
1 reply
-
Hi David Wipperfurth ,
The good news is that in Pyramid 2020 we introduced the 'Master Flow' which should enable you to sequence your dataflows.
You then can schedule the Model containing the master flow for daily process and it will process all the included data flows in the sequence they are ordered.
For that, you'll need to recreate your existing data flows (models) in the master flow.I'll try to illustrate in these images,
I hope this helps.
Kind Regards,
Yakov.