Load a whole folder and append multiple same-structure CSV file
Greetings!
I looking for a way to load and append a whole set of files into Pyramid (same-structure files from different divisions, or from different days). The equivalent feature in PowerBI would be the "folder" import where files are appened and a first column is created with the name of the file, to keep the origin of the data.
Any idea? Many thanks in advance,
Gauthier
8 replies
-
Hi Gauthier,
I've moved this post from the BI Office (our old platform) Q&A to the Pyramid Q&A as you had accidentally posted it to the wrong Forum!
The functionality you want exists today in the Model app in Pyramid. It is a little bit more fiddly than the Power BI approach due to the fact that Pyramid is server based and the simple upload of a single file via HTTP is not really suitable for multi file uploads that could be many MB of data.
Instead, you need to tell the Pyramid server where the folder is and make sure that it is accessible to the server, typically by being on a shared network drive.
Use the "Multi File" data source block in Pyramid and specify the shared folder location and a search string for the filenames, e.g. data*.csv. An option on properties panel of the block is to include the filename as a column in the data:
Hope that helps.
Ian
-
Thank you for the quick reply Ian. This is great. I'm just struggling with the pathname. If it is on a Mac, what would the syntax be? If it on the cloud (like Google Drive), what should be the url format?
-
Hi Gauthier Vasseur
Currently, we only connect to a shared folder, if the local google drive is mapped to a shared folder then yes, you'll be able to connect to it.
You can also point to a URL, but that doesn't enable multi-file uploads.
As for the Mac, the safari browser should work like any other browser.
The shared path should look like this.
Note, that the file name is at the end of the string, hence each Excel file you upload need to have it's own shared file string. -
Thank you! This is helpful. I just need 1 last bit of help (pardon my ignorance here). I'm not sure how to declare a shared folder on my Mac for Pyramid to access it. Or is there a place that could be a shared folder on the cloud where I could drop the files?
-
Good evening, I tried many syntaxes but could not make it work. Any chance we could hop on a screen share session?
-
Thank you! I look forward to it. I am on Pacific Time. +1 650 *** **** or any web conference system.