Visit the official Paxata Documentation portal for all of your doc needs.

Loading multiple datasets in a single project

I need to load multiple datasets in a single project. But the project allows to load only 1 time.

My requirement is to load different files, apply different processing to them and bring them to common format and then row-append them.

How do I do this under a single project in Paxata?

Best Answer

Answers

  • Hello,

    In order to do this, you would need to create Paxata projects for each one of your datasets to bring them to the desired common format. Once this is done in a project, you could publish these projects to the Paxata Library and then create a fresh Paxata project to row append all the desired projects.

    I hope this makes sense to you! Let me know if you need any further clarifications!


  • @Kumar Jayaram
    I don't see any "import" operation as you have shown in screenshot... may be, this is a new feature?

    In my case, the schema of the files itself is very different. I cant append before I do transformations on each single dataset.... 
  • I was able to find the "expand Append option" after which I can see the "import" step. This is nice.
    But what I am missing is that once I reach this step, I am unable to do schema transformation based on Lookups. That step alone is disabled while I am at the import step. All other options are available....Argh Argh! Any solutions
  • Hi @MagmaMan, it is not possible to do add a lookup/append step as a sub-step before a lookup/append. This is a product design decision. You can add the lookup step after the append is performed. Another option is to do your lookup in a separate project, publish the output (answerset), and append the answerset to your main project. Does that help?

  • MagmaManMagmaMan Posts: 15
    edited January 3, 2019 5:18AM
    @mikeblanko
    Thanks..... Ah... I can understand if Append is not allowed again...as it leads to a recursive trap...
    but why not lookup?
    btw.. I already accomplished this using a separate projects.... possibly tag all these projects to make them traceable as well... But, still if all are stored in a single project, it will be really good.
This discussion has been closed.