Help us build a better product, earn the PaxPro Advocate badge: Six short questions, one cool badge.
Intelligent Automation through the Automatic Project Flows (APF) feature allows you to intelligently operationalize curated data flows. With a single click, APF computes the entire sequence of data prep Steps across Paxata Projects, datasets and AnswerSets to produce an end-to-end, automated output Flow for your data. You can set the Flow to run on a recurring time-based schedule, or run it just once to produce an end-result AnswerSet. All runs can then be easily managed through the APF Monitoring Interface. For complete details, see Automatic Project Flows.
What are the key benefits of APF over the Automation feature?
Business Analysts and Data Engineers can simplify complex data flows by breaking them into smaller groups of Paxata Projects that can be operationalized—with each Project focused on performing a related or cohesive set of Steps for improved readability and limited complexity. When you're finished creating your Projects, simply select the final Project in the sequence as your "target" Project. APF takes care of the rest—sequencing, preparing and automating the entire end-to-end flow without any manual stitching required.
For teams that require input from both Business and the IT Leader, the data prep process can be simplified when members build Paxata Projects that depend on output AnswerSets created by others. Everyone completes their data prep work in their own Paxata Project, and then the entire sequence is operationalized from a single "target" Project. APF takes care of the rest with no manual stitching required, regardless of who created or owns the Projects and AnswerSets. Members of the team can then use the APF Monitoring Interface to view how their Projects and AnswerSets participate in the Flow's final output.
How do I migrate my legacy Automation job to APF?
The Paxata Customer Success team has built a utility that uses information about current Automation schedules and lists of Projects in a tenant, and then identifies all of the “Target Projects” for which an APF needs to be created. Once the APFs are created, users can set up their schedules, custom import options, and export configurations.
Can a customer have both Legacy Automation and Intelligent Automation?
No. Automation and APF cannot co-exist in the same tenant.
I don't see the APF feature in my software. What am I missing?
The APF is a feature is behind a feature flag. Contact Paxata Customer Success to enable the feature.
My Flow has a lot of input datasets and output projects. How do I see those details?
You can easily determine metadata statistics for the dataset inputs by hovering your mouse over a dataset name in the DATASETS column. The dataset's version, creation date and the user who added it to the Library, and the number of columns and rows are displayed in a pop-up window. The corresponding is true for Projects (Outputs) as well. From the pop-over, one can click on a button to navigate to the Dataset/Project as desired. For complete details, see Automatic Project Flows.
Legacy Automation allows me to configure import options. What about APF?
All of the datasets that are identified as part of a Flow are displayed on the Inputs tab. If a dataset was previously imported from a Connector data source, there is an option to re-import the dataset. When that option is selected, you’re provided with the ability to configure the re-import options, just like Legacy Automation.
With Automation, I can export to 3rd Party Data Sources. What about APF?
Yes! Export of Lens Output to data sources is just like how it works in Legacy Automation. The Outputs tab provides a list of all the output AnswerSets that are published from the Flow. Click the Configure Lens button for the lens and the Export panel opens at the bottom of the page. By default, AnswerSets are published to the Paxata Library. To also publish out to an external data source, click the drop-down for the Export Lens field and select "Library and Export". You can then specify the output location details and any export formatting options for that AnswerSet.
I have created an APF, but my Project script has changed since then. How do I update the Projects in a Flow?
You can update to the latest version of any of theProjects as long as the two Project versions have the same inputs and lenses. You can update a Flow to use the latest version of all Projects. This can be done from the "Actions" drop-down while on the Outputs tab. Select "Update Projects" and you are prompted to confirm your selection.For complete details, see Automatic Project Flows.
My Project consumes an AnswerSet that it produces. Can I create a Flow for it?
Yes. A Project consuming its own AnswerSet is called a self-loop and this is supported. When such a Project is involved in the Flow, the AnswerSet that causes the loop is brought in as a special input to the Project and is depicted by a dotted line for the looped input.
Does APF have REST API support?
Yes. Please contact Paxata Customer Success for the REST API documentation.
Do existing job limits/quotas apply for APF?
Yes. In the APF world, we consider each Dataset Import and Project Execution as a "chore". Before the execution of a chore begins, a check is performed to determine if the group/tenant has sufficient quotas available. If the quotas have been exhausted, we do not run the chore and then a run of the Flow is failed. Note: the quotas from the Legacy Automation feature are applied to APF.
When you visit a Project, you will see an icon in the top right-hand corner of the header next to the “Create Project Flow” button. Click this icon to find all the Flows in which the Project participates. Please note that if you don't have sufficient permissions to View a particular Flow, the search results will exclude it.
How do I share a Project Flow with my team?
In the Project Flows list page, every Flow that you have permissions to share has a clickable Permission button. Click on that button to adjust the Permissions on the Flow. You will be taken to a page that is very similar to how permissions work for Projects and datasets.
I have a Flow and there are many lenses in the Outputs tab that are enabled by default, and I am not able to disable them. Why?
If your lens has an indicator like the one displayed below, it means that the output from the lens is used by a Project in the Flow. Disabling such a lens would cause a failure when the Flow runs. So, we have prevented users from disabling these required lenses. You can, however, decide whether to publish the output to just Library or to also export it to a Data Source.
What happens if, during the run of a Flow, one of the imports or Projects fail?
If any chore (Dataset Import or Project Execution) fails during the run of a Flow, the entire run fails. You can see the chore status in the Run Details tab. Any chore that failed will have a “display errors and warnings” link, which, when clicked, opens a panel to display the errors encountered during execution.
Can I create more than one Flow for a Project?
Yes. However, best practices recommend that you determine if a Project is already participating in other Flows. If so, then you will want to be clear which versions of the Project each Flow is using.
How do I identify if a dataset/AnswerSet was produced as part of an APF Run?
In Library, on every dataset (or AnswerSet), a new menu item “Go to Run” has been added. If the dataset was produced by a Project Flow, the button is enabled. Clicking on the button takes you to the Run Details tab of the Run that produced the dataset.
Can I cancel the run of a Flow?
Yes, you can. While a run is in progress, you can visit the Run Details page for that run and click on the "Stop" button to stop a run that is in progress. This will put the run into a cancel mode and confirmation message is displayed. Any chores currently in progress will complete and then run will halt.
My Flow just started running with wrong configurations. What can I do now?
You can cancel the run of the Flow by clicking on the "Stop" button from the Run Details page. Once the UI acknowledges that a request to cancel has been made, you can run the Flow again.
ExcelSharing in Excel is very easy. All you have to do is send your file to another person or group of people. This is typically done via email, but it is also accomplished by putting the file on a shared drive or saving it in Sharepoint. At this point there is no security on your data and you need to put complete trust in the other individuals that they will not take the data out the door with them. It is important to note that sharing is not the same thing as collaborating. Collaborating suggests that the users are essentially equal and that everyone can do everything without having to coordinate with other users.
You probably have experienced some of these issues associated with sharing.
- Once the file gets larger, you may not be able to send it via email.
- When you get a newer version of the source data, you need to update the spreadsheet and send it out again. This makes you the bottleneck.
- You have a hard time taking vacations or even a sick day, because you are needed to complete the process.
- It becomes very easy to mistake an old version of the file as the current because all the prior versions are in someone else's email. People might come to the wrong conclusion because their information is out of date.
- If the person you shared the data with sees something that should be changed, if they change it the original owner no longer has the most recent version. Ownership becomes a problem.
- If the person just sends a message back to you to make the changes, you continue to be the bottleneck.
- ...and many others
PaxataSince everything in Paxata occurs on a server, just changing a few settings makes it very easy to give access of your data to others. If you are the owner of the dataset meaning that you are the one that initially created it, then you get to decide who else can work with it. If you were given rights to someone else's assets then that person has already decided if they wanted to give you the rights to share it with others. In Paxata, when the data is updated to a more recent version of the data, everyone that has rights to the dataset will automatically see it; no additional work is necessary. Since datasets are not copied, the dataset only exists once. It is not flooding your emails or file servers with various versions of the file. Most importantly, the data remains secure.
To share a dataset
Hover over the "More Actions" then select the menu "edit details"
On the left side of the screen, select the "permissions" menu
Select the "USERS" menu (or GROUPS if appropriate)
Select the user by opening the drop down then either scroll to the user or type in to search
Select the permissions that are appropriate, then click save (or repeat for other users)
In this example, I have now logged in as user2 and the following image shows the datasets that I can see. Take note that the file was created by a different user. You can also see that the delete button has been deactivated since the initial user didn't give user2 the right to delete the dataset.