Data Transformation: Liquid Templates as Maps in Logic Apps and integrating them with Power Automate

Table of Contents

  1. Intro
  2. Scenario
  3. Demonstration
    1. Logic App
    2. Power Automate
      1. Secondary option
      2. Result
  4. Recommendations

Intro

In an earlier post, I mentioned transforming data using inline JavaScript powered by an Azure Function. I want to resurrect that idea. This time, I want to extend that conversation and open it up to integrating with Azure Logic Apps and Liquid Templating.

If you don’t want to read this, skip to the meat of the content.

As a consultant, I feel it is part of my duty to inform. Much of what we do with our daily interactions with data is transform it. We take raw data and manipulate it to serve different purposes based on our primary function. Other functions then transform it to suit their needs. In my line of work, it often amazes me how many different objects, presentations, automations, or apps can live off the same dataset, ultimately serving the same purpose: to drive the organization forward.

Power Platform experts may ask why I need a Logic App workflow to do what I can do in Power Automate, and someone proficient in Logic Apps may ask the same thing about Power Automate. Well, besides the similarities between the two technologies, architecting solutions shouldn’t be limited to the tools we are familiar with. The best solutions are made up of the best parts of various tools.

What do I mean by the best parts of? The scenario I am going to present today could, in theory and practice, be created using an Office Script that lives in SharePoint or an Excel file in OneDrive. In this case, the “best part” of Excel would be the Office Script, as we would not deal with any other functionality tied to the workbook. We could run our scenario through that Office Script and return its output to the flow, achieving virtually the same result. However, that is a different article for a different time.

The concept I want us to grasp is the solution itself. As a Power Platform developer who has now spent time working with Azure resources, I want to understand how I can benefit from both worlds. I know Azure can be daunting because it is a vast platform encompassing many things. When one declares Azure, you have to ask yourself…Azure what?

Scenario

The data we are trying to transform or manipulate comes in many forms. Sometimes, data is returned as a CSV, XML, JSON, plain text, HTML, YAML, or binary data such as files or images. In some cases, it even comes in custom formats specific to an application or system. It is our job as developers to figure out the best way and best tools to transform this data and serve it to our end user in a usable way on the destination server.

We will be working with a JSON payload. This concept was inspired by Microsoft Learn; however, I will not detail the creation of the liquid template. Instead, I will demonstrate how to access the template from Power Automate.

If you have not done so already, please follow the link I previously provided to set up your template and Logic App. Additionally, please add an extra action to your Logic App beyond what is described in the Microsoft Learn documentation, as we will need to return the value to our Power Automate flow.

Demonstration

Logic App

Following the documentation should result in your logic app containing the following trigger and actions:

  • When a HTTP Request is received (Trigger)
  • Transform JSON To JSON
  • Response (needed to provide a response to Power Automate)

In the Logic App portal, you need to copy the URL of the Logic App workflow. This URL will then be invoked in your Power Automate flow.

Power Automate

In Power Automate, we will create a flow with a manual trigger. Following the trigger, we will initialize four variables: one each for FirstName, LastName, Phone, and Devices. These are the same keys used in the Microsoft Learn documentation for creating the liquid template.

As you proceed, please populate the variables with values that align with the documentation. For the ‘devices’ variable, ensure you input values separated by commas, akin to the example shown below. Ensuring that devices are represented as objects in our output will be crucial.

In our Compose action, we’ll construct a JSON object similar to the one shown in the image below. Naturally, we won’t be constructing it exactly as depicted; I’ve included the formulas for reference purposes.

Excluding the trigger, we have five actions and several formulas dedicated to handling data transformation, which constitutes the ideal workflow.

Secondary option

To create a new flow, start with a manual trigger and then add an HTTP action. The flow should resemble the one depicted below.

In parameters for the http action

  1. Paste the URI to your logic app
  2. Select post as the method
  3. In the body, paste JSON to be parsed
Result

Now when you run the flows you will end up with basically the same results.

Recommendations

You might be wondering, “What did I lose?” or “What did I gain?” Both inquiries are pertinent. Our brief demonstration revealed that multiple actions are required to achieve what one action could previously accomplish. However, with templating, we gain increased flexibility for complex transformations. Separating the transformation logic from the workflow also enhances maintainability, making updates easier over time. Additionally, there’s the benefit of integrating with a broader range of Azure services. As mentioned before, Azure’s capacity to handle more advanced workloads allows us to enhance our Power Automate flows significantly.

Ultimately, the decision depends on what you, as a developer and an organization, are comfortable with. For smaller workloads, a strict Power Automate approach may suffice, but for enterprise-level solutions, exploring integrations with Azure might be the better option.

Thank you for reading!

Power Automate: Transforming DevOps Queries into Automated Notifications

Since I began my journey in IT, I’ve managed all my projects through Azure DevOps. When used correctly, DevOps is a powerful tool for solution development. However, it’s easy to lose track of tasks—especially if, like me, you tend to complete tasks but sometimes forget to update the hours or close them. In such cases, setting up a reminder can be incredibly helpful.

To create this reminder, we’ll use Power Automate to pull information from a custom query set up in DevOps. While I won’t cover setting up queries in DevOps here, you can refer to Microsoft Learn’s documentation for guidance on managing and organizing your queries.

Once your query is ready, let’s move over to Power Automate. For this demonstration, we’ll use an instant trigger.

Step 1: Call the Azure DevOps Query

Our first action after the trigger will be to fetch the query results from Azure DevOps. Use the “Get query results” action, then select the project name and the query ID.

The query will return a JSON payload, and there are multiple ways to handle it. For our purposes, we’ll send an email to notify the relevant parties of their time status.

Step 2: Process and Format the JSON Payload

Power Automate’s Send Email action supports simple HTML and CSS styling, which allows us to format the email content. We will structure the JSON data as an HTML table. To start, we’ll initialize a string variable that will serve as our row constructor, holding the content for each column in the table.

To build our rows, we will use the “Append to String” variable action to add rows based on the number of rows returned in our JSON object. You’ll notice that we’ll be using HTML tags and inserting our dynamic content between these tags. We’ll also apply inline CSS styling, which will be useful when we construct the final email notification.

Once we insert our dynamic content, Power Automate will automatically create an “Apply to each” loop. This loop will iterate over each item in the JSON object, appending a new row to our table for each iteration. This approach ensures that every entry is neatly formatted and included in the final email notification.

Step 3: Send Email Notification

In our final step, we’ll send an email notification containing the content of our string variable. You’ll see that I’ve included the table tags to encapsulate our formatted rows, along with headers for each column and some additional styling for clarity.

One important detail to remember when setting up the “Send Email” action is to enable the HTML content option. This tells Power Automate that we are using our own HTML and CSS styling. By doing so, our email will properly display the structured table and styling we’ve applied, ensuring a professional and readable notification for the recipients.

By following these steps, you can ensure that your DevOps tasks are always up to date and that all team members are aware of their status, ultimately enhancing productivity and organization.

Streamlining Workflow: Comparing Objects with Inline JavaScript in Azure Functions Powered by Power Automate

  1. What Are Azure Functions?
  2. Exercise
    1. Code
    2. Full code
    3. Power Automate

The short of it is…Power Platform lives on Azure. Now this article is not an attempt to push people into Azure, but I do at least want you to consider it. If you are like me, your introduction into Azure was most likely Azure AD now known as Entra ID and next year probably something else.

Azure is a big universe that supports many different workloads. Of particular interest to me and for the benefit of this article I want to focus on Azure Functions.

What Are Azure Functions?

I don’t think there is a need to dive deep into the subject, but what I will say is, Azure Functions allow us to run serverless, on-demand code. Azure Functions support various programming languages such as C# and JavaScript. The ability to call these functions and have them execute a piece of code is beneficial to developers, especially to “low-coders”. Reason being is, we don’t have to be as seasoned as a traditional “pro-coder” we can focus only on the code that matters to us.

With the advent of Co-Pilot and ChatGpt we can write code that serves our needs. I’d hate to cause a stir so I will say this, be careful with a.i. generated code, make sure your prompt is as explicit as you can get it and if you are unsure about the piece of code you are using…ask for help.

Exercise

I once worked with a client who needed to compare two objects and identify their differences to create exceptions for their customer service representatives to address. Implementing this process in Power Automate significantly saved time, though the solution involved multiple actions, loops, and data transformations, making it less streamlined than desired. However, as I gained more expertise in Azure Functions, I discovered that the same task could be accomplished more efficiently using just a few lines of JavaScript and a single action within the flow.

Before we proceed with this exercise, it’s important to note that a foundational understanding of how to provision an Azure resource and set up your function app is essential. To ensure you’re adequately prepared, I strongly recommend reviewing the information provided in the following link: Getting started with Azure Functions | Microsoft Learn.

Keep in mind that we are utilizing JavaScript for this task, so make sure to configure your function accordingly. While you can create your function directly in the Azure portal, which is the simplest approach, there’s also the option to develop it in Visual Studio Code. This latter method is a bit more complex but certainly achievable if you’re up for the challenge. For our function app, we’ll be using an HTTP trigger as the primary mechanism to initiate the process.

Code

Keep in mind that a function app essentially serves as a container for your code snippet. In this exercise, we’ll be adding code to our function and then obtaining the URL to integrate it with Power Automate. While I utilized ChatGPT to generate my code, it’s crucial to be aware that certain parts of this code are indispensable. One such essential line of code is as follows:

module.exports = async function (context, req){}

It’s important to operate within this framework, as the context from Power Automate will be fed into this function. Once that’s in place, our code will handle the rest of the process. This setup ensures that the data from Power Automate is seamlessly integrated and processed by our Azure Function.

Full code

Simply copy and paste the entire code into your function. Pay special attention to the parameter names ‘object1’ and ‘object2’. These are the parameters through which we will pass our objects. The code will then perform the comparison and return an object highlighting the differences, which we can utilize on the Power Automate side.

module.exports = async function (context, req) {

context.log('JavaScript HTTP trigger function processed a request.');

try {
// Assuming the input is a JSON string representing the objects.
const obj1 = req.query.object1 ? JSON.parse(req.query.object1) : null;
const obj2 = req.query.object2 ? JSON.parse(req.query.object2) : null;

// Validate the input
if (typeof obj1 !== 'object' || obj1 === null || typeof obj2 !== 'object' || obj2 === null) {
context.res = {
status: 400,
body: "Please pass two JSON objects in the query string or in the request body",
headers: {
'Content-Type': 'application/json; charset=utf-8'
}
};
return;
}

// Find differences
const differences = findObjectDifferences(obj1, obj2, "root");
context.log('Differences:', differences);

// Return the differences as a JSON string with the header "differences"
context.res = {
body: { "differences": differences },
headers: {
'Content-Type': 'application/json; charset=utf-8'
}
};

} catch (error) {
context.res = {
status: 500,
body: "Error processing your request",
headers: {
'Content-Type': 'application/json; charset=utf-8'
}
};
context.log.error("Exception thrown", error);
}
};

function findObjectDifferences(obj1, obj2, path) {
let differences = [];
let allKeys = new Set([...Object.keys(obj1), ...Object.keys(obj2)]);

allKeys.forEach((key) => {
let newPath = path + "." + key;
if (obj1.hasOwnProperty(key) && !obj2.hasOwnProperty(key)) {
differences.push({ path: newPath, obj1: JSON.stringify(obj1[key]), obj2: 'Not present' });
} else if (!obj1.hasOwnProperty(key) && obj2.hasOwnProperty(key)) {
differences.push({ path: newPath, obj1: 'Not present', obj2: JSON.stringify(obj2[key]) });
} else if (obj1[key] && obj2[key] && typeof obj1[key] === 'object' && typeof obj2[key] === 'object') {
differences = differences.concat(findObjectDifferences(obj1[key], obj2[key], newPath));
} else if (obj1[key] !== obj2[key]) {
differences.push({ path: newPath, obj1: JSON.stringify(obj1[key]), obj2: JSON.stringify(obj2[key]) });
}
});

return differences;
}

After configuring your function, save it. Then, locate and click on the ‘Get function URL’ button found in the editor’s ribbon. This URL is crucial as you will need it to make the HTTP request in Power Automate.

Power Automate

  1. Set Up the Trigger: Begin by setting up your Power Automate flow with a trigger of your choice. This trigger will initiate the flow.
  2. Initialize Object Variables: Next, initialize two object variables named ‘varObject1’ and ‘varObject2’. Initially, I attempted to directly write the objects in the HTTP request but encountered issues. It’s more effective to initialize these variables and assign your arrays to them, allowing for a smoother conversion process.
  3. Create an HTTP Request:
    • Choose ‘POST’ as the method for your HTTP request.
    • In the query, you will need to add two lines:
      • The first line should have ‘object1’ as the key, with the value being the ‘varObject1’ variable.
      • The second line should have ‘object2’ as the key, with the value being the ‘varObject2’ variable.
  4. Paste the Function App URL: Paste the URL you obtained from your Azure Function app into the appropriate field in your HTTP request setup.
  5. Save and Run: After setting up everything, save your flow configuration. Then, run your flow to test and see it in action.
varObject1
varObject2
HTTP Request
successful run and differences in the body

The following is the returned exceptions (differences) in the objects. The object specifies the affected columns “root.Borrower2” and “root.Address”

{ "differences": [ { "path": "root.Borrower2", "obj1": "\"Marsurice DeVan\"", "obj2": "\"Marsurice Devan\"" }, { "path": "root.Address", "obj1": "\"Dean\"", "obj2": "\"Down\"" } ] }

Why Not Azure (or however you pronounce it)?

Alright, Azure and Power Platform. It’s easy to overlook, but remember, Power Platform is cozily nestled right into Azure. I’ve gone from saying I build apps to now, more accurately, saying I work in the cloud.

Azure is essentially the backbone of the cloud world for Microsoft 365 and Power Platform, now Azure OpenAi. Between you and me, Azure has long had Ai capabilities with Azure Cognitive Services.

Azure has replaced the need for those loud, space-eating server rooms with a sleek, digital space. I remember the days of hiding in server rooms to escape the constant barrage of IT queries. Now, with Azure, those days are just a funny memory.

So, why Azure? Well, it’s vast. When my wife expressed interest in learning it, I had to ask, “Which part?” because it’s that big. It’s not just one thing; it’s a whole ecosystem waiting to be explored.

The cost might make some hesitant. I’m not claiming to be a Microsoft licensing expert, but generally, if the benefits are clear, people find the costs justifiable. And with Azure, the benefits are often worth it.

My journey and new role have led me deeper into Azure, especially Azure Functions. These have been a game-changer in my Power App development. They allow me to execute code, like JavaScript and my new favorite PowerShell, for complex tasks I’d rather not handle directly in the app. It’s been a great push to brush up on my coding skills too. I’ve reacquainted myself with Visual Studio Code as the last 4 years or so, I’ve lived in the Power Apps formula bar and Power Automate expressions.

But it’s not just about using Azure for the sake of it. Integrating Azure into your Power Platform solutions can significantly enhance what you’re able to do. It’s about making your solutions more robust and efficient. Azure Logic Apps, for instance, are like Power Automate on steroids, offering more capabilities and flexibility.

In essence, leveraging Azure within your Power Platform solutions isn’t just a good idea; it’s a strategic move. It’s about expanding your capabilities and delivering more sophisticated solutions. So, consider taking that step into Azure. It’s a solid move for anyone looking to enhance their Power Platform projects.