Robotic Process Automation in Microsoft Power Platform. Document recognition

Hi all! It's no secret that artificial intelligence is now increasingly involved in various areas of our lives. We are trying to shift more and more routine tasks and operations to virtual assistants, thereby freeing up our time and energy to solve truly complex and often creative tasks. None of us likes to do the same work day in and day out, so the idea of ​​giving such tasks to artificial intelligence is perceived with great positiveness.

Robotic Process Automation in Microsoft Power Platform. Document recognition

So what exactly is Robotic Process Automation?

RPA or Robotic Process Automation is a technology that today allows computer software or "robots" to be configured to emulate the actions of humans working in digital systems in order to execute business processes. RPA robots use a user interface to collect data and use applications just like humans do. They interpret, initiate responses, and communicate with other systems to perform a wide variety of repetitive tasks. The only difference is that the RPA software robot never rests or makes mistakes. Well, almost does not.

For example, the RPA robot can process files attached to letters, recognize text, amounts, last names, after which the information received will automatically be entered in any accounting system. In fact, RPA robots are capable of imitating many, if not all, user actions. They can enter applications, move files and folders, copy and paste data, fill out forms, extract structured and semi-structured data from documents, and more.

RPA technology has not bypassed Microsoft Power Automate, which is well known to us. In past articles, I've covered how you can use Power Automate to automate a variety of processes, from posting messages in Microsoft Teams to managing approval to sending HTTP web requests. We've covered many scenarios that you can implement using Power Automate features. Today, let's see how to use RPA. Let's not waste time.

Let's try to "robotize" the demo process of submitting a ticket to the support service. The initial data is as follows - the client sends information about an error or wish by mail in the form of a PDF document with a table containing information about the appeal. The table format will be as follows:

Robotic Process Automation in Microsoft Power Platform. Document recognition

Now we go to the Power Automate portal and create a new artificial intelligence model:

Robotic Process Automation in Microsoft Power Platform. Document recognition

Next, specify the name for our future model:

Robotic Process Automation in Microsoft Power Platform. Document recognition

Power Automate warns us that it will take about 5 documents with the same layout to create a model in order to train our future "robot". Fortunately, there are more than enough of these templates available.

We load 5 document templates and start preparing the model:

Robotic Process Automation in Microsoft Power Platform. Document recognition

It takes a few minutes to prepare an artificial intelligence model, it's time to pour yourself some tea:

Robotic Process Automation in Microsoft Power Platform. Document recognition

After the preparation of the model is completed, it is necessary to assign certain labels to the recognized text, by which it will be possible to access the information:

Robotic Process Automation in Microsoft Power Platform. Document recognition

Bindings of labels and data are saved in a separate window. After you have tagged all the required fields, click "Confirm fields":

Robotic Process Automation in Microsoft Power Platform. Document recognition

In my case, the model asked me to tag the fields on a couple more document templates. I kindly agreed to help:

Robotic Process Automation in Microsoft Power Platform. Document recognition

After all the operations are done, it's time to start training the model, the button for which for some reason is called "Train". Let's go!

Robotic Process Automation in Microsoft Power Platform. Document recognition

Training the model, as well as its preparation, takes a few minutes, it's time to pour yourself another cup of tea. After the training is completed, you can publish the created and trained model:

Robotic Process Automation in Microsoft Power Platform. Document recognition

The model is trained and eager to work. Now let's create a SharePoint Online list in which we will add data from recognized PDF documents:

Robotic Process Automation in Microsoft Power Platform. Document recognition

And now, when everything is ready, we create a Power Automate flow, with a trigger "When a new email message arrives", recognize an attachment in an email and create an item in a SharePoint list. Flow example below:

Robotic Process Automation in Microsoft Power Platform. Document recognition

Let's check our flow. We send ourselves a letter with an attachment of the form:

Robotic Process Automation in Microsoft Power Platform. Document recognition

And the result of the flow execution is the automatic creation of an entry in the SharePoint Online list:

Robotic Process Automation in Microsoft Power Platform. Document recognition

Everything works like clockwork. Now about the nuances.

The first caveat - at the moment, RPA in Power Automate cannot recognize Russian text. It is likely that in the near future such an opportunity will be brought up, but now it is not there yet. So this aspect needs to be taken into account.

The second caveat is that using Robotic Process Automation in the Power Platform requires a Premium subscription. More specifically, RPA is licensed as an add-on to a PowerApps or Power Automate license. In turn, the use of RPA in Power Automate requires a connection to the Common Data Service environment, which is included in the Premium subscription.

In the following articles, we will look at even more opportunities for using RPA in the Power Platform and find out how you can make a smart chatbot based on Power Automate and RPA. Thank you for your attention and have a nice day everyone!

Source: habr.com