Configuring the BizTalk application
This is the final step of this process. In order to properly test the solution we have been developing in BizTalk Server, we need to configure the application that was created in the publication.
For this, we will access the BizTalk Server Administration console to create and configure the following artifacts:
- One receive port and location to process the input text files;
- One send port to save the processed files in XML format…
Open the BizTalk Server Administration console under “Start –> Programs –> Microsoft BizTalk Server 2010”. On the left tree menu, maximize “BizTalk Server Administration –> BizTalk Group (…) –> Applications”, and look for the application called “TransfFlatFilesEmXML Demo” and maximize also this application.
- Press the right button on top of the “Receive Port” menu and select “New –> One-Way Receive Port …” option;
- A new window appears allowing us to define the port properties:
- In the “General” tab, set the name of the receive port: “ReceiveTXT”.
- In the “Receive Locations” tab, select “New” and set the name to your receive location: “ReceiveTXT_to_XML”; in the property “Type” select the option FILE; and in the property “Receive pipeline” select from the drop-down box the pipeline we created earlier: “ReceivePipelineCustomPessoas”;
- On the same tab select the “Configure” button. In the “FILE Transport Properties” window, set “Receive Folder” property, with the folder you created earlier “<solution>\PORTS\IN”. On property “File Mask” put the following value: “*. txt”, setting that will be processed only files with the extension. “txt”. Finally, select the “Ok” button.
- To finish the process of creating the receive port press “Ok”.
The above steps mentioned were needed to create one Receive Port and its Receive Location. Now we will create a Send Port to save messages on the file system after they are processed by BizTalk:
- Press the right button on top of the “Send Ports” menu and select “New –> Static One-way Send Port…”
- A new window appears allowing us to define the port properties:
- In the “General” tab, set the name of the send port: ” SaveXMLFile “
- In the property “Type” select the option FILE;
- In the property “Send Pipeline” set the pipeline to “XMLTransmit”. This is a native pipeline that is used for processing XML messages.
- On the same tab select the “Configure” button. In the “FILE Transport Properties” window, set the “Destination Folder” property, with the folder you created earlier “<solution>\PORTS\OUT”. On property “File Mask” put the following value: “%MessageID%.xml”, this is a special tag (BizTalk Macro) that will set the name of each file written on the file system with the unique identifier of the message. Finally, select the “Ok” button to return to the previous window.
- In order for the Send Port subscribes to all files, we need to set a filter on the tab “Filters” with the following expression: “BTS.ReceivePortName == ReceiveTXT”.
- Note: This setting will force each message in the MessageBox which has originated from the Receive Port: ReceiveTXT, be routed and processed by the Send Port we are finished creating.
- To finish the process of creating the send port press “Ok”.
Finally, we need only to start our application. To do that press right button on top of the application name “TransfFlatFilesEmXML Demo” and select “Start …”.
Testing the solution
To test the BizTalk application we only need to copy a text file to the directory that is configured on the receive location: “<solution>\PORTS\IN”. The result should appear in the form of an XML document in the folder configured in the send port: “<solution>\PORTS\OUT”.
Note: The file disappears automatically from the input folder, so you should avoid doing drag & drop (move) with the test files at the risk of not being able to use it again.
Conclusion
As presented in this article, BizTalk Server helps us to solve many of the existing problems concerning message transformation: semantic and syntax Transformations, only with “out of the box” features within the product, in few steps and most of the time without developing any code.
The fact of all the definitions of the rules of parsing (transformation rules) are embedded in the form of annotations in XML Schema (XSD), such as the delimiter symbols, or the element size in a positional file, simplifies the reuse of all these schemes in different parts of the process. At any point, the document can be translated back into flat-file because the definition is declarative and symmetric, in contrast to what might happen with a “normal” programming language like C#, where we would have to create two components to translate text to XML and vice versa.
The model used by BizTalk schemas, centralizes all the message definitions in a single artifact, simplifying the maintenance and evolution of the specification and therefore simplifying future reuse and dependencies management. When we set up this type of project, is exactly this kind of guarantees that keeps the systems operating properly for many years, and at the same time, they make life easier to those who one day have to correct and/or develop any of these processes
Related links
- BizTalk Server: Transform text files (Flat Files) into XML – Introduction (Part 1)
- BizTalk Server: Transform text files (Flat Files) into XML – Necessary tools and artifacts (Part 2)
- BizTalk Server: Transform text files (Flat Files) into XML – A practical example (Part 3)
Source code
You can download the source code from:
Transform text files (Flat Files) into XML
GitHub
Guide very detailed and precise.
One question: if i need to work with file.csv out of Biztalk orchestration, the conversion is better to do to inside Biztalk or is better to convert outside?
I think i leave all in Biztalk….Wrong?
Hi Mauro,
I’m not sure I quite understand your question…
However be noted that within BizTalk orchestrations you are not working with flat file format (file.csv) but with the representation of that file in XML format, the conversion of these formats should always be carried in pipelines.
If you are you asking if these conversions must be made in BizTalk or in external systems, it depends on the architecture of the systems, and what is the cost that such operation represent or even if it is possible to change the format which these systems can communicate.
Clear that if we can remove unnecessary transformations we will be improving the solution performance, but without reason in contrary, I will perform all these transformations in BizTalk.
Hi Sandro and sorry for the ambiguity of my question! 🙂
I know that Biztalk works only with xml files and my doubt is about this thing:
the transformation is due to another external system that receive only csv files (that must be the output of my BT application).
I think that this operation cannot be improved the performance, so following your idea, i try to perform these transformations in Biztalk…How can i do this?
My idea was to work with maps, but how?
Otherwise, can i personalize a receive pipeline that makes this transformations?
Many Thanks
Hi Mauro,
What you must do is:
1- get a complete sample with the correct syntax of the CSV file that has to be consumed by the external system
2- and use this file to create the Flat File Schema (as described in Part 3 of this guide) and of course in the end you should test that the Flat File Schema generated can correctly “translated” the CSV in its XML representation
3- Create a Map that transform you input schema into the flat file schema generated.
4- And use a send pipeline to “translate” the XML message to CSV format.
and of course this also depends on the type of structure of CSV file (structure format), sometimes is not possible to accomplish this with this strategy, usually with complex flat files, we must develop a pipeline component for dealing with this transformation. But I believe that the majority of the scenarios can be implemented using the flat file schema and a pipeline without any custom pipeline component.
Thanks Sandro, really clear…i try to do this and then i’ll post my solution!
Hi Sandro,
I have another question: in a BizTalk project (for example called General) i want to realize a sort of General Catalog within all the records that frequently i can use in my process (made as Data Structure Type), so that when i add other many projects (PROJECT A, PROJECT B…) i can recall the same records made in General without generate again them. This is possible or is completely wrong? And if yes, how can i recall my Data Structure Type?
Sorry for my English, but my idea is a little bit complicated to explain :).
Thanks
Hi,
If I understand the question well, you want to create a “general project” with all common Data Structure Type like Phone (CountryCode, AreaCode, Number and Extension) and then reuse this structure in many projects.
This is possible. Example scenario:
Inside Visual Studio you need to:
1. You need to create a “General Structure Project” with these schemas
2. In your Visual Studio BizTalk Project A you need to
2.1. Import the DLL of this “General Structure Project”.
2.2. In your schema, select imports -> add XSD import and from the BizTalk Type picker select References->you DLL General Structure Project ->you schema with common Data Structure Types
2.3. Then all the Data structure type which exist in your schema will be available
3. Publish “General Structure Project” to a common BizTalk Application, let’s call “BizTalk Common Application”
4. And Publish Project A, let’s call “BizTalk Application A”
In BizTalk Administration Console you need to:
1. Go to “BizTalk Application A” properties and add the reference to “General Common Application”
Sandro,
Do you think you can do a tutorial or give me guidance on using the Flat File Wizard to model complex data that meets this type of requirement.
1. The tag identifier for each data line is not at the position 0 but 8.(Off Set Tag)
2. Each type of data line can appear arbitrary number of time and in arbitrary order.(Random Ordering of Data)
Hi Jordan,
Can you send me one sample of the flat file? and i will try to do the tutorial
Sent, Thanks in Advance
Hi,
I drag and drop my txt file to IN folder but OUT folder always be empty.
Do you help me?
Hi,
I want to process an inhouse EDI flat file containing more than one interchange and a interchange can contain more than one invoice. The invoice has a head record and a number of line records. when i used the flat file wizzard I can not get the XSD schema so that the structure of the XML messages is correct. Can you help me to fix the schema?