How to handle reading in large amount of bytes (60k) for parsing multiple data items from?

I am finding that I can successfully read my MTConnect response into memory as an array[0..59999], but I cannot convert over 32k to a string for parsing with. What is the best practice for reading that many bytes and needing to parse it into several data items? Do I write it to a local file path instead? the MTConnect response is a XML by nature. Are there any libraries I should be aware of for easily parsing an XML instead of doing it manually with individual functions?

I’ve never worked with the library directly myself, but have you tried XML_Utility_SL? It provides for both directly reading an XML file and reading from a Data Array.

Edit to add: This library is included in the IIoT Libraries SL that can be downloaded from Codesys (not a WAGO library). The license for the IIoT Library suite is included in WAGO controllers so you can just download it, no need to purchase a license.

Is there an easy way to update a library? I have 1.9.0.0 and it comes with zero documentation

I suggest downloading the latest package from the Codesys store. It will install the most recent library versions to your repository.

hello,
I made some XML functions to get keys
ParseXML.project (247.5 KB)

I use to search for strings in strings, but here is version for you where you can search strings in byte arrays.
I’m little bit lazy, so I use WagoAppString, but its easy to use :wink:

I have essentially arrived at the same approach. I am definitely finding myself struggling with the nuances of ST coming from a heavy python background.

I have copied the bytes to a Global Variable (array[0..59999] of byte). From there I am using a DataExtraction FB and Method to use inputs defining the start and end tag of the data I want to extract (keeping it dynamic). Then I iterate through the bytes looking for the matching byte pattern of the start tag. Once that is found, I start and the end of the found start tag and look for the matching byte pattern for the end tag. Everything in between is the data I want extracted (DataArray).

This is where I am beating my head against the wall. Trying to convert that DataArray to a string inside the FB is crashing on every approach (Mem_to_PrintableString, MemCpy (to another Global Variable), iterating through each byte and using Char_to_String). The crash always occurs on these lines. I have validated that the inputs are all correct and should produce a string.

Any idea what might be causing this to crash?

UPDATE: I am just inexperience with debugging this language. I was using breakpoints and stepping into the code to debug. The second I ran the code without stopping it, it works as intended. Now I am dealing with performance.

I am seeing a lag in the final destination being updated. I have the MTC get request running in one PRG in a cyclic task @200ms. Then I have the parsing PRG running in another cyclic task @200ms. The first one write the global array of bytes, and the second one uses the global array of bytes to parse and write the final global variable value.

That’s exactly how this function works, exactly as you described—it iterates through the characters, looking for a pattern, then searching for the end tag and returning the entire contents of the string. I’ve limited the function to returning STRING(1024), but you could just as easily return another byte array, or even not process it at all. You can take udiPositionClosure and udiPositionEnd as output to get start and end of the data in the table and use this to delve deeper.

To get STRING from byte array you can just use WagoSysPlainMem.MemCopy. Inside FuXMLGetSingleKeyValue I use this function to copy data from inside tag to string, but you can also copy data to another table or whatever you like :wink:

I send it open so you can fit this function to your needs