Page 1 of 3
Problem with DC_Xml2ObjectTree
Posted: Fri Jul 22, 2016 9:51 am
by Piotr D
H, Roger,
I tried read an XML file into arrays. I use your DC_Xml2ObjectTree() function, these wok good with files with about 20 MB
(ca 400 000 lines). But when I use these function with bigger XML file (ca 40 MB, more than 800 000 lines), then program
terminated with XPPFATAL.LOG with error code 1006 and XPP:15. Computer was with Win7, 16GB RAM. Is it true, that
so big XML file can't be translated to object tree?
Piotr
Re: Problem with DC_Xml2ObjectTree
Posted: Fri Jul 22, 2016 10:31 am
by Auge_Ohr
Piotr D wrote:... XPPFATAL.LOG with error code 1006 and XPP:15.
1006 two many memory-objects: there are not enough handles.
15 XPP_ERR_MEMORY_FULL Not enough memory or swapping space available
try to increase STACK ... or ask Alaska for Patch to get more Handles.
Re: Problem with DC_Xml2ObjectTree
Posted: Fri Jul 22, 2016 11:43 am
by rdonnay
I was not aware of this.
Please send me your XML file.
I don't know if it is a problem with the XML parser or the DC_XmlNode() class.
Re: Problem with DC_Xml2ObjectTree
Posted: Sat Jul 23, 2016 10:15 am
by Piotr D
Roger,
Monday I send you via e-mail these XML file, XPPFATAL.LOG and source code.
Piotr
Re: Problem with DC_Xml2ObjectTree
Posted: Mon Jul 25, 2016 7:56 am
by Piotr D
Roger,
attached is XML file, error log and source code.
I was compiled with Xbase 2.0 with Express 264
Piotr
Re: Problem with DC_Xml2ObjectTree
Posted: Mon Jul 25, 2016 8:50 am
by Victorio
Hi,
I do not know function DC_Xml..., but I had similar memory problem with reading large text files, read file about 400000MB to string was not problem, but when read bigger, then memory error if I want some string operations or read part of file to array.
Because I must break reading large files to parts and process separately.
Re: Problem with DC_Xml2ObjectTree
Posted: Wed Jul 27, 2016 12:37 pm
by rdonnay
Piotr -
In my opinion, you are using XML for a purpose it was not intended, even though it does not invalidate any XML specifications.
Your XML file is 44,233,975 bytes. It includes 855,798 nodes.
Using DC_Xml2ObjectTree() would require building 855,798 separate DC_XmlNode() objects in a single tree.
This is totally impractical for many reasons.
It appears that you have converted a database of about 53,430 records to an XML document.
What do you intend to do with this data?
For what purpose?
Xbase++ must allocate far too much memory to be able to handle this. It locked up my sample program.
When I tried to load the XML into Internet Explorer it also locked up Internet Explorer.
Apparently, you are trying to find a mechanism to send a database via the internet.
Rather than converting to XML, it is probably best that you use Base64 encoding to attach the entire database to an XML document.
Re: Problem with DC_Xml2ObjectTree
Posted: Wed Jul 27, 2016 1:45 pm
by rdonnay
Here is an example of embedding a database into an XML document.
Here are the steps:
1. Load the database contents into a binary string.
2. Convert the string to a Base64 encoded string.
3. Build the XML with the encoded string.
4. Write the XML to a file.
5. Load the XML into an object tree using DC_Xml2ObjectTree()
6. Find the node containing the encoded data.
7. Convert the BASE64 encoded data back to a binary string.
8. Write the binary string to a new database.
9. Open the new database.
Code: Select all
#INCLUDE "dcdialog.CH"
#INCLUDE "fileio.CH"
#Pragma Library("dcxml.lib")
#DEFINE CRLF Chr(13)+Chr(10)
FUNCTION Main()
LOCAL cString, nHandle, nSize, cEncodedString, cXml, oRoot, ;
oDataNode, cDatabaseName, oNode
nHandle := FOpen('books.dbf')
nSize := FSeek( nHandle, 0, FS_END )
cString := Space(nSize)
FSeek( nHandle, 0, FS_SET )
FRead( nHandle, @cString, nSize )
cEncodedString := Bin2Base64(cString)
FClose( nHandle )
nHandle := FCreate('BOOKS.XML')
cXml := '<?xml version="1.0" encoding="UTF-8"?>' + CRLF
cXml += '<Database Name="Books2.dbf">' + CRLF
cXml += ' <MimeType>application/dbf</MimeType>'
cXml += ' <EncodedData>' + CRLF
cXml += cEncodedString + CRLF
cXml += ' </EncodedData>' + CRLF
cXml += '</Database>'
Fwrite( nHandle, cXml )
FClose( nHandle )
oRoot := DC_Xml2ObjectTree('BOOKS.XML')
oNode := oRoot:findNode('Database',,.t.)
cDatabaseName := oNode:getAttr('Name')
oDataNode := oRoot:findNode('EncodedData',,.t.)
cEncodedString := oDataNode:content
cString := Base642Bin(cEncodedString)
nHandle := FCreate(cDatabaseName)
FWrite( nHandle, cString )
FClose(nHandle)
USE (cDatabaseName)
DC_Dot()
RETURN nil
* ---------
PROC appsys ; RETURN
Re: Problem with DC_Xml2ObjectTree
Posted: Wed Jul 27, 2016 1:54 pm
by Auge_Ohr
hi
rdonnay wrote:2. Convert the string to a Base64 encoded string.
...
7. Convert the BASE64 encoded data back to a binary string.
Code: Select all
#INCLUDE "dcdialog.CH"
#INCLUDE "fileio.CH"
#Pragma Library("dcxml.lib")
...
cEncodedString := Bin2Base64(cString)
...
cString := Base642Bin(cEncodedString)
where do i find both function ?
Re: Problem with DC_Xml2ObjectTree
Posted: Wed Jul 27, 2016 2:06 pm
by rdonnay
where do i find both function ?
These functions are part of the Foundation Subscription of Xbase++ 2.0.
They are in the XPPRT1.DLL.