==================
Parsing XML & JSON
==================
*Both XML and JSON are common formats to transfer data between systems in a human readable format. This sections aims to introduce all options to handle/parse xml & json formats.*
Conversion Between XML & JSON
-----------------------------
Convert XML to JSON
===================
::
CONVERTXMLJSON( {input}, 'JSON');
.. code-block:: xml
::
OBJECT:
STRING STRJSON,
STRING STRINGVAR3;
STRJSON = CONVERTXMLJSON(STRINGVAR3, 'JSON');
STRINGVAR3 = STRJSON;
.. code-block:: json
{
"@id": "file",
"@value": "File",
"popup": [
{
"@value": "New",
"@onclick": "CreateNewDoc()"
},
{
"@value": "Open",
"@onclick": "OpenDoc()"
},
{
"@value": "Close",
"@onclick": "CloseDoc()"
}
]
}
Convert JSON to XML
===================
::
CONVERTXMLJSON( {input}, 'XML'[, 'NOTYPEHINT' | 'JSONPREFIX'] );
.. code-block:: json
{"menu": {
"id": "file",
"value": "File",
"popup": {
"menuitem": [
{"value": "New", "onclick": "CreateNewDoc()"},
{"value": "Open", "onclick": "OpenDoc()"},
{"value": "Close", "onclick": "CloseDoc()"}
]
}
}}
::
OBJECT:
STRING STRXML,
STRING STRINGVAR3;
STRXML = CONVERTXMLJSON(STRINGVAR3,'XML','NOTYPEHINT');
STRINGVAR3 = STRXML;
.. code-block:: xml
Reading XML Structure
---------------------
::
READXMLSTRUCTURE {xmlcontent} TO {targettable};
+-------------+---------------------------------+
| ID | |
+-------------+---------------------------------+
| PID | |
+-------------+---------------------------------+
| NAME | |
+-------------+---------------------------------+
| PATH | |
+-------------+---------------------------------+
| VALUE | |
+-------------+---------------------------------+
| LEVEL | |
+-------------+---------------------------------+
| ISATTRIBUTE | |
+-------------+---------------------------------+
::
10.10.2015
btan
Tokyo
Payment Systems
Madrid
Mobile Applications
::
READXMLSTRUCTURE STRINGVAR3 TO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
Some Useful XML Functions
-------------------------
Formatting XML Documents
========================
::
OBJECT:
STRING STRXML,
STRING STRINGVAR3;
STRXML = 'Team 10';
STRXML = STRXML + 'Team 2';
STRXML = STRXML + '1';
STRINGVAR3 = WRAPXML(STRXML);
::
Team 1
0
Team 2
1
CLEARDOCUMENT() Function
========================
::
OBJECT:
STRING STRXML,
STRING STRINGVAR3;
STRXML = 'Team 10';
STRXML = STRXML + 'Team 2';
STRXML = STRXML + '1';
STRINGVAR3 = CLEARDOCUMENT(STRXML,'TEAM');
Validating XML Documents
------------------------
::
VALIDATEXML {validatorpath} {xmlpath} WITH {valtype} [TO {details}];
VALIDATEXML TEXT {validatortext} {xmlastext} WITH {valtype} [TO {details}];
.. code-block:: xml
Person must have title
Person should have Name / Gender
Name must appear before element Gender
If the Title is "Mr", gender must be "Male"
.. code-block:: xml
Eddie
Male
::
OBJECT:
STRING STRXMLPATH,
STRING STRSCHPATH
STRING STRINGVAR3;
STRXMLPATH = 'xml.xml';
STRSCHPATH = 'validator.sch';
COPYFILE '*C:\TMP\xml.xml' INTO STRXMLPATH;
COPYFILE '*C:\TMP\validator.sch' INTO STRSCHPATH;
VALIDATEXML STRSCHPATH STRXMLPATH WITH SCHEMATRON TO STRINGVAR3;
STRINGVAR1 = SYS_STATUS + ' ' + SYS_STATUSERROR;
Parsing XML Documents
---------------------
Defining XML Maps
=================
::
CREATEXMLMAP {mapname};
::
CLEAR XMLMAP {mapname};
::
MAP ELEMENT {elementname} AS [XMLTABLE | XMLROOTTABLE] {tablename} IN {mapname};
MAP ATTRIBUTE {attributename} OF {elementname}
AS XMLCOLUMN {columnname} IN {mapname};
MAP CHILD {elementname} OF {elementname} AS XMLCOLUMN {columnname} IN {mapname};
MAP PCDATA OF {elementname} AS XMLCOLUMN {columnname} IN {mapname};
MAP RELATION {elementname} TO {elementname} LINK {primarykey}
WITH {foreignkey} GENERATE YES | NO IN {mapname};
Parsing XML with XML Maps
=========================
::
PARSEXML {docpath} USING {mapname};
::
CONVERTXMLTABLE {xmltable} INTO {table} FROM {mapname};
Parsing without XML Map
=======================
::
PARSEXML TEXT {xmlastext} INTO {tablename};
PARSEXML {filepath} INTO {tablename};
XML Parsing Examples
--------------------
Example 1: Using Auto Parser
============================
.. code-block:: xml
OBJECT:
STRING STRINGVAR3;
STRINGVAR3 = '
IAS Software
Turkey
XYZ Technology
Turkey
ABC Design
Turkey
';
OBJECT:
TABLE CUSTOMERS;
PARSEXML TEXT STRINGVAR3 INTO CUSTOMERS;
COPY TABLE CUSTOMERS INTO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
Example 2: Multiple Auto Parsers
================================
.. code-block:: xml
OBJECT:
STRING STRINGVAR3;
STRINGVAR3 = '
100
Invalid username or password
false
';
PARSEXML TEXT STRINGVAR3 INTO TMPTABLE;
PARSEXML TEXT TMPTABLE_XMLData INTO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
Example 3: Using PCData
=======================
.. code-block:: xml
IAS Software
XYZ Technology
ABC Design
::
OBJECT:
TABLE CUSTOMERS;
CLEAR XMLMAP DEMOMAP;
CREATEXMLMAP DEMOMAP;
MAP ELEMENT CustomerList AS XMLROOTTABLE CustomerList IN DEMOMAP;
MAP ELEMENT Customer AS XMLTABLE Customer IN DEMOMAP;
MAP PCDATA OF Customer AS XMLCOLUMN Customer IN DEMOMAP;
MAP RELATION Customer TO CustomerList LINK 'asdf'
WITH DUMMY GENERATE NO IN DEMOMAP;
COPYFILE '*C:\TMP\XML\ex3.xml' INTO '.\XMLParsing\ex3.xml';
PARSEXML '.\XMLParsing\ex3.xml' USING DEMOMAP;
CONVERTXMLTABLE Customer INTO TABLE CUSTOMERS FROM DEMOMAP;
COPY TABLE CUSTOMERS INTO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
Example 4: Reading Childs
=========================
.. code-block:: xml
Technology
XYZ Industries
Tokyo
ABC Technology
Madrid
IAS Software
Istanbul
::
OBJECT:
TABLE CUSTOMERS,
INTEGER ROWCOUNT;
CLEAR XMLMAP DEMOMAP;
CREATEXMLMAP DEMOMAP;
MAP ELEMENT CustomerList AS XMLROOTTABLE CustomerList IN DEMOMAP;
MAP CHILD Sector OF CustomerList AS XMLCOLUMN Sector IN DEMOMAP;
MAP ELEMENT Customer AS XMLTABLE CUSTOMER IN DEMOMAP;
MAP CHILD CustName OF Customer AS XMLCOLUMN CustName IN DEMOMAP;
MAP CHILD City OF Customer AS XMLCOLUMN City IN DEMOMAP;
MAP RELATION Customer TO CustomerList LINK Sector
WITH Sector GENERATE YES IN DEMOMAP;
COPYFILE '*C:\TMP\XML\ex4.xml' INTO '.\XMLParsing\ex4.xml';
PARSEXML '.\XMLParsing\ex4.xml' USING DEMOMAP;
CONVERTXMLTABLE CUSTOMER INTO TABLE CUSTOMERS FROM DEMOMAP;
ROWCOUNT = CUSTOMERS_ROWCOUNT;
COPY TABLE CUSTOMERS INTO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
Example 5: Childs with Extra Relation
=====================================
.. code-block:: xml
100
Invalid username or password
false
::
OBJECT:
TABLE DETAILRESULT;
CLEAR XMLMAP DEMOMAP;
CREATEXMLMAP DEMOMAP;
MAP ELEMENT PCARD_CARD_GET_DETAILResponse
AS XMLROOTTABLE PCARD_CARD_GET_DETAILResponse IN DEMOMAP;
MAP ELEMENT PCARD_CARD_GET_DETAILResult
AS XMLTABLE PCARD_CARD_GET_DETAILResult IN DEMOMAP;
MAP CHILD ConditionNo OF PCARD_CARD_GET_DETAILResult
AS XMLCOLUMN ConditionNo IN DEMOMAP;
MAP CHILD ConditionMessage OF PCARD_CARD_GET_DETAILResult
AS XMLCOLUMN ConditionMesaj IN DEMOMAP;
MAP CHILD Success OF PCARD_CARD_GET_DETAILResult
AS XMLCOLUMN Success IN DEMOMAP;
MAP CHILD ConditionNo OF PCARD_CARD_GET_DETAILResult
AS XMLCOLUMN ConditionNo IN DEMOMAP;
MAP RELATION PCARD_CARD_GET_DETAILResult TO PCARD_CARD_GET_DETAILResponse LINK
DUMMY WITH DUMMY GENERATE NO IN DEMOMAP;
COPYFILE '*C:\TMP\XML\ex5.xml' INTO '.\XMLParsing\ex5.xml';
PARSEXML '.\XMLParsing\ex5.xml' USING DEMOMAP;
CONVERTXMLTABLE PCARD_CARD_GET_DETAILResult
INTO TABLE DETAILRESULT FROM DEMOMAP;
COPY TABLE DETAILRESULT INTO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
Example 6: Reading Attributes
=============================
.. code-block:: xml
XYZ Industries
Tokyo
ABC Technology
Madrid
IAS Software
Istanbul
::
OBJECT:
TABLE CUSTOMERS,
INTEGER ROWCOUNT;
CLEAR XMLMAP DEMOMAP;
CREATEXMLMAP DEMOMAP;
MAP ELEMENT CustomerList AS XMLROOTTABLE CustomerList IN DEMOMAP;
/* map attribute for root element*/
MAP ATTRIBUTE Sector OF CustomerList AS XMLCOLUMN Sector IN DEMOMAP;
MAP ELEMENT Customer AS XMLTABLE CUSTOMER IN DEMOMAP;
MAP CHILD CustName OF Customer AS XMLCOLUMN CustName IN DEMOMAP;
MAP CHILD City OF Customer AS XMLCOLUMN City IN DEMOMAP;
MAP RELATION Customer TO CustomerList
LINK Sector WITH Sector GENERATE YES IN DEMOMAP;
COPYFILE '*C:\TMP\XML\ex6.xml' INTO '.\XMLParsing\ex6.xml';
PARSEXML '.\XMLParsing\ex6.xml' USING DEMOMAP;
CONVERTXMLTABLE CUSTOMER INTO TABLE CUSTOMERS FROM DEMOMAP;
ROWCOUNT = CUSTOMERS_ROWCOUNT;
COPY TABLE CUSTOMERS INTO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
Example 7: Attributes On Childs
===============================
.. code-block:: xml
XYZ Industries
Tokyo
ABC Technology
Madrid
IAS Software
Istanbul
::
OBJECT:
TABLE CUSTOMERS,
INTEGER ROWCOUNT;
CLEAR XMLMAP DEMOMAP;
CREATEXMLMAP DEMOMAP;
MAP ELEMENT CustomerList AS XMLROOTTABLE CustomerList IN DEMOMAP;
MAP ATTRIBUTE Sector OF CustomerList AS XMLCOLUMN Sector IN DEMOMAP;
MAP ELEMENT Customer AS XMLTABLE CUSTOMER IN DEMOMAP;
/* attribute for child */
MAP ATTRIBUTE CustomerId OF Customer AS XMLCOLUMN CustomerId IN DEMOMAP;
MAP CHILD CustName OF Customer AS XMLCOLUMN CustName IN DEMOMAP;
MAP CHILD City OF Customer AS XMLCOLUMN City IN DEMOMAP;
MAP RELATION Customer TO CustomerList LINK Sector
WITH Sector GENERATE YES IN DEMOMAP;
COPYFILE '*C:\TMP\XML\ex7.xml' INTO '.\XMLParsing\ex7.xml';
PARSEXML '.\XMLParsing\ex7.xml' USING DEMOMAP;
CONVERTXMLTABLE CUSTOMER INTO TABLE CUSTOMERS FROM DEMOMAP;
ROWCOUNT = CUSTOMERS_ROWCOUNT;
COPY TABLE CUSTOMERS INTO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
Example 8: Multiple Tables & Relations Childs-Attributes
========================================================
.. code-block:: xml
XYZ Industries
Tokyo
2013
100.013
ABC Technology
Madrid
2011
100.011
2012
100.012
IAS Software
Istanbul
2010
100.010
2014
100.014
::
OBJECT:
TABLE CUSTOMERS,
TABLE ORDERS,
INTEGER ROWCOUNT;
CLEAR XMLMAP DEMOMAP;
CREATEXMLMAP DEMOMAP;
MAP ELEMENT CustomerList AS XMLROOTTABLE CustomerList IN DEMOMAP;
MAP ATTRIBUTE Sector OF CustomerList AS XMLCOLUMN Sector IN DEMOMAP;
MAP ELEMENT Customer AS XMLTABLE CUSTOMER IN DEMOMAP;
MAP ATTRIBUTE CustomerId OF Customer AS XMLCOLUMN CustomerId IN DEMOMAP;
MAP CHILD CustName OF Customer AS XMLCOLUMN CustName IN DEMOMAP;
MAP CHILD City OF Customer AS XMLCOLUMN City IN DEMOMAP;
MAP RELATION Customer TO CustomerList LINK Sector
WITH Sector GENERATE YES IN DEMOMAP;
MAP ELEMENT Order AS XMLTABLE ORDER IN DEMOMAP;
MAP ATTRIBUTE OrderId OF Order AS XMLCOLUMN OrderId IN DEMOMAP;
MAP CHILD Year OF Order AS XMLCOLUMN Year IN DEMOMAP;
MAP CHILD GrandTotal OF Order AS XMLCOLUMN GrandTotal IN DEMOMAP;
MAP RELATION Order TO Customer LINK CustomerId
WITH CustomerId GENERATE NO IN DEMOMAP;
COPYFILE '*C:\TMP\XML\ex8.xml' INTO '.\XMLParsing\ex8.xml';
PARSEXML '.\XMLParsing\ex8.xml' USING DEMOMAP;
CONVERTXMLTABLE CUSTOMER INTO TABLE CUSTOMERS FROM DEMOMAP;
CONVERTXMLTABLE ORDER INTO TABLE ORDERS FROM DEMOMAP;
COPY TABLE CUSTOMERS INTO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
RETURN;
COPY TABLE ORDERS INTO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
Example 9: Multiple Relations/Extra Orders Column Without OrdersId
==================================================================
.. code-block:: xml
XYZ Industries
Tokyo
2013
100.013
ABC Technology
Madrid
2011
100.011
2012
100.012
IAS Software
Istanbul
2010
100.010
2014
100.014
2015
100.015
::
OBJECT:
TABLE CUSTOMERS,
TABLE ALLORDERS,
TABLE CUSTOMERORDERS;
CLEAR XMLMAP DEMOMAP;
CREATEXMLMAP DEMOMAP;
MAP ELEMENT CustomerList AS XMLROOTTABLE CustomerList IN DEMOMAP;
MAP ATTRIBUTE Sector OF CustomerList AS XMLCOLUMN Sector IN DEMOMAP;
MAP ELEMENT Customer AS XMLTABLE CUSTOMER IN DEMOMAP;
MAP ATTRIBUTE CustomerId OF Customer AS XMLCOLUMN CustomerId IN DEMOMAP;
MAP CHILD CustName OF Customer AS XMLCOLUMN CustName IN DEMOMAP;
MAP CHILD City OF Customer AS XMLCOLUMN City IN DEMOMAP;
MAP RELATION Customer TO CustomerList LINK Sector
WITH Sector GENERATE YES IN DEMOMAP;
MAP ELEMENT Orders AS XMLTABLE CUSTOMERORDER IN DEMOMAP;
MAP RELATION Orders TO Customer LINK CustomerId
WITH CustomerId GENERATE YES IN DEMOMAP;
MAP ELEMENT Order AS XMLTABLE ORDER IN DEMOMAP;
MAP ATTRIBUTE OrderId OF Order AS XMLCOLUMN OrderId IN DEMOMAP;
MAP CHILD Year OF Order AS XMLCOLUMN Year IN DEMOMAP;
MAP CHILD GrandTotal OF Order AS XMLCOLUMN GrandTotal IN DEMOMAP;
MAP RELATION Order TO Orders LINK CustomerFK
WITH CustomerFK GENERATE YES IN DEMOMAP;
COPYFILE '*C:\TMP\XML\ex9.xml' INTO '.\XMLParsing\ex9.xml';
PARSEXML '.\XMLParsing\ex9.xml' USING DEMOMAP;
CONVERTXMLTABLE CUSTOMER INTO TABLE CUSTOMERS FROM DEMOMAP;
CONVERTXMLTABLE CUSTOMERORDER INTO TABLE CUSTOMERORDERS FROM DEMOMAP;
CONVERTXMLTABLE ORDER INTO TABLE ALLORDERS FROM DEMOMAP;
COPY TABLE CUSTOMERS INTO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
COPY TABLE ALLORDERS INTO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
COPY TABLE CUSTOMERORDERS INTO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
LOOP AT ALLORDERS
BEGIN
LOCATERECORD BINARYSEARCH COLUMNS CustomerFK VALUES
ALLORDERS_CustomerFK ON CUSTOMERORDERS;
STRINGVAR2 = STRINGVAR2 + ' ' + CUSTOMERORDERS_CustomerId;
ALLORDERS_CustomerFK = CUSTOMERORDERS_CustomerId;
ENDLOOP;
COPY TABLE ALLORDERS INTO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
COPY TABLE CUSTOMERORDERS INTO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
Example 10: Relation with Non-Generated Column
==============================================
.. code-block:: xml
ORDERORG1
ORDERORG1
ORDERORG1
::
OBJECT:
TABLE ORDERLINES,
TABLE ORDERS;
CLEAR XMLMAP DEMOMAP;
CREATEXMLMAP DEMOMAP;
MAP ELEMENT string AS XMLROOTTABLE string IN DEMOMAP;
MAP ATTRIBUTE xmlns OF string AS XMLCOLUMN xmlns IN DEMOMAP;
MAP ELEMENT Orders AS XMLTABLE ALLORDERS IN DEMOMAP;
MAP RELATION Orders TO string LINK xmlns WITH xmlns GENERATE NO IN DEMOMAP;
MAP ELEMENT Order AS XMLTABLE CUSTOMERORDER IN DEMOMAP;
MAP ATTRIBUTE id OF Order AS XMLCOLUMN orderid IN DEMOMAP;
MAP CHILD Order_org OF Order AS XMLCOLUMN Order_org IN DEMOMAP;
MAP RELATION Order TO Orders LINK CustomerId WITH DUMMY GENERATE NO IN DEMOMAP;
MAP ELEMENT Orderline AS XMLTABLE ORDERLINE IN DEMOMAP;
MAP ATTRIBUTE id OF Orderline AS XMLCOLUMN orderlineid IN DEMOMAP;
MAP CHILD OrderLine_org OF Orderline AS XMLCOLUMN OrderLine_org IN DEMOMAP;
MAP RELATION Orderline TO Order LINK orderid
WITH orderid GENERATE YES IN DEMOMAP;
COPYFILE '*C:\TMP\XML\ex10.xml' INTO '.\XMLParsing\ex10.xml';
PARSEXML '.\XMLParsing\ex10.xml' USING DEMOMAP;
CONVERTXMLTABLE CUSTOMERORDER INTO TABLE ORDERS FROM DEMOMAP;
CONVERTXMLTABLE ORDERLINE INTO TABLE ORDERLINES FROM DEMOMAP;
CONVERTXMLTABLE ALLORDERS INTO TABLE ALLORDER FROM DEMOMAP;
COPY TABLE ORDERLINES INTO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
COPY TABLE ORDERS INTO TMPTABLE;
SET TMPTABLE TO TABLE TMPTABLE;
Exercise 1: Parse using READXMLSTRUCTURE
----------------------------------------
Calculate sum of GrandTotal values of all orders listed in xml below, using READXMLSTRUCTURE command.
.. code-block:: xml
XYZ Industries
Tokyo
2013
100.013
ABC Technology
Madrid
2011
100.011
2012
100.012
IAS Software
Istanbul
2010
100.010
2014
100.014
Exercise 2: Parsing Simple XML Documents
----------------------------------------
Calculate grand total value (sum of Price*Quantity for each product) for the products listed in XML document below:
.. code-block:: xml
Product 1
6.30
5
Product 2
2.50
3
Product 3
1.20
2
Here is the solution:
::
OBJECT:
TABLE PRODUCTLIST,
TABLE PRODUCT,
INTEGER ROWCOUNT;
CLEAR XMLMAP DEMOMAP;
CREATEXMLMAP DEMOMAP;
MAP ELEMENT ProductList AS XMLROOTTABLE ProductList IN DEMOMAP;
MAP ELEMENT Product AS XMLTABLE Product IN DEMOMAP;
MAP CHILD Name OF Product AS XMLCOLUMN Name IN DEMOMAP;
MAP CHILD Price OF Product AS XMLCOLUMN Price IN DEMOMAP;
MAP CHILD Quantity OF Product AS XMLCOLUMN Quantity IN DEMOMAP;
MAP RELATION Product TO ProductList LINK DUMMY WITH DUMMY GENERATE NO IN DEMOMAP;
COPYFILE '*C:\TMP\XML\ex12.xml' INTO '.\XMLParsing\ex12.xml';
PARSEXML '.\XMLParsing\ex12.xml' USING DEMOMAP;
CONVERTXMLTABLE Product INTO TABLE PRODUCT FROM DEMOMAP;
REMOVE COLUMN DUMMY PERMANENT FROM PRODUCT;
APPEND COLUMN GRANDTOTAL, DECIMAL, 10 TO PRODUCT;
COPY TABLE PRODUCT INTO TMPTABLE;
LOOP AT TMPTABLE
BEGIN
DECIMALVAR1 = TMPTABLE_Price;
DECIMALVAR2 = TMPTABLE_Quantity;
TMPTABLE_GRANDTOTAL = DECIMALVAR1 * DECIMALVAR2;
ENDLOOP;
SET TMPTABLE TO TABLE TMPTABLE;
Exercise 3: Parsing Complex XML Documents
-----------------------------------------
Exercise 4: Validation
----------------------