Check for missing fields in xml

Submitted by loet on Mon, 2008-01-21 10:41 in Magic Parser

Hello David,

In my XML file in some records there are some fields missing.
When I parse the file the database is messed up because some fields are missing.
Is there a way to check for missing fields?
As you can see in the example in the additional fields in some records these fields are missing

<field name="accomodatietype" value="Hotel" />

<field name="accommodatie" value="Hotel Garni Dolomiti" />

<field name="aantalsterren" value="3" />

This is a small piece of the xml:

<product>

<productID>EZ74727</productID>

<name>RONDREIS HIGHLIGHTS VAN MAROKKO</name>

<price currency="EUR">879.00</price>

<productURL>http://www.kras.nl/tradetracker/index.asp?tt=839_0_6948_&amp;r=http%3A%2F%2Fwww.kras.nl%2Fvakantie%2Ftrip.htm%3Ftripid%3D74727%26osc%3Dttracker%26utm_source%3Dttracker%26utm_medium%3Dcps</productURL>

<imageURL>http://www.kras.nl/wsmedia/000/064/183.jpg</imageURL>

<description><![CDATA[Marokko is een land met vele gezichten. Vlakbij Europa gelegen, komt u toch in een geheel andere wereld terecht. De steden en ommuurde kashbaâ€™s ademen een mysterieuze sfeer uit. De koperslagers en leerlooiers oefenen nog dagelijks hun oude ambacht op straat uit. De kleurrijke souks ruiken naar een melange van kruiden en specerijen. Dit Marokko staat centraal tijdens onze 15-daagse afwisselende ...]]></description>

<categories>

<category name="reizen/rondreizen" path="reizen/rondreizen" />

<category name="reizen/Marokko" path="reizen/Marokko" />

</categories>

<additional>

<field name="subtitel" value="Busreis Marokko met o.a. Casablanca" />

<field name="vervoer" value="BusVliegtuig" />

<field name="verzorging" value="Vlg.reisbeschrijving" />

<field name="land" value="Marokko" />

<field name="land-iso" value="MA" />

<field name="reistype" value="rondreizen" />

<field name="reisduuralgemeen" value="15" />

</additional>

</product>

<product>

<productID>EW74644</productID>

<name>SKIÃ‹N IN ITALIÃ‹ - DOLOMIETEN - ANDALO</name>

<price currency="EUR">645.00</price>

<productURL>http://www.kras.nl/tradetracker/index.asp?tt=839_0_6948_&amp;r=http%3A%2F%2Fwww.kras.nl%2Fvakantie%2Ftrip.htm%3Ftripid%3D74644%26osc%3Dttracker%26utm_source%3Dttracker%26utm_medium%3Dcps</productURL>

<imageURL>http://www.kras.nl/wsmedia/000/160/064.jpg</imageURL>

<description><![CDATA[Waarom kiest u dit jaar niet eens voor skiÃ«n op de Paganella? Dit gebied strekt zich uit over de toppen van de berg Monte Paganella (60 km sneeuwzekere skipistes) tot aan het Meer van Molveno en de Brenta Dolo&shy;mieten. Naast het skiÃ«n worden er door de reisleiding allerlei gezellige aktiviteiten georganiseerd.Liever met de auto naar Andalo, klik dan ...]]></description>

<categories>

<category name="reizen/autovakantie" path="reizen/autovakantie" />

<category name="reizen/ItaliÃ«" path="reizen/ItaliÃ«" />

</categories>

<additional>

<field name="subtitel" value="Wintersportreis per vliegtuig met verblijf in het 3-sterren Hotel Olimpia inclusief ontbijt en diner" />

<field name="vervoer" value="Vliegtuig" />

<field name="verzorging" value="Halfpension" />

<field name="accomodatietype" value="Hotel" />

<field name="accommodatie" value="Hotel Olimpia" />

<field name="aantalsterren" value="3" />

<field name="land" value="ItaliÃ«" />

<field name="plaats" value="Andalo" />

<field name="land-iso" value="IT" />

<field name="reistype" value="autovakantie" />

<field name="reisduuralgemeen" value="8" />

</additional>

</product>

<product>

<productID>EW74691</productID>

<name>SKIÃ‹N IN ITALIÃ‹ - DOLOMIETEN - DIMARO</name>

<price currency="EUR">399.00</price>

<productURL>http://www.kras.nl/tradetracker/index.asp?tt=839_0_6948_&amp;r=http%3A%2F%2Fwww.kras.nl%2Fvakantie%2Ftrip.htm%3Ftripid%3D74691%26osc%3Dttracker%26utm_source%3Dttracker%26utm_medium%3Dcps</productURL>

<imageURL>http://www.kras.nl/wsmedia/000/183/216.jpg</imageURL>

<description><![CDATA[Het kleine plaatsje Dimaro is gelegen in het Val di Sole (wat `vallei van de zon` betekent... dat belooft wat!), een prachtig dal in het Italiaanse Trentino! De skimogelijkheden zijn hier werkelijk ongekend! In het nabijgelegen gebied Folgarida-Marilleva kunt u al heerlijke sportieve dagen doorbrengen, maar voor de fervente wintersporter is daar natuurlijk het allesomvattende gebied ...]]></description>

<categories>

<category name="reizen/wintersport" path="reizen/wintersport" />

<category name="reizen/ItaliÃ«" path="reizen/ItaliÃ«" />

</categories>

<additional>

<field name="subtitel" value="Wintersportreis per vliegtuig met verblijf in het 3-sterren Garni Hotel Dolimiti inclusief ontbijt" />

<field name="vervoer" value="Vliegtuig" />

<field name="verzorging" value="Logies/ontbijt,Halfpension" />

<field name="accomodatietype" value="Hotel" />

<field name="accommodatie" value="Hotel Garni Dolomiti" />

<field name="aantalsterren" value="3" />

<field name="land" value="ItaliÃ«" />

<field name="plaats" value="Dimaro" />

<field name="land-iso" value="IT" />

<field name="reistype" value="wintersport" />

<field name="reisduuralgemeen" value="8" />

</additional>

</product>

<product>

<productID>VERZ74699</productID>

<name>EXCURSIEREIS KEIZERLIJK BEIJING INCLUSIEF VERLENGING</name>

<price currency="EUR">814.00</price>

<productURL>http://www.kras.nl/tradetracker/index.asp?tt=839_0_6948_&amp;r=http%3A%2F%2Fwww.kras.nl%2Fvakantie%2Ftrip.htm%3Ftripid%3D74699%26osc%3Dttracker%26utm_source%3Dttracker%26utm_medium%3Dcps</productURL>

<imageURL>http://www.kras.nl/wsmedia/000/043/071.jpg</imageURL>

<description><![CDATA[Unieke reis naar keizerlijk Beijing, prachtige stad vol indrukwekkende paleizen, tempels, torens en verrassende wijken, pleinen en straten! Boekt u het uitgebreide excursiepakket (met natuurlijk aandacht voor het Plein van de Hemelse Vrede, de Verboden Stad en de Grote Muur), dan leert u Beijing van binnen en van buiten kennen.   HOTELU verblijft in een goed en ...]]></description>

<categories>

<category name="reizen/bus-excursiereizen" path="reizen/bus-excursiereizen" />

<category name="reizen/China" path="reizen/China" />

</categories>

<additional>

<field name="subtitel" value="Stedentrip China met excursies naar o.a. de Ming Graven" />

<field name="vervoer" value="Vliegtuig" />

<field name="verzorging" value="Logies/ontbijt" />

<field name="land" value="China" />

<field name="plaats" value="Beijing" />

<field name="land-iso" value="CN" />

<field name="reistype" value="bus-excursiereizen" />

<field name="reisduuralgemeen" value="14" />

</additional>

</product>

<product>

<productID>VERZ74700</productID>

<name>EXCURSIEREIS KEIZERLIJK BEIJING</name>

<price currency="EUR">655.00</price>

<productURL>http://www.kras.nl/tradetracker/index.asp?tt=839_0_6948_&amp;r=http%3A%2F%2Fwww.kras.nl%2Fvakantie%2Ftrip.htm%3Ftripid%3D74700%26osc%3Dttracker%26utm_source%3Dttracker%26utm_medium%3Dcps</productURL>

<imageURL>http://www.kras.nl/wsmedia/001/168/241.jpg</imageURL>

<description><![CDATA[Unieke reis naar keizerlijk Beijing, prachtige stad vol indrukwekkende paleizen, tempels, torens en verrassende wijken, pleinen en straten! Boekt u het uitgebreide excursiepakket (met natuurlijk aandacht voor het Plein van de Hemelse Vrede, de Verboden Stad en de Grote Muur), dan leert u Beijing van binnen en van buiten kennen.   HOTELU verblijft in een goed en ...]]></description>

<categories>

<category name="reizen/bus-excursiereizen" path="reizen/bus-excursiereizen" />

<category name="reizen/China" path="reizen/China" />

</categories>

<additional>

<field name="subtitel" value="Stedentrip China met excursies naar o.a. de Ming Graven" />

<field name="vervoer" value="Vliegtuig" />

<field name="verzorging" value="Logies/ontbijt" />

<field name="land" value="China" />

<field name="plaats" value="Beijing" />

<field name="land-iso" value="CN" />

<field name="reistype" value="bus-excursiereizen" />

<field name="reisduuralgemeen" value="8" />

</additional>

</product>

Is there a way to check for those fields so when they do not exist they get filled with NULL in the database?
I hope you can understand the problem.

Loet

Hello Loet, Yes, you can use

Submitted by support on Mon, 2008-01-21 12:26

Hello Loet,

Yes, you can use PHP's isset() function to check for the required fields before you go on to load the database.

However, that would still return true if the field were there but empty; so it might be better to check for an actual value.

You could do that quite simply, at the top of your myRecordHandler() function, like this:

if (!$record["FIELD-HOTEL"])

{

  return;

}

This will exit from myRecordHandler if $record["FIELD-HOTEL"] is blank. If you want to check more than 1 field, do it like this:

if (!$record["FIELD-HOTEL"] || !$record["OTHER-FIELD"] || !$record["ANOTHER-FIELD"])

{

  return;

}

Hope this helps!
Cheers,
David.

Hello David, Thanks for your

Submitted by loet on Mon, 2008-01-21 16:18

Hello David,

Thanks for your reply.
I have tried your solution but that makes no difference.
Maybe it is in the wrong place?
The database gets filled with the wrong data when some fields are missing.
Say I have 6 records from the xml, $record["1"] to $record["6"] and 6 fields in the databse
When all the records are present in the xml there is no problem but when record["2"] and record["3"] are missing in the xml the database is filled in field2 and field3 with the data from record["4"] and record["5"]. So a lot of data is in the wrong field.
This is my code with the function:

$counter = 0;

function kras($record)

	{

		if (!$record["ADDITIONAL/FIELD@1-VALUE"] || !$record["ADDITIONAL/FIELD@2-VALUE"] || !$record["ADDITIONAL/FIELD@3-VALUE"] || !$record["ADDITIONAL/FIELD@4-VALUE"])

		{

		return;

		}

	global $counter;

    global $sql;

    $counter++;

	$sql = "INSERT INTO kras (productID, name, price, productURL, imageURL, description, cat_1, cat_2, subtitel, vervoer, verzorging, accomodatietype, accomodatie, sterren, land, plaats, land_iso, reistype, reisduur) VALUES (

		'".$record["PRODUCTID"]."',

		'".mysql_real_escape_string($record["NAME"])."',

		'".$record["PRICE"]."',

		'".$record["PRODUCTURL"]."',

		'".$record["IMAGEURL"]."',

		'".mysql_real_escape_string($record["DESCRIPTION"])."',

       	'".mysql_real_escape_string($record["CATEGORIES/CATEGORY-NAME"])."',

		'".mysql_real_escape_string($record["CATEGORIES/CATEGORY@1-NAME"])."',

		'".mysql_real_escape_string($record["ADDITIONAL/FIELD-VALUE"])."',

		'".mysql_real_escape_string($record["ADDITIONAL/FIELD@1-VALUE"])."',

		'".mysql_real_escape_string($record["ADDITIONAL/FIELD@2-VALUE"])."',

		'".mysql_real_escape_string($record["ADDITIONAL/FIELD@3-VALUE"])."',

		'".mysql_real_escape_string($record["ADDITIONAL/FIELD@4-VALUE"])."',

		'".mysql_real_escape_string($record["ADDITIONAL/FIELD@5-VALUE"])."',

		'".mysql_real_escape_string($record["ADDITIONAL/FIELD@6-VALUE"])."',

		'".mysql_real_escape_string($record["ADDITIONAL/FIELD@7-VALUE"])."',

		'".$record["ADDITIONAL/FIELD@8-VALUE"]."',

		'".$record["ADDITIONAL/FIELD@9-VALUE"]."',

        '".$record["ADDITIONAL/FIELD@10-VALUE"]."')";

  mysql_query($sql);

  //echo $sql;

  echo $counter;

  echo "  XXXXXXXXX";

  print mysql_error();

  echo "<br />";

}

Loet

Hello Loet, That might not

Submitted by support on Mon, 2008-01-21 16:32

Hello Loet,

That might not work if the fields contain spaces. Try this for your IF statement instead:

 if ((trim($record["ADDITIONAL/FIELD@1-VALUE"])=="") || (trim($record["ADDITIONAL/FIELD@2-VALUE"])=="") | (trim($record["ADDITIONAL/FIELD@3-VALUE"])=="") || (trim($record["ADDITIONAL/FIELD@4-VALUE"])==""))

Cheers,
David.

Hello David, I was still

Submitted by loet on Sun, 2008-02-03 15:28

Hello David,

I was still strugling with this problem.
The given options did not work and I tried to figure out why.
I created this in my function and now it works.
Maybe it can be written in another way but now it checks if a field is missing.
When the field is missing it still enters a value for it in the database so the table does not mess up everything.

//2- INSERT NEW DATA

$counter = 0;

function kras($record)

	{

			if ($record["ADDITIONAL/FIELD@3-NAME"]!= "accomodatietype")	{

			$accomodatietype = "";

			} else {

			$accomodatietype = $record["ADDITIONAL/FIELD@3-VALUE"];

		}

		if ($record["ADDITIONAL/FIELD@4-NAME"]!= "accommodatie")	{

			$accommodatie = "";

			} else {

			$accommodatie = $record["ADDITIONAL/FIELD@4-VALUE"];

		}

		if ($record["ADDITIONAL/FIELD@5-NAME"]!= "aantalsterren")	{

			$aantalsterren = "";

			} else {

			$aantalsterren = $record["ADDITIONAL/FIELD@5-VALUE"];

		}

		if ($record["ADDITIONAL/FIELD@6-NAME"]!= "land")	{

			$land = "";

			} else {

			$land = $record["ADDITIONAL/FIELD@6-VALUE"];

		}

		if ($record["ADDITIONAL/FIELD@7-NAME"]!= "plaats")	{

			$plaats = "";

			} else {

			$plaats = $record["ADDITIONAL/FIELD@7-VALUE"];

		}

		if ($record["ADDITIONAL/FIELD@8-NAME"]!= "land-iso")	{

			$land_iso = "";

			} else {

			$land_iso = $record["ADDITIONAL/FIELD@8-VALUE"];

		}

	global $counter;

        global $sql;

        $counter++;	

	$sql = "INSERT INTO kras (productID, name, price, productURL, imageURL, description, cat_1, cat_2, subtitel, vervoer, verzorging, accomodatietype, accomodatie,  sterren, land, plaats, land_iso, reistype, reisduur) VALUES (

		'".$record["PRODUCTID"]."',

		'".mysql_real_escape_string($record["NAME"])."',

		'".$record["PRICE"]."',

		'".$record["PRODUCTURL"]."',

		'".$record["IMAGEURL"]."',

		'".mysql_real_escape_string($record["DESCRIPTION"])."',

       	'".mysql_real_escape_string(str_replace("reizen/","",$record["CATEGORIES/CATEGORY-NAME"]))."',

		'".mysql_real_escape_string(str_replace("reizen/","",$record["CATEGORIES/CATEGORY@1-NAME"]))."',

		'".mysql_real_escape_string($record["ADDITIONAL/FIELD-VALUE"])."',

		'".mysql_real_escape_string($record["ADDITIONAL/FIELD@1-VALUE"])."',

		'".mysql_real_escape_string($record["ADDITIONAL/FIELD@2-VALUE"])."',

		'".mysql_real_escape_string($accomodatietype)."',

		'".mysql_real_escape_string($accommodatie)."',

		'".mysql_real_escape_string($aantalsterren)."',

		'".mysql_real_escape_string($land)."',

		'".mysql_real_escape_string($plaats)."',

		'".$land_iso."',

		'".$record["ADDITIONAL/FIELD@9-VALUE"]."',

        '".$record["ADDITIONAL/FIELD@10-VALUE"]."')";

  mysql_query($sql);

  print mysql_error();

}

MagicParser_parse($file, "kras", "xml|PRODUCTS/PRODUCT/");

print "<p>Ingevoerd zijn ".$counter." records.</p>";

print "<p>Last SQL statement was: ".$sql."</p>";

print mysql_error();

Loet

Hello Loet, Yes, that makes

Submitted by support on Sun, 2008-02-03 15:37

Hello Loet,

Yes, that makes sense. The options above were aborting the record if those fields were not present,
which sounds like it was not what you wanted - they way you are doing it is probably as efficient as
this sort of check / set code can be...

Cheers,
David.