Convert A sitemap.xml File To A HTML Sitemap With PHP

Wednesday, August 13, 2008 - 10:12

I have already talked about converting a sitemap.xml file into a urllist.txt file, but what if you want to create a HTML sitemap? If you have a sitemap.xml file then you can use this to spider your site, scrape the contents of each page and populate the HTML file with this information.

The following code does this. For every page it looks for the <title> tag, the description meta tag and the first <h2> tag on the page. These items are then used to construct a segment of HTML for that page.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
<?php
$header = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<title>HTML Sitemap</title>
</head>
<body>';
 
set_time_limit(400);
 
$currentElement = '';
$currentLoc = '';
 
$map = "<h1>HTML Sitemap</h1>"."\n";
 
function parsePage($data)
{
 global $map;
 /*
 if you want to trap a certain file extention then use the syntax below...
 stripos($data, ".php")>0
 stripos($data, ".htm")>0
 stripos($data, ".asp")>0
 */
 if ( stripos($data,".pdf") > 0 ) {
  // if the url is a pdf document.
  $map .= '<p><a href="'.$data.'">PDF document.</a></p>'."\n";
  $map .= '<p>A pdf document.</p>'."\n";
 } elseif ( stripos($data, ".txt")>0 ) {
  // if the url is a text document
  $map .= '<p><a href="'.$data.'">Text document.</a></p>'."\n";
  $map .= '<p>A text document.</p>'."\n";
 } else {
  // try to open it anyway...
  // make sure that you can read the file
  if ( $urlh = @fopen($data, 'rb') ) {
   $contents = '';
   //check php version
   if ( phpversion()>5 ) {
    $contents = stream_get_contents($urlh);
   } else {
    while ( !feof($urlh) ) {
     $contents .= fread($urlh, 8192);
    };
   };
 
   // find the title
   preg_match('/(?<=\<[Tt][Ii][Tt][Ll][Ee]\>)\s*?(.*?)\s*?(?=\<\/[Tt][Ii][Tt][Ll][Ee]\>)/U', $contents, $title);
   $title = $title[0];
 
   // find the first h1 tag
   $header = array();
   preg_match('/(?<=\<[Hh]2\>)(.*?)(?=\<\/[Hh]2\>)/U', $contents, $header);
   $header = strip_tags($header[0]);
 
   if ( strlen($title) > 0 && strlen($header) > 0 ) {
    // print the title and h1 tag in combo
    $map .= '<p class="link"><a href="'.str_replace('&','&amp;',$data).'" title="'.(strlen($header)>0?trim($header):trim($title)).'">'.trim($title).(strlen($header)>0?" - ".trim($header):'').'</a></p>'."\n";
   } elseif ( strlen($title) > 0 ) {
    $map .= '<p class="link"><a href="'.str_replace('&','&amp;',$data).'" title="'.trim($title).'">'.trim($title).'</a></p>'."\n";
   } elseif ( strlen($header) > 0 ) {
    $map .= '<p class="link"><a href="'.str_replace('&','&amp;',$data).'" title="'.trim($header).'">'.trim($header).'</a></p>'."\n";
   };
 
   // find description
   preg_match('/(?<=\<[Mm][Ee][Tt][Aa]\s[Nn][Aa][Mm][Ee]\=\"[Dd]escription\" content\=\")(.*?)(?="\s*?\/?\>)/U', $contents, $description);
   $description = $description[0];
 
   // print description
   if ( strlen($description)>0 ) {
    $map .= '<p class="desc">'.trim($description).'</p>'."\n";
   };
   // close the file
   fclose($urlh);
  };
 };
};
 
/////////// XML PARSE FUNCTIONS HERE /////////////
// the start element function
function startElement($xmlParser, $name, $attribs)
{
 global $currentElement;
 $currentElement = $name;
};
 
// the end element function
function endElement($parser, $name)
{
 global $currentElement,$currentLoc;
 if ( $currentElement == 'loc') {
  parsePage($currentLoc);
  $currentLoc = '';
 };
 $currentElement = '';
};
 
// the character data function
function characterData($parser, $data) 
{
 global $currentElement,$currentLoc;
 // if the current element is loc then it will be a url
 if ( $currentElement == 'loc' ) {
  $currentLoc .= $data;
 };
};
 
// create parse object
$xml_parser = xml_parser_create();
// turn off case folding!
xml_parser_set_option($xml_parser, XML_OPTION_CASE_FOLDING, false);
// set start and end element functions
xml_set_element_handler($xml_parser,"startElement", "endElement");
// set character data function
xml_set_character_data_handler($xml_parser, "characterData");
 
// open xml file
if ( !($fp = fopen('sitemap.xml', "r")) ) {
 die("could not open XML input");
};
 
// read the file - print error if something went wrong.
while ( $data = fread($fp,4096) ) {
 if ( !xml_parse($xml_parser, $data,feof($fp)) ) {
  die(sprintf("XML error: %s at line %d",xml_error_string(xml_get_error_code($xml_parser)), xml_get_current_line_number($xml_parser)));
 };
};
 
// close file
fclose($fp);
 
$footer = '</body>
</html>';
 
// write output to a file
$fp = fopen('sitemap.html', "w+");
fwrite($fp,$header.$map.$footer);
fclose($fp);
 
// print output
echo $header.$map.$footer;
?>

This script prints out the sitemap and also saves the sitemap to a file for later use. This is essential as the script can take a long time to run due to all of the page accessing that it has to do.

This script is failry complicated and has gone through several versions since I first created it so if you find any improvements or bugs then let me know and I will incorporate them.

Category: 
philipnorton42's picture

Philip Norton

Phil is the founder and administrator of #! code and is an IT professional working in the North West of the UK.
Google+ | Twitter

Comments

amazing script! but what is the file it creates so it dosen't have to keep recreating the same sitemap ? I can't seem to find it after I load the html sitemap thanks ! john
philipnorton42's picture
Submitted by philipnorton42 on Wed, 07/28/2010 - 21:07

The file is called sitemap.html. The write function is on line 137. Although the script is quite old now so I would probably have written it in a different way if I did the same thing now.

This script is incredible, I was wondering if you could help me modify it so that it does a pagination, like http://example.com/sitemap.php?page=1 (After 1000 Links on the Page) = http://example.com/sitemap.php?page=2 etc. is it possible? I checked the whole internet and could not find any, the only things people offer are site scrapers that make a sitemap in xml, i need the reverse xD

Add new comment