Zend Framework

The FlashMessenger in Zend Framework has a bit of an odd name as it has nothing to do with Adobe Flash at all. It is a controller action helper defined in the class Zend_Controller_Action_Helper_FlashMessenger, which is used to store and retrive messages that are temporarily stored in the user's session. This is useful if you want to provide form validation and redirection at the same time as you can print out messages after the page has been loaded. If you are familiar with Drupal then this class acts in the same kind of way as the drupal_set_messages() function.

Because FlashMessenger is an action helper it can be initialized in a number of different ways, however, using the _helper property (the helper broker) of the controller is probably the easiest way. The FlashMessenger can be retrived through the helper broker in the following ways:

If you have been following the last four posts you should now have an application that will allow you to view and edit PDF metadata, extract the document contents for search indexing, and allow users to search that index.

The one final thing to do is to sort out what happens when any PDF metadata is changed. At the moment the application will allow us to change the metadata as much as we like, but these changes will not be replicated in our search index. In order to do this we have to fully re-index everything. This is obviously the wrong way to go about things, and the solution is quite simple. All we need to do is up the file controllers/PdfController.php and change the editmetaAction() method so that when the PDF metadata is saved, the search index is updated. Add the following code to the editmetaAction() method, just before the redirect.

Last time we had indexed our PDF documents and were ready to add a search form to our application. Adding search requires two things, the form to enter the search terms into and an action to control what happens when the form is submitted.

The easiest task is to set up the search view, create the view file search.phtml (in views/scripts/search/) and add the following code. This is just a basic form with a single text input. Note the addition of the queryString parameter, this will allow us to print out the last searched for value in the search box, which is good usability practice.

Last time we had reached the stage where we had PDF meta data and the extracted contents of PDF documents ready to be fed into our search indexing classes so that we can search them.

The first thing that is needed is a couple of configuration options to be set up. This will control where our Lucene index and the PDF files to be indexed will be kept. Add the following options to your configuration files (called application.ini if you used Zend Tool to create your applcation).

luceneIndex = \path\to\lucene\index
filesDirectory = \path\to\pdf\files\

Last time we looked at viewing and saving meta data to PDF documents using Zend Framework. The next step before we try to index them with Zend Lucene is to extract the data out of the documents themselves. I should note here that we can't extract the data perfectly from every PDF document, we certainly can't extract any images or tables from the PDF into any recognisable text. There is a little issue with extracting the text because we are essentially looking at compressed data. The text isn't saved into the document, it is rendered into the document using a font. So what we need to do is extract this data into some format the Lucene can tokenize. Because we are just getting the text out of the document for our search index we can take a few short-cuts in order to get as much textual data out of the document as possible. All of this data might not be fully readable and we will definitely loose any formatting and images, but for the purposes we are using it for we don't really need it. The idea is that we can retrieve as much relevant and indexable content for Zend Lucene to tokenize. Also, it is not possible to extract the data out of encrypted PDF documents.

Zend Lucene is a powerful search engine, but it does take a bit of setting up to get it working properly. One thing that I have had trouble getting up and running in the past is indexing and searching PDF documents. The difficulty here is that it isn't immediately apparent how you can index the contents of a PDF document with ease. I came across a couple of functions you can try out, but even if that doesn't work it is possible to create and edit PDF meta data using the Zend_Pdf library. Because there is a lot to cover on this subject I thought I would create a blog post in multiple parts. For this post I will be looking at how to add and edit this meta data. This meta data can be used to classify your PDF documents and allow you to index them and provide a decent search solution using Zend Lucene.

Every application has a locale, even if that is just the locale of the author. Through the use of locals you can make your application aware of what sort of language, currency and even timezone that the user would like to see. In Zend Framework this is accoumplished via Zend_Locale.

There are many things to do with locale once, but first you need to determine where the user is based. To find this out you simply create a new instance of the Zend_Locale object. The following code will create the Zend_Locale object and print out the language and region of the user.

The latest versions of Zend Framework come with a handy little tool that gets you up and running with a basic Zend Framework install within a few moments. When you have downloaded Zend Framework you will notice that in minimal and full there is a directory called bin. This seems to be intended to contain lots of different tools, but at the moment it contains three files that are part of Zend Tool.

The important file in this directory is zf.php which is the actual core of the tool. The other two files are used depending on what operating system you are running. If you are running Windows then it is zf.bat that will be used, otherwise it is zf.sh that is important.

Zend_Search_Lucene offers some powerful document scanning capabilities, and there are a few different formats that are useful for the search engine to index.

To allow the indexing and searching of Excel documents using Zend_Search_Lucene you need to use the Zend_Search_Lucene_Document_Xlsx class. However, to use this class you must have the Zip module installed with PHP. For Windows users this means editing your php.ini file and uncommenting the following line:

For Linux users you will need to recompile PHP with the --enable-zip configure option.

Create and/or open the index in the normal way and you can index Excel documents using the following code.

Whilst creating a site the other day I thought about how I would manage the sitemap.xml file. This file is basically a XML file containing a list of URLs. Most major search engines understand (and look for) this file, so having it present on a site is a definite must.

I have been down the route before of having a sitemap.xml file created by the application every time a new record or something was added, but as this was a high traffic, multi-user site this approach just had to many problems. The main problem (aside from the potential performance hit) was that I would have to spend hours tying the calls to the sitemap.xml creation file into my application.