A common issue I have come across in the past is that I have a CMS system, or an old copy of Wordpress, and I need to create a set of keywords to be used in the meta keywords field. To solve this I put together a simple function that runs through a string and picks out the most commonly used words in that list as an array. This is currently set to be 10, but you can change that quite easily.
The first thing the function defines is a list of "stop" words. This is a list of words that occur quite a bit in English text and would therefore interfere with the outcome of the function. The function also uses a variant of the slug function to remove any odd characters that might be in the text.
function extractCommonWords($string){ $stopWords = array('i','a','about','an','and','are','as','at','be','by','com','de','en','for','from','how','in','is','it','la','of','on','or','that','the','this','to','was','what','when','where','who','will','with','und','the','www'); $string = preg_replace('/[^a-zA-Z0-9 -]/', '', $string); // only take alphanumerical characters, but keep the spaces and dashes too… $matchWords = $matchWords[0]; foreach ( $matchWords as $key=>$item ) { } } foreach ( $matchWords as $key => $val ) { $wordCountArr[$val]++; } else { $wordCountArr[$val] = 1; } } } return $wordCountArr; }
The function returns the 10 most commonly occurring words as an array, with the key as the word and the amount of times it occurs as the value. To extract the words just use the implode() function in conjunction with the array_keys() function. To change the number of words returned just alter the value in the third parameter of the array_slice() function near the return statement, currently set to 10. Here is an example of the function in action.
$text = "This is some text. This is some text. Vending Machines are great."; $words = extractCommonWords($text);
This produces the following output.
some,text,machines,vending
Update
After lots of versions of this code submitted by users I think the most reliable version is this one from Cenk.
function extractKeyWords($string) { $string = preg_replace('/[\pP]/u', '', trim(preg_replace('/\s\s+/iu', '', mb_strtolower($string)))); $matchWords = array_filter(explode(' ',$string) , function ($item) use ($stopwords) { return !($item == '' || in_array($item, $stopwords) || mb_strlen($item) <= 2 || is_numeric($item));}); }
This will produce the following sort of output.
print implode(',', extractKeyWords("This is some text. This is some text. Vending Machines are great.")); // prints "this,text,some,great,are,vending,machines" print implode(',', extractKeyWords('হো সয়না সয়না সয়না ওগো সয়না এত জ্বালা সয়না ঘরেতে আমার এ মন রয়না কেন রয়না রয়না')); // prints "সয়না,রয়না,কেন,আমার,ঘরেতে,ওগো,জ্বালা"
If you use this then you should be aware that it requires PHP 5.5+ due to the use of a closure, but then that shouldn't be a problem :)
Comments
Anonymous - Sun, 06/20/2010 - 12:29
$str="";
and i will split it up so i can change the values and sample it again to work with echo $str; sorry for my bad english. i have searched the entire net and nothing is what i need so i hope anyone can help me. my solution is not good because it bet big and time consuming. and i have find some on the internet that look a like that i need but i cant sample it again without destroying anything. sincerely LAT D-Grund.dk Administrator my email is [email protected]philipnorton42 - Sun, 06/20/2010 - 17:28
Jaco - Fri, 07/02/2010 - 12:21
philipnorton42 - Fri, 07/02/2010 - 14:15
Dusan - Tue, 07/20/2010 - 00:49
philipnorton42 - Tue, 07/20/2010 - 08:51
Ronnie - Fri, 09/03/2010 - 18:00
strtolower()
call inside both foreach loops is really not needed either since you have already converted $string to lowercase beforehand.Guillermo - Wed, 01/19/2011 - 04:17
Guillermo - Wed, 01/19/2011 - 04:23
philipnorton42 - Wed, 01/19/2011 - 10:18
Guillermo - Wed, 01/19/2011 - 22:06
Matt Hawkins - Thu, 01/27/2011 - 10:39
Brit - Wed, 03/23/2011 - 10:52
Brilliant. Thanks for this piece of code. I was after doing something very similar so have used this as a base and made a few tweaks here and there so it fits my needs.
Mkj - Sat, 05/21/2011 - 09:37
Hi
Great code and is working a treat on my site but I have a problem with words containing 'ss'. The double ss is being removed so a word such as 'password' is being seen as 'paword' in the keywords.
Regards
Tolenca - Wed, 06/22/2011 - 19:46
Hi, would you please to let me know how I could include spanish characters too?.
For example I need to print: 'tamaño' instead I get: 'tamao'
Thanks in advance
Tolenca
philipnorton42 - Wed, 06/22/2011 - 22:20
You need to add those special characters to the regular expression on line 6 of the examples above. You just need to make sure it takes alphanumeric characters as well as the spanish letters. I think that should work.
Joe Bowman - Fri, 09/02/2011 - 14:09
instead of
$wordCountArr = array();I did:
To get all assoc array $wordCoundArr of [word] => [occurences], ignoring words that have occurred 1 or 2 (or specified number of) times.
philipnorton42 - Fri, 09/02/2011 - 14:11
Interesting take on it. I like it! :)
Gerhard Racter - Thu, 10/27/2011 - 15:49
I changed line 13 to not include 'keywords' that are just numbers using the is_numeric() function
if ( $item == '' || in_array($item, $stopWords) || strlen($item) >= 3 || is_numeric($item) ) {
HSMoore - Tue, 11/08/2011 - 09:43
Thanks, this is quite useful!
Anonymous - Thu, 05/03/2012 - 15:23
Make sure this line:
reads like this:
wookiester - Fri, 05/04/2012 - 10:53
This script just returns a string of comma separated keywords and supports multibyte characters.
Adjust your stopwords for your locale.
wookiester - Fri, 05/04/2012 - 11:11
Whoops.. Ignore my previous code block.. it was bugged.. try this.
Mohamed saad - Thu, 05/17/2012 - 15:51
philipnorton42 - Thu, 05/17/2012 - 15:55
Bob Davies - Tue, 08/14/2012 - 07:14
$string = preg_replace('/[^a-zA-Z0-9 -]/', '', $string);
with this:$string = preg_replace('/[^\w\d -]/', '', $string);
It should support extended characters, not sure off the top of my head how much unicode is covered by \w (if any), but it works for a bunch of quick tests I've put through it.summsel - Wed, 11/07/2012 - 16:57
preg_replace('/[ ]{2,}/sm', ' ', $text)
philipnorton42 - Wed, 11/07/2012 - 22:32
weter - Tue, 01/01/2013 - 06:36
Keybo Wordsy - Sat, 10/19/2013 - 13:38
Roshan - Sun, 12/08/2013 - 08:26
jasa pembuatan web - Wed, 08/13/2014 - 23:13
Gary - Wed, 12/17/2014 - 22:37
M - Thu, 12/25/2014 - 14:13
philipnorton42 - Fri, 12/26/2014 - 10:06
$string = preg_replace('/[^a-zA-Z0-9 -]/', '', $string);
To this:$string = preg_replace('/[^a-zA-Z0-9 ]/', '', $string);
Line 7 in the code example above.Kinley Chriatian - Thu, 01/29/2015 - 18:51
oscar - Thu, 11/26/2015 - 07:52
Cenk - Sat, 01/09/2016 - 23:00
NNO - Sun, 04/24/2016 - 08:15
raphael - Thu, 04/28/2016 - 16:58
philipnorton42 - Thu, 04/28/2016 - 17:16
delatosca - Fri, 10/07/2016 - 09:20
Dzulfikar - Fri, 02/17/2017 - 02:37
Mugo - Thu, 03/09/2017 - 07:08
AJ - Tue, 03/28/2017 - 18:44
felipe - Wed, 10/04/2017 - 00:11
Mind Roaster - Mon, 11/20/2017 - 10:10
Sarthak Gophane - Wed, 01/03/2018 - 06:56
David Maina - Sat, 10/06/2018 - 12:16
This code helped develop a WordPress plugin that generates tag and keywords for each post.
Add new comment