Finding Duplicate Values In A MySQL Table

Note: This post is over two years old and so the information contained here might be out of date. If you do spot something please leave a comment and we will endeavour to correct.

6th February 2008 - 3 minutes read time

To find duplicate values you need to use the MySQL COUNT() function and then pick out all of the counts that are greater than one.

SELECT value,COUNT(value) AS Count
FROM test
GROUP BY value
HAVING (COUNT(value) > 1)
ORDER BY Count DESC;

Conversely you can also select the rows that only have a single entry.

SELECT value,COUNT(value) AS Count
FROM test
GROUP BY value
HAVING (COUNT(value) = 1)
ORDER BY Count DESC;

However, it is very nice to pick out the duplicate entries in a table, but you might still need to do something with them. Here is a query to delete any duplicate rows from a table. It does a simple self join and deletes the row value with the lowest ID.

DELETE bad_rows.*
FROM tests AS good_rows
INNER JOIN tests AS bad_rows ON bad_rows.number = good_rows.number
AND bad_rows.id > good_rows.id;

More information on this deletion query and other methods of deleting duplicates can be found at Xaprb.com.

select

Add new comment

Related Content

Solving Query Performance Problems With The MySQL Slow Query Log

21st January 2024

MySQL's slow query log is a key component in your MySQL administration setup. Whilst normal logging can help you in terms of tracking down issues with your database system, the slow query log can help you track down issues in your database setup before they become problematic.

Case Insensitive Like Searches In MySQL

2nd July 2011

I needed to create a query that did a case insensitive search using the LIKE command in MySQL and I quickly realised that in order to do this I would need to alter both the parameter and the table data to be the same case. This can be done by using the MySQL UPPER() command on the table data and the strtoupper() PHP function on the input data.

Overriding Drupal 6 Automatic Select Element Validation

29th March 2011

Whilst creating a large and complex form in Drupal 6 recently I hit upon a problem that took me a couple of hours to solve so I am posting the solution here in case anyone else gets similarly stuck. I am also writing this down so that I can remember the strategy in case I have to do the same thing again.

Connecting To A Non Standard MySQL Socket In PHP

11th February 2011

Connecting to a MySQL database in PHP is usually not a difficult thing to do, in fact it is one of the first things that many tutorials will go though. However, there are certain curcumstances that require more information than the standard host, password and username details.

Count Number Of Characters In A String With MySQL

30th September 2010

Today I needed to grab some data from a table where there was more than one occurrence of a string within another string. Basically, I needed to find all URL's from a table that were more than 3 levels deep (i.e. with 3 slashes), but realised there wasn't a function to do this in MySQL.

Drupal 6: Change Views Exposed Filter Form Select Element To A List Of Checkboxes

12th May 2010

Whilst creating a view for a Drupal site the other day I created an exposed form but rather than the usual select boxes that get displayed I needed them to be a list of check boxes.