Duplicate records in databases do not only cause high costs, they can also lead to many other problems.
Not least when consolidating different data inventories, e.g. during fusion or adding value from acquired data,
the duplicate search is an indispensable and critical business process. This is, why data quality
more and more becomes a significant value for every business.
Fuzzy Duplicate Search
When searching for such duplicate records, you must distinguish between sharp (with exact accordance) and similar duplicates.
Grouping for exact matches is something, what any DBMS can do in seconds.
On the other hand is the detection of similar records something, that can be hardly achieved and what is computationally intensive. This only succeeds with specialized tools.
A problem is, that without the capable methods you cannot even estimate, in which dimension such duplicates exist in your data. You just can't see it.
On the Search for similar Records
Over years, phonetic algorithms (e.g. SoundEx) are used to find what sounds similar. This approach already brings up some results, which go beyond any sharp comparison.
Thereby permutations of the strings (like twisting and mirroring) stay unconsidered and a too strong emphasis is layed upon the first letter.
Much better is the use of pattern-matching algorithms (e.g. Levenshtein Distance). Such algorithms can consider permutations, but are highly calculation cost intensive.
So another problem is the total running time of the calculation: When using pattern-matching algorithms, in principle each record must be compared with each other.
This means for n records the total number of (n - 1) * n / 2 comparisons. That are ½ trillion (1012) complex calculations for a datatable with 1 million records. The calculation could last for years.
The FuzzyDupes Method
The FuzzyDupes method was developed within 9 years by now, completely and originally done by Kroll-Software. The calculation kernel contains more than 7.000 lines of code.
FuzzyDupes makes use of a Trigram-Hashidex for building clusters. This is a mathematical exact and reliable way to preselect good candidates for the deeper search, which does not depend on phonetical algorithms.
The deep search pattern-matching algorithm was also developed by Kroll-Software and it can better consider all permutations than any other known algorithm.
All used algorithms are based completely on pattern-matching and are therefore language- and culture independent. Unicode is fully supported and so this works not only with latin characters.
It is mathematically verifiable, that all similarities are detected consistently and reliably.
FuzzyDupes 2012 uses parallel Execution and 64-bit
The two critical resources for the duplicate search application are memory usage and calculation time.
With large data tables, the process has a high demand on RAM, which can only be allocated on 64-bit systems. 32-bit systems can only address a maximum of 2,4 gigabyte of memory.
We put in much development effort to parallelize the algorithms and unleash the power of modern multi-core cpu's. FuzzyDupes scales very good with the number of cores.
So the current version offers the search in bigger data tables on todays standard computers.
Why is FuzzyDupes so affordable compared with other duplicate search software ?
Duplicate search programs used to be specialized solutions and the preserve of a very limited clientele. In addition, these applications could only be run on mainframe computers due to the high computing power required. As a result, the applications were very expensive.
We believe that the ability to perform fuzzy duplicate searches is crucial for every company maintaining a customer database. We want to make our application accessible to small and medium sized enterprises and recognize that the price must stand in direct relation to the benefit gained. These considerations form the basis for the price of our product. However, the benefit for your company can far exceed the cost of a FuzzyDupes license.
Why does the new demo version offer full functionality ?
We have learned from experience that it is difficult to persuade new users of the necessity of a duplicate search if the demo version is too limited. Until now, we have only issued full-featured versions on request.
However, we believe that a demo version that is too limited is of no use to anybody. With this in mind, our latest demo version allows you to try out the full functionality of the program and assess a complete list of search results.
Please note: This free demo version is supplied for trial purposes only as a means of helping you to decide whether the application lives up to its claims and whether or not you have a need for a duplicate search program.
To use the search results, you will need to purchase a license. We call this "fair software". Please be fair and obtain a license for this software if you wish to use it in a productive environment.
New in FuzzyDupes Version 2012
Notebly higher power and speed through parallel execution and 64-bit
Thereby practically unlimited size of data searchable
Full usage of modern Core-iX cpu's and 64-bit systems
Uses DotNet 4.0 Framework
Display of the match-factor in the results
Support of MS-Access and MS-Excel data sources even on 64-bit systems
The new 32-bit launcher offers access to 32-bit data sources on 64-bit systems
Supported Data Sources / DBMS:
MS-Access, MS-Access 2007* and 2010* on 32-bit and 64-bit systems
MS-Excel, MS-Excel 2007* and 2010* on 32-bit and 64-bit systems
Other Datasources with ODBC-Driver or OLEdb Provider, e.g. Oracle, IBM DB2, MySQL, dBase, Foxpro, Paradox, FileMaker, Cache, PostgreSQL, etc.
Search and deletion from MS-Outlook contact folders.
This makes FuzzyDupes the solution for cleansing Outlook contacts
BulkMailer Address Database
32-bit data sources can be accessed on 64-bit systems using the FuzzyDupes 32-bit launcher
Use this SetUp
for all Windows Versions
(Windows XP, Server 2003/2008, Vista, Windows 7/8)
This download can be used for a limited 30-day evaluation.
The full version can then be activated by purchasing a licence.
You can test FuzzyDupes 2012 for free and with no obligation for 30 days,
after which you are required to usage license, if you want to continue using the program.
A single work-station** license costs EUR 349.- net amount*
*) All prices are net amounts.
If and how much VAT you have to pay depends on how and from where you place your order.
More information can be found at Shop
**) The license permits use of the software at one work station for an unlimited time. There are no further costs.
Price includes updates to all 8.x versions and free support by email or phone.
Order FuzzyDupes 2012
Secure payment via credit card, bank transfer or check
over the company ShareIt! element 5 AG, Koeln
The update is free for registered users of version 5.x, 6.x and 7.x, if you ordered the software after Jan. 1, 2010.
If your license is older, you can order an update license for EUR 174,-
if you are not sure, please ask for the update. If you are sure, please order your update
Sample Search Result
(FuzzyDupes finds similar records in address databases.)
If you have questions about this product or ordering,
dont hesitate to contact us. Well be happy to advise you.
Phone: +41-41-5351767 (Switzerland)
These customers have decided for FuzzyDupes:
Arnold & Porter LLP,
BFB Branchen-Fernsprechbuch GmbH,
BOTAMENT Systembaustoffe GmbH & Co.KG,
Bundesanstalt für Arbeitsschutz und Arbeitsmedizin,
CAQ AG Factory Systems,
CompuMED GmbH & Co.KG,
CreditPlus Bank AG,
Deutsche Bahn AG,
Deutsche Lufthansa AG,
DHL Solutions GmbH,
Dresdner Bank Luxemburg S.A.,
DuPont Performance Coatings GmbH & Co. KG,
E.ON Ruhrgas AG,
European Businessguide GmbH,
Fresenius Netcare GmbH,
Hewlett Packard EMEA GmbH,
Hirschfeld Touristik Event GmbH & Co.KG,
InterRisk Versicherungs AG,
LIDL Stiftung & Co KG,
Liechtensteinische Post AG,
Maritim Hotelgesellschaft mbH,
music-city Steinbrecher GmbH & Co.KG,
Oberfinanzdirektion Karlsruhe Landeszentrum f. Datenverarbeitung,
OÖ. Tourismus Technologie GmbH,
ORWO Media GmbH,
P&I Personal & Informatik AG,
SCA Packaging Deutschland,
Siemens VDO Automative AG,
Toys "R" Us GmbH,
Volksbank Bad Saulgau eG,
WTS Schaltgeräte GmbH,
Xella Baustoffe GmbH,
Zürcher Hochschule für Angewandte Wissenschaften,
Youll find more information on FuzzyDupes at our online help page.
This software product was tested in the Softpedia labs.
Softpedia guarantees that FuzzyDupes 2012 is 100% CLEAN, which means is does not contain any form of malware, including but not limited to: spyware, viruses, trojans and backdoors.
FuzzyDupes Duplicate Search in your Applications
You want to integrate FuzzyDupes into your applications ?
We provide you with a DotNet 2.0 and 4.0 Assembly or a COM-Object and free developer support.