Optimal Hardware for Aim-Smart’s Data Quality Excel Add-in

Customers often ask me what our minimum and recommended hardware specifications are.  Well, I’ll give a short answer, followed by some additional explanation.

Aim-Smart has been designed with performance in mind from the beginning.  For this reason users are able to parse hundreds of thousands of records in just a few seconds.  When building the software, if at any point we thought performance was suboptimal, we spent significant effort to pinpoint the cause.  Each time, we’d find that, with a few more optimizations, we were able to improve the code and reduce the processing time by orders of magnitude in most cases.

We implemented our software in this fashion because we never knew what kind of hardware our customer’s might have.  In addition, because Excel doesn’t run on large powerful servers but rather individual laptops and desktops, we knew we didn’t have the luxury of sloppy code.

The hardest challenge to-date has been optimizing our fuzzy matching and deduplication algorithms. Having said all of this, our  fuzzy matching performance is excellent with nice linear behavior as the number of input records increases.  In addition, if you have additional demands on speed, we can work with you and come up with a solution (we have some tricks up our sleeves but it requires some extra configuration).

Despite everything we’ve done to reduce the need to have powerful computers, it of course doesn’t hurt and can only help.  The single most important thing when running Aim-Smart is the speed of the hard drive.  Because working with data is highly I/O dependent, a small investment in a fast SSD hard drive is well worth it.