This theme uses "root" links to dependencies and assets (CSS, JS, images, etc.). Place all theme files and directories at the root of your web server for this page to render as intended.
Senior Design Team sdmay18-27 • Data Analytic Tools for Inconsistency Detection in Large Data Sets
Project Description
Kingland processes a large amount of data that it receives from its
clients everyday. This data can be relating to customers, companies, or
agreements between entities. This data is compared to a central inconsistency
database in order to detect inconsistencies and then added to the database.
An example of an inconsistency would be two customer records containing the
same social security number, but different names. This is an issue, since a
social security number should be unique. The database contains over 100 million
records, and around 10% of these records are updated or inserted daily. Due to
its size, this comparison takes several hours to run every day. This time stems
from the fact the entire database cannot be loaded into main memory at one time
and the use of SQL inner join statements to check for inconsistencies, which is
inefficient. Kingland would like to process 100 million records for
inconsistencies in an hour or less. Additionally this detection must begin with
the latest version of the inconsistency database after the reports come in.