Data mining technique helps official identify corruption

CAMBRIDGE, England, June 15 (UPI) -- A group of data-literate sociologists say they've developed a new data analysis technique for identifying corruption in government.

Throughout the democratic world, most public procurement data (info on the allocation and use of tax dollars) is now publicly available in digital form. The Internet Age has ushered in a new era of transparency, but it hasn't necessarily put an end to corruption, or the misuse of public funds.

Even with a gluttony of information, spotting corruption and misspending isn't easy. It takes a trained eye and hours of hard work to sort through the relevant data. Many journalists and campaigners do their best, but a certain amount of luck is needed.

Researchers at the University of Cambridge has developed a series of algorithms than mine public procurement data for "red flags" -- signs of the abuse of public finances. Scientists interviewed experts on public corruption to identify the kinds of anomalies that might indicate something fishy.

Their research allowed them to hone in on a series of red flags, like an unusually short tender period. If a request for proposal is issued by the government on a Friday and a contract is awarded on Monday -- red flag.

In Europe, where the researchers focused their efforts, the process of requesting proposals and awarding public contracts is called tendering.

Some of the other red flags identified by the researchers includes tender modifications that result in larger contracts, few bidders in a typically competitive industry, and inaccessible or unusually complex tender documents.

By scanning for these red flags, researchers say they can identify areas at risk of corruption. And the analysis technology is scalable, meaning their data mining technology can hone in on corruption risks at specific companies or within niche industries -- or zoom out and highlight regional and national risks.

"Using our methodology, institutionalised corruption can be measured right down to the level of individual contracts and tenders in about 50 countries around the globe since 2008 to 2009 -- opening up a whole universe of scientific and policy applications," researcher Lawrence King, a sociology professor at Cambridge, said in a press release. "We aim to make CRI available to citizens, civil society groups and journalists, to hold politicians and political parties accountable for corrupt behavior."

King's research partner and fellow sociologist, Mihaly Fazekas, is looking to pair the new technology with crowdsourced data.

"Imagine a mobile app containing local CRI data, and a street that's in bad need of repair. You can find out when public funds were allocated, who to, how the contract was awarded, how the company ranks for corruption," explained Fazekas. "Then you can take a photo of the damaged street and add it to the database, tagging contracts and companies."

"The idea that the public are going to be able to interrogate this data on a very localised basis and contribute to it themselves through things like smartphone apps is a compelling one!"