BIG DATA – ROLE OF AUDITOR
The audit profession has gone through many changes from time to time from review of the financial
figures to shifting the focus to risk assessment and internal control, and consideration of fraud in the
financial statement that could perpetrate externally or internally. These changes were due to how the
business works and the changes taking place in the business scenario. With the emergence of new
technology and the way the business is carried, new risks emerges and the audit profession is
expected to be updated and to acquire or at least be updated with the business changes that are taking
place in the business environment. In the changing scenario, big data has become popular for the
organizations which are widely spread across the globe and which are growing exponentially.
With the advent of Big data, the auditing profession has to start looking at how to leverage big data in
audits. Rapid changes in the business model possess a threat to the audit profession to match the foot
step of the way the business is done. The auditor is needed to acquire new skill and has to start looking
into every transaction structured or unstructured rather than audit by exception and analyse the same.
This article tries to bring out the challenges of using the big data and the emerging role of auditors in
the changing scenario.
Big data today has become a popular term which is used to describe the exponential growth and
availability of data both structured and unstructured. Big data is collection of data sets so large and
complex that it becomes difficult to process using on-hand data management tools or the traditional
data processing applications. The big data generally includes complex data sets that exceed the
processing capabilities of traditional infrastructure due to their size, format diversity and speed of
generation of data. Unlike the traditional Enterprise Resource Planning (ERP) data, where the data
used is homogeneous, big data has variety of heterogeneous sources like the customer feedback,
promotional calls made, exchange of various mails in the organization.
Large organisations today need to maintain large amount of both structured and unstructured data for
operational and compliance purposes. The resultant is huge growth in computing power and storage
capacity. Big data provides solution to handle such huge data. Businesses today are using big data not
only to boost performance but also to reduce risk and prevent loss. The big data is mostly used by
marketing, sales, engineering, product development, operations, logistics, and strategy development
groups to do smarter and faster jobs. Big data takes into consideration the compliance with applicable
privacy laws, reputational risks, security considerations and data destruction policies. The Board of
Directors and the top management of company are needed to be aware of the compliance
requirements, reputational risks, and ethical considerations surrounding privacy and manipulation of
sensitive personal data for the commercial gains. Companies need to be sure they are capturing the
right data and delivering it in real time to the right people. Big data inherently has its own challenges
which are needed to be addressed by the top management. The general principle which is followed
where big data is employed is; the greater the amount of data, the greater the risk. Any time an
organization uses big data to make decisions, there is risk that it could be misused or used
ineffectively. Auditors and other risk professionals obviously should monitor big data use to address this
CHALLENGES OF USING BIG DATA
Inconsistent data formats: Each company or each department in a company has different method of
describing the data. It can be describing the ledger or expenses or describing the location.
1. Understanding significance of each variable: Assessment of each variable in its own right to avoid
the problem of generalization.
2. Ratifying expert institution: An experience expert who has dealt with similar data can state whether
the data has any inconsistencies.
The real challenge of big data is not that you are acquiring large amount of data. It is what you do with
the data that matters. Big data can improve the decision making process and increase the profitability,
but it also creates significant risks from potential data breaches to privacy and compliance concerns.
Big data provides important opportunities to deliver value from information but an enterprise will be
more successful in long run if proper policies and procedures are in place.
Some important areas which are needed to be answered include:
a. Reliability of data
b. Protection of sources, processes and decisions from theft and corruption.
c. Collection of information without exposing the enterprise to legal and regulatory compliances.
d. Trend of actions taken that can be exploited by rivals
The audit professionals should use this opportunity to re-examine what aspects of technology should
be part of the audit practitioners skill set; given that society’s attitude towards technology has clearly
changed. The audit profession should understand what data is present in the big data and how this
data can be correlated with financial data to make it more effective and efficient audit. The task of
auditor in this regard environment increases. They should not only help organization identify the risks
that the use of big data can pose. At the same time, they must be alert to the opportunities the big data
can offer. The auditors should be approaching big data in proactive manner, identifying opportunities to
use large sets in new ways that will enable them to operate with greater efficiency. The auditors has
important role to play in relation to big data and exploiting big data to identify and mitigating the existing
risk. The opportunities and risks that big data presents are significant and complex –too significant and
complex to be considered solely or even primarily a technology concern.
EMERGING ROLE OF AUDITOR AND OTHER FINANCE PROFESSIONAL 
The role of auditor, CFO and controller can be separated into three components
1. The risk and business control components that defines the traditional role of finance and audit
2. Enhancements to internal and external audit. These are so significant they deserve their own
3. Applications of big data and predictive analysis that promise to support finance professionals as the
new pro-active stewards of better corporate governance.
Role of auditor while handling big data in traditional audit risk and business control includes:
a. Planning and forecasting: Big data adds new data sources and increased detail and makes the
forecasts fast to update. Improvements to predictive analytics make them more accurate.
b. Strategic Financial management: Big data adds depth to the detail, speed to updates, and the
opportunity to incorporate new meaningful data types both internal to the company and from outside
sources. Dashboards can track both leading and lagging indicators to give a more accurate view of the
c. Enhance financial closing, statutory reporting, and analysis of variances: Big data
technologies can allow incorporation of both new and traditional data types at very high levels of detail
and allow the data to be retrieved and processed in a fraction of time. The new technologies used in
the big data can achieve this integration cost effectively.
d. Compliance and regulation: The amount of risk, time and resources drained from these activities
can be substantially reduced through application of both big data technologies and predictive analysis.
e. Fraud detection: Using the greater variety and detail of data available through big data to power
predictive analytics to highlight potential fraud increases the likelihood of detection. It also decreases
the incidence of deviation resulting in better, faster detection and protection and less cost in human
f. Asset recovery : Detection of duplicate invoices and duplicate payments is one area where many
accountant professionals start with big data. It is now practical to use automated analytic sorting to
evaluate years of detailed invoice and payments to identify and recover duplicates due to low cost, high
speed Hadoop – based big data storage and retrieval systems.
g. Depth of audit: Big data technologies removes the requirement of sampling by handling massive
amounts of data and many different varieties of data at a cost, and in a retrieval and analysis time
frame that was previously impossible. On the operational side, it is this ability to look literally at
everything that now makes it possible for technology to enforce compliance in real time.
h. Self-service discovery: Big data tools and analytics have largely automated tests allowing
continuous monitoring without the need for an increase in IT audit human resources. These new
technologies can deliver results up to 90% faster than traditional tools.
i. Audit test automation: Even the most complex financial transactions can be modelled and
examined with the new analytic tools shortening audit cycles, lowering cost, and allowing deeper
j. Data reconciliation: Big data analytics tests and compares transactions across multiple systems
and simultaneously tests for compliance to policy and procedures.
k. Report verification: Analytics can compare the accuracy of reports prepared by separate systems
to test and ensure consistency and accuracy
l. Recalculation: Based on the analytics ability to model and replicate complex transactions and
postings, testing, reproducing, and confirming the accuracy of transactions under multiple scenario is
easy and quick
NEW EMERGING AREAS
a. Efficiency of execution: Big data technologies and analytics empower the auditors to constantly
monitor and recalibrate predictions based on enhanced ability to understand and analyse internal
processes, competitive pressures, and customer response. Big data also helps in quickly finding
answers to specific problems and modifying forward forecasts in near real time allows financial
professionals to play an influential role in steering their companies.
b. Growth and Innovation: The big data and analytic capabilities that evaluate efficiency also offer up
suggestions for growth opportunities in markets or complimentary products and services. Big data
enables deeper and broader insights through analytics that can be the difference between successful
and unsuccessful acquisition and business development strategies.
c. Valuation and customer equity: Traditional valuation techniques based on discounted free cash
flows can be greatly enhanced with a clear understanding of customer lifetime value. Big data and
analytics offers a clear path to better understanding customer behaviour, buying patterns, and wallet
share which can be aggregated up to customer equity measures.
d. Customer risk: The risk involved in the customer equity is customer risk. Big data and analytics
offers a path to better understanding of customer behaviour. Big data and analytics allows this with new
depth and accuracy and this data can be used to correlate these measures with cash flows, profit and
other financial measures to guide strategic decisions.
e. Reputational risk: Reputational risk runs parallel to the operational risk. This risk can arise from
customers, non-customers and even from employees. The audit professionals has to recognized the
need to constantly monitor and respond to the threats from social media which may be customer
comments about products, service, price of quality, but it can also be inappropriate leaks of sensitive or
incorrect private financial or transactional plans and data. Big data and analytics now offers a way for
financial professionals to develop and maintain a listening post to spot and defend against these risks
which have the potential materiality impact financial results.
ROLE OF BIG DATA IN FRAUD SPOTTING 
Detecting of fraud has never been an easy process. For many years, auditors have relied on the
whistle-blowers, or a wrong step taken by perpetrators or good amount of luck to uncover wrongdoing
both inside and outside the companies. In most of the cases, where the rule based queries were used
to detect fraud, it was easily evaded by smart thieves who know how to hide the numerical patterns.
In the rule based queries, many of the bribery schemes in the cases on record lasted for years before
they were detected, and some involved several executives. Similarly in many of the cases, detecting
improper payments largely comes down to what misguided employees enter into free-text field of a
payment description in the account payable system.
The traditional way of detecting fraud can be changing with the advent of big data era. Big data which
are high powered analytical tools can crunch through enormous quantities of structured and
unstructured data, producing an exponential greater set of comparisons within the data on any given
time. Big data also helps to analyse text and other unstructured data with e-mails and social media
activity yielding plenty of insights about what is going on beyond the numbers.
With the exponential growth of the business across the globe and with the huge data collected through
various means, big data is here to stay and it is the time the audit profession also take the leverage of
big data for the purpose of audit.