IS Audit Basics: Auditors and Large Software Projects, Part 3: Can Auditors Prevent Project Failure?

This column (and the previous two, published in the ISACA Journal volume 5¹) focuses on a serious concern to business managers: What causes large software projects to have huge cost and timescale overruns and/or fail to meet expectations or, at worst, be abandoned before completion?

Part 1 of these series explored three areas that appear in the early stages of a project: The business case, the project risk analysis and the requirements definition. Part 2 explored three key management (and political) decisions: whether to buy or build software, establishing the project plan, and selecting the project manager.

This column focuses on auditing how the inevitable changes to the project are managed: Poor change control is a frequent cause of projects going wrong. COBIT 5 devotes two sections (BAI06 and BAI07) to this topic.

Change Management (Also Referred to As Change Control)

ISACA’s Knowledge Center has an excellent article² on this topic that complements perfectly the change control process flow defined in the Information Technology Infrastructure Library version 3 (ITIL v3).³ There are also several web sites that describe in detail the life cycle of changes.⁴

Some 40 years ago, while working in a critical information infrastructure operating 24/7/365, our director was a very capable man and also a bit of a dictator. Change control was mandatory and nonnegotiable, and a homemade workflow management system was used to support this process.

Years later and in other organizations, it became apparent that not everyone shared the view that poor change control leads to firefighting in operational activities and problems in software development. At that time, many internal auditors were not yet practicing risk-based audit and were unfamiliar with ITIL, which was introduced in the UK in the early 1990s.

Since the change management life cycle is straightforward, it is not difficult to buy or design a suitable application (there are several commercial offerings for the reader to explore). However, implementing such a system without a change management policy is pointless.

The challenge is getting people to comply with this policy for all changes to configurations, systems, application software, access rights and system privileges, and project plans. The usual reaction is to criticize, object and obstruct the initiative (despite briefings and explanations of the advantages of implementing this practice). Examples of objections include: “I have been doing this work for 20 years and I know what I am doing—I do not need more bureaucracy.” Or, “UNIX programmers do not work like this.”

The person with the 20 years of experience was invited to leave the organization (for other reasons). This led to the discovery that critical data center processes had been customized (2 million lines of partially documented code containing logical bombs to prevent their removal). Several times, the external auditors had highlighted the risk associated with this “indispensable individual,” but management held this person in high esteem and had declined to act.

While removing this customization, the team understood the need for formalizing change managements and became its champion. Others had doubts until the day the global corporate network collapsed.

On a Monday, the day had started well, but by midmorning, the performance of the global network began to decline and then it died—nothing in and nothing out. A wide search for a possible cause was unsuccessful. Then a member of the team asked if anyone had been in the data center during the weekend. The data center’s access control revealed that the networking lead engineer, who thought change control was a waste of time, had spent a morning there before leaving on a walking vacation in Norway.

Just as well, cellular telephony enabled us to contact him. He said he had an idea to optimize the network’s router tables, and no, there was no record of it in the change control system because this was a simple change and there was no point in bureaucracy.

The network was reset to its original (documented) settings and everything was soon back to normal. After the networking engineer’s return, we held a postmortem of the incident and reiterated how change control records would have saved the organization and its many users a lot of anxiety and aggravation. Shortly after this, the engineer decided to pursue his career elsewhere and there was no more argument about the mandatory nature of the change control system.

Auditing Change Control Processes

There is no point in attempting to duplicate the set of excellent documents of which these are just a small sample:

• Change Control Audit Program and Internal Control Questionnaire⁵
• Change Management Audit/Assurance Program⁶
• “Change Management”⁷ guidelines from the Internal Audit Office, University of Queensland, Australia

Among other sources, there is a Change Management Body of Knowledge (CMBoK)⁸ that has valuable guidelines for practitioners and auditors. CMBoK also covers organizational change and emergency changes; the latter appears rarely in projects, but is common in IT operations.

Here a somewhat different audit perspective on change management, in particular on capability areas and their maturity levels, is presented.

The model described here is a composite of several good practices and has six critical capability areas:

• Leadership—Sponsoring the institutionalization of change management; demonstrable senior management engagement in the application of this discipline; and defining business rules, policies and procedures, and ensuring compliance with them
• Communications—Establishing a culture that recognizes the value of change management, that the organization shares a common definition of what change management is, and that its use is regularly evaluated and improved
• Application—Making resources available for the practice of change management and defining those areas and/or functions where a common approach is mandatory, aiming for uniformity in practices and tools
• Competencies—Providing training and documentation, encouraging interchanges between experienced practitioners and learners, ensuring project teams collaborate and share change management knowledge
• Authorities—Change management policies and procedures define the approval mechanisms for proposed changes depending on the criticality, complexity and impact of the proposed change. This should also provide clear definitions of the minimum requirements for segregation of duties (SoD).
• Standardization—Aiming to have a standard approach to change management and a standard set of tools to support it, integrating the tools with project delivery processes, and ensuring that expertise and advice on change management can be readily accessed and shared

Each of these headings can be split into individual lines of audit and their maturity assessed and then summarized in a table similar to the one shown in figure 1. The goal is to reach levels 4 and 5.

Conclusion

This column assumes that everyone shares the objective that projects should be completed on time and on budget and with functionality meeting expectations and causing no disruption.

However, despite progress in governance, risk management, project management and certifications, media constantly remind us that project overruns, operational disruptions and management frustration with IS/IT in their businesses still occur more frequently than one would wish.

Auditors who find that change management is not practiced as well as it ought to be should remind their auditees that those who go around looking for trouble usually find it. Thus, it is important to have the courage to raise the issue with senior management and the audit committee.

Endnotes

¹ Gelbstein, Ed; “Auditors and Large Software Projects, Part 1: Can Auditors Prevent Project Failure?,” and “Auditors and Large Software Projects, Part 2: Can Auditors Prevent Project Failure?,” ISACA Journal, vol 5., 2015, h04.v6pu.com/Journal/Pages/default.aspx
² ISACA, Change Management Audit/Assurance Program, USA, 2001, h04.v6pu.com/auditprograms
³ Axelos Ltd., ITIL v3, 2011, www.itil-officialsite.com/
⁴ Doherty, Peter; Peter Waterhouse; Change Management: A CA IT Service Management Process Map, CA Inc., June 2006, www.itsmcampus.com/ca_public/30264_change_mgmt_processmap.pdf
⁵ PRINCE2 Primer.com, Prince2 Video Primer, www.prince2primer.com/change-control-procedures
⁶ Op cit, ISACA
⁷ Internal Audit Office, “Change Management,” University of Queensland, Australia, 2002
⁸ Change Management Institute, The Effective Change Manager: The Change Management Body of Knowledge (CMBoK), http://www.change-management-institute.com/buycmbok

Ed Gelbstein, Ph.D., 1940–2015, worked in IS/IT in the private and public sectors in various countries for more than 50 years. Gelbstein did analog and digital development in the 1960s, incorporated digital computers in the control systems for continuous process in the late ‘60s and early ‘70s, and managed projects of increasing size and complexity until the early 1990s. In the 1990s, he became an executive at the preprivatized British Railways and then the United Nations global computing and data communications provider. Following his (semi) retirement from the UN, he joined the audit teams of the UN Board of Auditors and the French National Audit Office. Thanks to his generous spirit and prolific writing, his column will continue to be published in the ISACA Journal posthumously.