United States Department of Justice

Office of the Inspector General

Audit Division

A

UDIT

R

EPORT

F

EDERAL

B

UREAU OF

I

NVESTIGATION

’

S

M

ANAGEMENT OF

I

NFORMATION

T

ECHNOLOGY

I

NVESTMENTS

D

ECEMBER

2002

03-09

FEDERAL BUREAU OF INVESTIGATION’S MANAGEMENT

OF INFORMATION TECHNOLOGY INVESTMENTS

EXECUTIVE SUMMARY

Following the September 11, 2001, terrorist attacks, the Attorney

General and the Director of the Federal Bureau of Investigation (FBI)

made clear that prevention of terrorism is the top priority of the

Department of Justice (DOJ) and the FBI. Effective use of information

technology (IT) is crucial to the FBI’s ability to meet this priority as well

as its other critical responsibilities.

However, reviews conducted by the Office of the Inspector

General (OIG) and the General Accounting Office (GAO) have found

major weaknesses associated with the FBI’s IT. The FBI has listed

upgrading its information technology as one of its top ten highest

priorities. In June 2002 Congressional testimony, the FBI

acknowledged that its IT infrastructure is severely outdated.

Because of the importance of the FBI’s management of its IT

systems, we performed this audit to: (1) determine whether the FBI

was effectively managing its IT investments; and (2) assess the FBI’s

IT-related strategic planning and performance measurement activities.

1

We also examined the FBI’s efforts to develop enterprise architecture

2

and project management capabilities.

In this audit, we conducted approximately 85 interviews with

70 officials from the FBI, DOJ, GAO, and the Office of Management and

Budget (OMB). The FBI officials interviewed were from the Director’s

office, Information Resources Division, Criminal Justice Information

Services Division, Laboratory Division, Inspection Division, and Finance

1

During our audit fieldwork, we initiated work relating to a third objective: to

determine if the FBI has implemented prior information technology related

recommendations directed toward improving information technology. We will issue a

separate report on this objective.

2

Enterprise architecture is the organization-wide blueprint that defines an

entity’s functions and systems, including IT systems. It provides a comprehensive

view (through models, narratives, and diagrams) of the interrelationships of an

organization’s operations and structures and how these structures align with the

organization’s mission. The Clinger-Cohen Act of 1996 recognizes the

interrelationship between enterprise architecture and IT investment management by

requiring federal agencies to develop an enterprise architecture.

- i -

Division. Additionally, OIG auditors and analysts traveled to FBI

laboratory facilities in Quantico, VA, and five FBI field offices to

conduct interviews and assess the FBI’s implementation of IT

initiatives. We also reviewed more than 200 documents, including the

FBI’s IT management policies and procedures, project management

guidance, strategic and program plans, IT project proposals and

management plans, budget documentation, organizational structures,

Congressional testimony, and prior OIG and GAO reports.

1. Summary of Audit Findings

We concluded that the FBI has not effectively managed its IT

investments because it has not fully implemented the management

processes associated with successful IT investments. The foundation

for sound IT investment management (ITIM) includes the following

fundamental elements:

•

defining and developing IT investment boards;

•

following a disciplined process of tracking and overseeing each

project’s cost and schedule milestones over time;

•

identifying existing IT systems and projects;

•

identifying the business needs for each IT project; and

•

using defined processes to select new IT project proposals.

The FBI failed to implement these critical processes. We found

that the FBI does not have fully functioning IT investment boards that

are engaged in all phases of IT investment management. The FBI was

not following a disciplined process of tracking and overseeing each

project’s cost and schedule milestones. The FBI failed to document a

complete inventory of existing IT systems and projects, and did not

consistently identify the business needs for each IT project. The FBI

did not have a fully established process for selecting new IT project

proposals that considered both existing IT projects and new projects.

Because the FBI has not fully implemented the critical processes

associated with effective IT investment management, the FBI

continues to spend hundreds of millions of dollars on IT projects

without adequate assurance that these projects will meet their

intended goals.

- ii -

We concluded that these shortcomings primarily resulted from

the FBI not devoting sufficient management attention in the past to IT

investment management.

However, FBI management has recognized that its past methods

to manage IT projects have been deficient, and the FBI recently has

committed to changing those practices. In January 2002, the FBI

developed a conceptual model for selecting, controlling, and evaluating

IT investments. The model seeks to define a process that will promote

a Bureau-wide perspective on IT investment management, so that only

IT projects with the best probability of improving mission performance

are selected. Further, the process is intended to provide the methods,

structures, disciplines, and management framework that governs the

way IT projects are controlled and evaluated.

In addition to developing a conceptual model for a new ITIM

process, in early 2002 the FBI began a pilot test of the new process for

the selection of IT proposals. We found that the FBI made

improvements during the pilot testing of the new selection process.

Pursuant to the new process, the FBI created three IT investment

review boards that reviewed IT proposals for technical compliance and

“mission fit.” These boards, comprised of the FBI Director, FBI

executives and IT managers, selected new IT proposals that will be

considered for inclusion in the Fiscal Year (FY) 2004 budget request.

While the FBI has made efforts to improve its IT investment

management practices, the FBI must take further actions to ensure

that it can implement the fundamental processes necessary to build an

IT investment foundation, as well as the more mature processes

associated with highly effective IT investment management. These

actions include:

•

fully developing and documenting its new IT investment

management process – which is necessary to completely

implement the activities defined in the FBI’s conceptual model;

•

requiring increased participation from IT program managers and

users – which is necessary to ensure senior management

acceptance and foster understanding and institutionalization of

the ITIM process; and

•

further developing the FBI’s project management and enterprise

architecture functions – which is necessary to execute the

- iii -

control and evaluate components of the ITIM process as well as

advance its investment management capability.

Our audit also reviewed the FBI’s management of Trilogy, the

FBI’s largest and most critical IT project. We found that the lack of

critical IT investment management processes contributed to missed

milestones and led to uncertainties about cost, schedule, and technical

goals. Specifically, despite $78 million in additional funding, the FBI

missed its July 2002 milestone date for completing the physical IT

infrastructure upgrades to field offices, including new computer

hardware and networks.

3

FBI officials stated that they are not

expecting the physical infrastructure components of Trilogy to be

completed until March 2003. In addition, the user application

component of Trilogy, recognized by FBI officials as the most

important aspect of the project in terms of improving agent

performance, is at high risk of not being completed within the funding

levels appropriated by Congress. In our judgment, the management

problems associated with Trilogy demonstrate the FBI’s urgent need

for enhanced IT investment management.

We also concluded that the FBI’s IT strategic planning and IT

performance measurement are inadequate. We found that the FBI's

strategic plan does not include goals for IT investment management,

and the FBI’s strategic plan and performance plan are not consistent

with the DOJ’s annual performance plan.

The remainder of this executive summary provides more

background and details on our audit findings and recommendations to

help improve the FBI’s management of its IT investments.

2. Background

The Clinger-Cohen Act of 1996 requires each federal agency to

implement a process for maximizing the value of its IT investments.

This process is intended to ensure that IT projects are being

implemented at acceptable costs and within reasonable time frames,

and that the projects are contributing to enhanced mission

performance. Specifically, the Clinger-Cohen Act requires federal

agencies to: (1) develop an enterprise architecture framework, and

3

With the $78 million in additional funding, Trilogy’s total appropriation was

$458 million as of June 2002.

- iv -

(2) follow a “select/control/evaluate” approach to managing IT

investments.

In May 2000, the GAO developed the IT Investment

Management Framework (Framework) to provide a common

methodology for assessing IT capital planning and investment

management practices at federal agencies. The Framework specifically

describes the organizational processes required to carry out sound IT

investment management.

The Framework, based on best practices of leading

organizations, is a hierarchical model comprised of five maturity

stages. These maturity stages represent steps toward achieving stable

and mature investment management processes. As agencies advance

through these stages, their capability to effectively manage IT

increases. With the exception of the first stage, each maturity stage is

comprised of critical processes that must be implemented and

institutionalized for the agency to satisfy the requirements of that

stage. These critical processes are further broken down into key

practices an agency should perform to successfully implement each

critical process.

An agency using these critical processes is in a better position to

successfully invest in IT and use its IT investments to achieve its

priorities. Conversely, an agency that does not have these critical

processes in place is at high risk that its IT projects will fail to support

the achievement of priorities.

To determine whether the FBI was effectively managing its IT

investments, we utilized the Framework because it is: (1) a

standardized tool for internal and external evaluations of an agency’s

IT investment management process; (2) a consistent and

understandable mechanism for reporting the results of these

assessments; and (3) a road map agencies can use for improving their

IT investment management process.

In addition, the Government Performance and Results Act of

1993 (Results Act) requires strategic planning and performance

measurement throughout the federal government. The Results Act

seeks to improve the effectiveness, efficiency, and accountability of

federal programs by requiring federal agencies to establish goals for

program performance and measurement. The Results Act requires

agencies to prepare a strategic plan, annual performance plan, and

annual performance report.

- v -

While IT strategic planning is a function somewhat independent

of IT investment management, these two functions are interrelated

and complementary. The DOJ has recognized the importance of

integrating strategic planning with IT management. In July 2002, the

DOJ released its IT Strategic Plan that included a strategic initiative to

establish and improve investment management processes.

3. The FBI’s Management of IT Investments

Our audit found that the FBI has not established an IT

investment foundation and therefore is in Stage One maturity

according to the ITIM Framework. Stage One maturity is characterized

by inconsistent, unstructured, and unpredictable investment

processes. Our observations of the FBI’s IT investment processes

found that the FBI’s actual processes are consistent with these

Stage One deficiencies.

The critical processes necessary to establish an IT investment

foundation include: (1) defining investment review board operations,

(2) developing project-level investment control processes,

(3) identifying IT projects and systems, (4) identifying the business

needs for each IT project, and (5) developing a basic process for

selecting new IT proposals.

We found that the FBI failed to implement these critical

processes. The FBI did not have a fully established investment review

board operation because the FBI did not provide adequate resources

for operating the IT investment boards. Additionally, we found

insufficient evidence to demonstrate that: (1) organization executives

and line managers supported and carried out IT investment board

decisions and (2) board members understood the investment board’s

policies and procedures and exhibited core competencies in using the

IT investment approach via training, education, or experience.

Specifically, the FBI did not provide ample time to adequately prepare

and train IT board members prior to initiating the pilot test of its

recently developed ITIM process. This resulted in inadequate training

of board members and minimal preparation time to develop IT

proposals. For example, Technical Review Board members had only

three business days to review over 50 IT proposals prior to their first

board meeting.

Additionally, we found that the FBI is not effectively overseeing

its IT projects. For example, while the FBI has issued project

management guidance, the guidance is not being followed on a

- vi -

consistent basis. Depending on whom we talked to, we obtained

different answers as to which document represented the FBI’s official

project management guidance.

Without effective oversight of IT projects, FBI officials do not

have adequate assurance that IT projects are being developed on

schedule and within established budgets. According to a former Chief

Information Officer at the FBI, the lack of effective oversight of IT

projects has prevented IT project managers from being held

accountable for cost and schedule overruns and the ultimate

performance of projects. Senior FBI officials also told us that the

Bureau’s budget formulation process focuses only on the acquisition

costs for IT projects and not the full life-cycle costs, especially

operations and maintenance costs.

We also found that the FBI’s investment review boards are not

aware of all the IT projects and resources for which the boards are

responsible. FBI Divisions maintained some version of an IT inventory

for the projects and systems under their jurisdiction, and there was no

centralized office responsible for maintaining a uniform listing Bureau-

wide. FBI managers told us they were in the process of developing an

IT asset inventory, but at the time of our audit they were unable to

provide an estimated date for completing the inventory.

FBI personnel told us that staff shortages are the primary cause

for the incomplete IT asset inventory. In our judgment, staff

shortages may be a contributing factor, but the lack of centralized

management over IT investments was the significant reason for this

problem. Until June 2002, the FBI did not have a centralized project

management office to assist the investment boards in overseeing IT

projects. The FBI maintained three separate division-level project

management offices to manage IT projects.

We also determined that the FBI did not have a fully established

process for selecting IT proposals. FBI officials told us that, prior to

March 2002, individual divisions determined IT needs in a “stovepipe,”

without knowledge of the business needs and priorities of the Bureau

as a whole. The FBI did not have a clearly designated official to

manage the proposal selection process. According to Information

Resources Management Section personnel, the Finance Division

managed the IT selection process. However, according to Finance

Division personnel, the Information Resources Management office was

responsible for managing the proposal selection process.

- vii -

Without a comprehensive proposal selection process that

includes adequate resources and training, the FBI cannot ensure that it

is selecting the best IT projects that meet mission-critical needs.

Because the FBI did not fully implement any of the critical

processes associated with Stage Two, the FBI continues to spend

hundreds of millions of dollars on IT projects without having adequate

selection and project management controls in place to ensure that IT

projects will deliver their intended benefits.

The FBI began pilot testing the select phase of its new ITIM

process in March 2002, and since then has made measurable progress

towards implementing the key practices that comprise the critical

processes – particularly in the area of selecting new proposals for IT

projects. Specifically, at the beginning of our audit in January 2002,

the FBI only was executing 4 of the 38 required key practices;

however, as of June 2002, the FBI was executing 14 of the key

practices.

With the pilot testing of its new ITIM process, the FBI created an

IT investment process guide containing policies and procedures to

direct board operations, and created and defined three investment

review boards integrating both IT and business knowledge.

Additionally, the FBI has designated an official responsible for

managing the IT project and system identification process and

ensuring that the inventory meets the needs of the investment

management process. Further, during the test pilot of the ITIM

process, the board reviews of IT project proposals provided assurance

that business needs were clearly identified and defined. Also during

the test pilot, we determined that FBI IT investment board members

analyzed and prioritized new IT proposals according to established

selection criteria for the FY 2004 budget cycle.

Despite the progress made, full implementation of the ITIM

process will require the FBI to (1) fully develop and document its new

ITIM process; (2) require more input and participation from IT

managers and users; and (3) further develop its project management

and enterprise architecture functions. Completion of the initial steps

taken by the FBI will ensure that IT projects are developed within cost

and schedule requirements, and meet performance expectations. The

Trilogy project provides an example of how the non-implementation of

fundamental IT investment management practices can put a project at

risk of not delivering what was promised, within cost and schedule

requirements.

- viii -

4. Trilogy

We also performed a case study of the FBI’s implementation of

its Trilogy project. We selected Trilogy because it is the FBI’s largest

ongoing IT project and is considered vital to the FBI’s ability to

perform its mission. Trilogy is intended to upgrade the FBI’s:

(1) hardware and software – referred to as the Information

Presentation Component (IPC), (2) communication networks – referred

to as the Transportation Network Component (TNC), and (3) five most

important investigative applications – referred to as the User

Applications Component (UAC). The IPC and TNC upgrades will

provide the physical infrastructure needed to run the applications from

the UAC portion. The UAC portion is intended to upgrade and

consolidate five of the FBI’s 42 investigative applications. Because of

the 37 other investigative applications and approximately 160 non-

investigative applications that Trilogy will not cover, Trilogy is only a

starting point towards upgrading the FBI’s entire IT infrastructure.

According to the FBI, Trilogy is not designed to provide the FBI with

state-of-the-art IT; it is intended to provide the foundation so that the

FBI can eventually attain state-of-the-art IT.

In November 2000, Congress appropriated $100.7 million for the

first year of the $379.8 million Trilogy project, which was to be funded

over a three-year period (from the date contractors were hired). The

$100.7 million was a combination of new program funding and a

re-direction of base resources. When the FBI requested contractor

support for Trilogy, it combined the IPC and TNC portions for

continuity as both encompass physical IT infrastructure enhancements.

The contractor for the IPC/TNC portions was hired in May 2001, and

the originally scheduled completion date for these components was

May 2004. A different contractor was hired in June 2001 to complete

the UAC portion of Trilogy by June 2004.

After the terrorist attacks on September 11, 2001, the urgency

of completing Trilogy increased, and the FBI explored options to

accelerate the deployment of all three components of Trilogy. The FBI

informed Congress in February 2002 that, with an additional

$70 million, the FBI could accelerate the deployment of Trilogy. This

acceleration would include completion of the IPC/TNC phase by

July 2002 and rapid deployment of the most critical analytical tools

included as part of the UAC phase.

- ix -

In January 2002, Congress supplemented Trilogy’s FY 2002

budget with $78 million

4

to expedite the deployment of all three

components. This supplemental appropriation increased the total

funding of Trilogy from approximately $380 million to $458 million.

Even with these additional funds, the FBI missed its July 2002

milestone date for completing the IPC and TNC phases. FBI officials

stated that they are not expecting these components of Trilogy to be

completed until March 2003. In addition, the user application

component of Trilogy, recognized by FBI officials as the most

important aspect of the project in terms of improving agent

performance, is at high risk of not being completed within the funding

levels appropriated by Congress. Further, despite receiving an

additional $78 million from Congress in January 2002, FBI managers

have acknowledged to us that the last phase of UAC will not be

completed any sooner than originally planned (in June 2004).

In terms of a cost baseline, FBI officials told us that the rapid

procurement and deployment of Trilogy has prevented the project

managers from performing earned value management,

5

as promised

to Congress. While FBI officials were confident they know how much

money has been spent on Trilogy to date, and how much funding has

been committed, they have less assurance as to whether Trilogy is on

budget, over budget, or under budget.

A schedule baseline for Trilogy has never been well-established.

First, FBI officials said they would complete IPC/TNC deployment in

May 2004. Then, they said it could be finished in June 2003. Next,

they said it would be finished by December 2002. After receiving

$78 million of supplemental funding, they said it would be done by

July 2002. Then, they said they could not make the July 2002

deadline and moved it to October 2002. As of June 2002, FBI officials

have said deployment will probably not be complete until March 2003.

Also as of June 2002, the FBI was still in the process of building a

comprehensive schedule of Trilogy milestones.

Regarding the technical requirements for Trilogy, we were told

that some aspects of Trilogy as submitted to Congress did not turn out

to be technically feasible. For example, FBI officials told us that the

4

The $78 million is comprised of the $70 million that FBI requested for

acceleration, plus $8 million for contractor support.

5

Earned value management is a project monitoring method that compares

the value of products and services received with funds that have been expended.

- x -

thin-client strategy was not pursued because it was found that this

type of network could not be achieved given the technical

requirements of the FBI.

6

Another example is web-enablement of the

Automated Case Support (ACS) system, which was also discontinued

when it was realized that it would require more resources than

anticipated.

7

Had a more rigorous proposal selection process been in

place to require sufficient documentation of the technical requirements

and risks of the project, the expending of time and resources on thin-

client technology and web-enablement of ACS may have been

minimized.

Another technical issue involves the development of the UAC

portion of Trilogy. Because the UAC portion is focused on making

significant changes to, or possibly complete replacements of, five of

the FBI’s investigative systems, documentation for the exact

configuration of these systems is critical to designing the requirements

for UAC. According to a senior FBI official, the FBI must know what it

has before it can define the right solution to fix the problem. Lack of

documentation for the configuration of these five investigative systems

has caused the FBI to engage in a process of reverse engineering,

which is trying to determine the structure and components of the

systems after deployment. Because the FBI has to perform reverse

engineering on the FBI’s five investigative systems, there are

limitations as to how rapidly UAC can be developed and deployed.

Our observations at five FBI field offices indicated that

deployment of the IT physical infrastructure was still ongoing as of

June 2002. For two field offices, additional installation work remained

to be completed, and for four field offices hundreds of desktop

computers still remained to be delivered. A lack of clear

communication between FBI Headquarters and the field offices

contributed to the confusion over the number of desktop computers to

be delivered and shortages of fiber optic cable. Additionally contractor

maintenance support for the Trilogy architecture was inefficient,

resulting in agents being without computers for weeks at a time.

Improvements in agent and support personnel training, procurement

of trouble-shooting equipment for the Trilogy architecture, and timely

6

According to the FBI, a thin-client strategy would utilize application software

that is run from the server computer, and consequently permit desktop computers to

function with few hardware resources such as processors and memory.

7

Web-enablement refers to the ability of the software application to interface

with the Internet through a browser, thereby extending information access.

- xi -

completion of FBI unique macros for Microsoft Word will enhance user

utilization of the Trilogy architecture.

The new Trilogy project executive, hired in March 2002, has

taken a different approach to managing Trilogy. She has emphasized

the importance of having more structured oversight of the project.

She has been developing a comprehensive schedule for all three

components. Additionally, she has indicated that there are limitations

to how fast Trilogy can be deployed, without risking the security of the

system. In our judgment, while these actions taken since March 2002

represent positive changes to Trilogy’s project management function,

the project’s completion time, final cost, and ultimate performance

remain uncertain. Also, we concluded that for the Trilogy project

management function to be effective, it must include oversight from IT

investment review boards to provide much needed monitoring.

5. FBI’s IT Strategic Planning and Performance Measurement

We also assessed the FBI’s IT strategic planning and

performance measurement. We found that the FBI’s strategic plan

does not include IT investment management goals and the FBI’s

strategic plan and performance plan are not consistent with the DOJ’s

annual performance plan. Also, as of the end of June 2002, the FBI

did not have a current strategic plan dedicated to IT. Instead,

individual FBI divisions had program plans that included the use of IT

within particular programs.

This occurred because the FBI has not updated its strategic plan

since 1998, and its performance plan does not include the same

strategic objectives, goals, and strategies relating to IT as does the

DOJ's annual performance plan. We believe that the FBI will have

difficulty improving its IT investment management process without

incorporating it into the strategic plan. Additionally, without adequate

strategic planning and performance measurements, there is a

heightened risk that the FBI may not be appropriately allocating

resources to meet the DOJ’s strategic priorities.

In our judgment, the FBI must change the division-specific IT

focus and implement a Bureau-wide IT strategic plan. The purpose of

the FBI’s ITIM process is to move away from the decentralized IT focus

to a centralized one. As a result, we recommend that the FBI update

its IT strategic plan and performance plans to (1) fully integrate these

plans with the FBI’s ITIM process; and (2) include those performance

goals and indicators defined in the DOJ’s IT Strategic Plan.

- xii -

6. OIG Recommendations

In this report, we make 30 recommendations that focus on

specific and immediate steps the FBI should take to help improve its IT

investment management. These recommendations include:

•

Ensure that the FBI continues its efforts to establish a

comprehensive enterprise architecture that is integrated with the

ITIM process.

•

Require the ITIM Program Office to plan for and allocate

sufficient time for IT investment review board members and

other ITIM users to execute assigned responsibilities

competently.

•

Ensure that members of IT investment boards and other ITIM

users receive sufficient training to execute assigned

responsibilities effectively.

•

Ensure that official project management guidance is used for all

FBI IT projects through management oversight from the IT

investment review boards.

•

Ensure that each IT project has a project management plan,

approved by the IT investment review boards, that includes cost

and schedule controls.

•

Ensure that a complete IT asset inventory is developed, and

information from the IT asset inventory is made available to, and

used by, the IT investment review boards as necessary.

•

Ensure that the FBI develops written policies and procedures for

identifying the business needs (and the associated users) of each

IT project.

•

Ensure that identified users participate in project management

throughout a project's life-cycle.

•

Ensure that the policies and procedures of the ITIM process are

expanded, documented, and made available to ITIM users.

•

Ensure that the ITIM Program Office and the ITIM contractor

incorporate the input from various ITIM users through

- xiii -

working group sessions as the ITIM process is being further

developed and refined.

•

Ensure that the FBI develops and implements a specific plan

detailing how and when it will integrate the ITIM process with a

system development life-cycle methodology.

7. Conclusion

The underlying practices we assessed are fundamental to any

project management endeavor. However, the FBI has not executed

the majority of these tasks to select and manage its IT resources. For

example, organizational policies were not clearly established to ensure

that critical IT investment policies endure. Additionally, there were no

clearly defined, uniform procedures for project management, tracking

project performance, and taking corrective actions as necessary. Prior

to the development of its ITIM process in early 2002, the FBI did not

give sufficient attention to IT investment management. Since the FBI

developed its ITIM process in early 2002, it has focused more

management attention in this area and has made progress towards

attaining a basic IT investment management foundation. Despite the

progress, the FBI did not fully implement any of the critical processes

necessary to build an IT investment foundation. As a result, the FBI

continues to spend hundreds of millions of dollars on IT projects

without having adequate selection and project management controls in

place to ensure that IT projects will deliver their intended benefits.

- xiv -

TABLE OF CONTENTS

INTRODUCTION .............................................................................1

1. Background ..........................................................................1

2. The FBI’s Management of IT Infrastructure ................................2

3.

Prior Reports on the FBI’s IT and DOJ Oversight of

Components’ IT .....................................................................4

4.

The FBI’s Current IT Investment Efforts ....................................9

5. Trilogy: The FBI’s Largest IT Investment................................ 10

6. Framework for Assessing IT Investment Management ............... 12

7. The DOJ’s ITIM Guidance ...................................................... 17

8. The FBI’s Recent Efforts to Implement an ITIM Process ............. 18

OIG FINDINGS AND RECOMMENDATIONS ....................................... 22

1. The FBI’s Management of IT Investments................................ 22

A. The FBI’s Progress Toward Attaining a Basic IT

Investment Management Foundation................................. 22

B. The FBI’s Ability to Improve its IT Investment

Practices ....................................................................... 60

C. Trilogy Case Study .......................................................... 86

2. The FBI’s IT Strategic Planning and Performance

Measurement .................................................................... 114

A. Background on Strategic Planning ................................... 114

B. Strategic Planning’s Relationship to the ITIM Process......... 116

C. Results of our Assessment of the FBI’s IT Strategic

Planning and Performance Measurement.......................... 117

D. Summary .................................................................... 118

E. Recommendation ......................................................... 118

STATEMENT ON COMPLIANCE WITH LAWS AND REGULATIONS ........ 119

STATEMENT ON MANAGEMENT CONTROLS .................................... 120

APPENDIX 1: OBJECTIVES, SCOPE, AND METHODOLOGY................ 121

APPENDIX 2: FLOWCHART OF FBI’S ITIM CONTROL PHASE ............. 125

APPENDIX 3: FLOWCHART OF FBI’S ITIM EVALUATE PHASE ............ 126

APPENDIX 4: JMD’S ASSESSMENT OF THE FBI’S ITIM

PROCESS ............................................................. 127

APPENDIX 5: GAO’S FIVE STAGES OF ENTERPRISE

ARCHITECTURE MATURITY...................................... 133

APPENDIX 6: FBI’S ENTERPRISE ARCHITECTURE MATURITY

SURVEY ............................................................... 135

APPENDIX 7: FBI

’

S RESPONSE TO THE DRAFT REPORT .................. 136

APPENDIX 8: OIG, AUDIT DIVISION ANALYSES AND

SUMMARY OF ACTIONS NECESSARY TO

TO CLOSE REPORT ................................................ 153

INTRODUCTION

1. Background

The Federal Bureau of Investigation (FBI or Bureau) is the

principal investigative arm of the Department of Justice (DOJ). To

execute its responsibilities, the FBI’s Headquarters in Washington, D.C.

provides program direction and support services to 56 field offices,

approximately 400 satellite offices known as resident agencies and

more than 40 foreign liaison posts.

As of June 2002, the FBI had over 11,000 Special Agents and

over 16,000 other employees who performed professional,

administrative, technical, clerical, craft, trade, or maintenance

operations. The FBI’s budget authority increased 31 percent from

$3.339 billion in FY 2001 to nearly $4.371 billion in FY 2002.

8

Of this

budget authority, $714 million was allocated to information technology

(IT) projects in FY 2002 compared to $353 million in FY 2001.

The terrorist attacks of September 11, 2001, prompted the

Attorney General to make counterterrorism the DOJ’s highest priority.

The DOJ reflected these new priorities in its Strategic Plan for Fiscal

Years 2001 – 2006, which was issued in November 2001. In the

Strategic Plan, the Attorney General recognized that the fight against

terrorism requires the DOJ “to improve the integrity and security of its

computer systems and make more effective use of information

technology.”

In response to the DOJ’s new priorities following September 11,

2001, the FBI proposed fundamental changes in its strategic priorities

and business practices. In May 2002, the Director of the FBI

announced a major reorganization that dedicates more resources to

the prevention of terrorism.

9

Although the core missions of the FBI

remain intact, the proposed changes would transform the Bureau’s

role from reactive to preventive. To accomplish this transition, FBI

officials have repeatedly told Congress that new and improved IT is

required to support a redesigned and refocused FBI. In testimony

8

These figures were taken from the DOJ’s website (

www.usdoj.gov

). They

include a $745 million Counterterrorism Supplemental for FY 2002 and exclude

Federal Retiree and Health Benefit Costs.

9

This reorganization was approved by Congress on July 31, 2002.

- 1 -

before the Senate Judiciary Committee on June 6, 2002, the Director

released the FBI’s top ten priorities in the post-September 11 era, with

the number one priority being protecting the United States from

terrorist attacks. Number ten on the list of priorities is upgrading

technology to successfully perform the FBI’s mission. Clearly, the

FBI’s future ability to prevent terrorism and other crimes depends on

modern information technology and effective management of

technology.

2. The FBI’s Management of IT Infrastructure

The FBI has three divisions that manage major IT projects: the

Information Resources Division (IRD), the Criminal Justice Information

Services Division (CJIS), and the Laboratory Division. As discussed

below, the FBI is attempting to centralize the management of IT,

rather than manage IT within divisions.

The IRD provides the day-to-day support services to manage the

information systems of the FBI. The IRD’s responsibilities include

management of all hardware, software, and IT peripheral equipment

located at the FBI’s Headquarters, field offices, and other offsite

locations.

The IRD has been restructured in recent years to increase the

oversight and jurisdiction of the Chief Information Officer. Until

November 2001, the Chief Information Officer of the FBI was the

Assistant Director of IRD who reported to the Director. However, to

give the Chief Information Officer greater authority over the entire

FBI, the Chief Information Officer was moved out of IRD and into the

Director’s office, pursuant to a restructuring approved by Congress on

November 30, 2001. Additionally, to support the Chief Information

Officer, the Information Resources Management Section

10

was moved

out of IRD and into the Chief Information Officer’s office, following

another restructuring in February 2002. Also, in February 2002, the IT

Investment Management Program Office was formed (within the

Information Resources Management Section) and was staffed with one

individual whose responsibility was to manage the FBI’s IT investment

management program. Based on these actions, the FBI recognizes

that centralizing the management of IT requires a Chief Information

Officer to have Bureau-wide oversight and jurisdiction, rather than be

isolated within a division.

10

The Information Resources Management Section is responsible for

managing IT investments and enterprise architecture.

- 2 -

The CJIS Division uses several significant IT systems to manage

and disseminate relevant criminal justice information to the FBI and

other law enforcement agencies. For example, the

National Crime Information Center 2000 is a nationwide information

system that supports federal, state, and local law enforcement

agencies. Additionally, the CJIS Division is responsible for

managing

the Integrated Automated Fingerprint Identification System and the

National Incident-Based Reporting System. To support the

management of these systems, the CJIS Division maintains a

Contract Administration Office, which provides quality assurance,

configuration management, and project management support services

necessary to manage these and other systems under its jurisdiction.

The Laboratory Division manages several forensic computer

systems that provide forensic and technical services to law

enforcement agencies. A significant system includes the Combined

DNA Index System (CODIS), which provides software and support

services to state and local laboratories to establish databases of

criminals, unsolved crime scenes, and missing persons. A component

of CODIS, the National DNA Index System, shares DNA profiles from

convicted offenders and crime scenes to laboratories throughout the

United States. To manage these systems, the Laboratory Division

maintains its own project management office.

The FBI has recognized that its IT infrastructure was significantly

outdated and did not effectively support user needs. Although recent

upgrades have changed these numbers, as of September 2000, over

13,000 desktop computers were 4 to 8 years old and could not run

basic software packages, some communication networks were up to

12 years old and were obsolete, and multiple user-applications existed

that were neither web-enabled

11

nor user-friendly.

12

On June 6, 2002,

the Director stated to the Senate Judiciary Committee:

You’ve heard me talk about the necessity for upgrading our

technology. And upgrading our technology means not just

getting the computers on board, the hard drives. It means

everybody from top to bottom becoming facile with the

11

Web-enablement refers to the ability of the software application to interface

with the Internet through a browser, thereby extending information access.

12

According to FBI officials, the FBI acknowledged these needs to Congress in

the late 1990s, in addition to the technology upgrade plan prepared in September

2000.

- 3 -

computer, understanding the computer and understanding

how technology can assist us to do our jobs better. And

that is somewhat of a transformation for an organization

such as the FBI, which is years behind where it should be,

in terms of having the technological infrastructure.

3. Prior Reports on the FBI’s IT and DOJ Oversight of

Components’ IT

Reports issued by the Office of the Inspector General (OIG) over

the past 12 years have highlighted many IT inefficiencies at the FBI.

In 1990, the OIG issued a report entitled, “The FBI’s Automatic Data

Processing General Controls.” This report found

11 major internal control weaknesses, many of which are still

applicable today. Specifically the report stated that:

•

the FBI’s phased implementation of its 10-year Long Range

Automation Strategy, scheduled for completion in 1990, was

severely behind schedule and may not be accomplished;

•

the FBI’s Information Resources Management program was

fragmented and ineffective, and the FBI’s Information Resources

Management official did not have effective organization-wide

authority;

•

the FBI had not developed and implemented a data architecture;

•

the FBI had not adequately involved top management in FBI

Headquarters or the field offices in systems development

through an Executive Review Committee; and

•

the FBI’s major mainframe investigative systems were labor

intensive, complex, untimely, and non-user friendly and few

Special Agents used these systems.

Regarding the first weakness, the FBI’s IT infrastructure is still

severely outdated, as we previously mentioned. Regarding the second

weakness, the FBI has recently restructured the IRD and Information

Resources Management Section to reduce the fragmented

management structure that existed among the three divisions

responsible for managing IT. Regarding the third weakness, as

discussed later in the report, the FBI is still developing an enterprise

architecture framework, which includes the technical or data

architecture. Regarding the fourth weakness, as discussed later in the

- 4 -

report, the FBI did not have formally established IT investment review

boards or committees until March 2002. Regarding the fifth weakness,

the FBI’s major investigative systems remain labor intensive, complex,

non-user friendly, and many Special Agents still do not use these

systems.

The OIG’s July 1999 special report on the handling of intelligence

information related to the DOJ’s campaign finance task force

13

stated

that FBI personnel were not well versed in the Automated Case

Support (ACS) system

14

and other databases. Additionally, a

November 1999 report on the death of a federal inmate, Kenneth

Michael Trentadue, noted deficiencies in uploading key evidence into

the ACS.

A March 2002 report entitled, “An Investigation of the Belated

Production of Documents in the Oklahoma City Bombing Case,”

analyzed the causes for the belated production of many documents in

the Oklahoma City bombing case. This report concluded that the ACS

system is extraordinarily difficult to use, has significant deficiencies,

and is not the vehicle for moving the FBI into the 21

st

century. The

report noted that inefficiencies and complexities with the ACS

combined with the lack of a true information management system

were contributing factors in the FBI’s failure to provide hundreds of

investigative documents to the defendants in the Oklahoma City

Bombing Case. These reports illustrate that the FBI has not given

sufficient attention to correcting its deficiencies in information

management and the ACS.

In May 2002, pursuant to the FY 2002 Government Information

Security Reform Act, the OIG issued a report on the FBI’s

administrative and investigative mainframe systems. This report

identified continued vulnerabilities with management, operational, and

technical controls. Significant vulnerabilities were noted in the

following areas:

13

The report, “Handling of FBI Intelligence Information Related to the Justice

Department’s Campaign Finance Investigation,” was issued in July 1999.

14

The ACS is the FBI’s primary investigative computer application that

uploads and stores case files electronically.

- 5 -

•

security policies, procedures, standards, and guidelines;

•

physical controls;

•

system and network backup and restoration controls;

•

password management;

•

logon management;

•

account integrity management;

•

system auditing management; and

•

system patches.

The report stated that these vulnerabilities occurred because the

DOJ and FBI security management had not enforced compliance with

existing security policies, developed a complete set of policies to

effectively secure the administrative and investigative mainframes, or

held FBI personnel responsible for timely correction of recurring

findings. Further, the report indicated that FBI management has been

slow to correct identified weaknesses and implement corrective action.

Therefore, many of these deficiencies repeat year after year in

subsequent audits.

In March 2002, the Commission for the Review of FBI Security

Programs issued a report titled, “A Review of FBI Security Programs.”

This Commission, chaired by former FBI Director William H. Webster,

was established to investigate the espionage of a FBI Supervisory

Special Agent, Robert Hanssen.

15

The report identified a wide range of

problems affecting the FBI’s computer systems and information

security policies, including the following:

• Classified information had been moved into systems not

properly accredited for its protection.

15

According to the report, over a period of 22 years, Robert Hanssen gave

the Soviet Union and Russia vast quantities of documents and computer diskettes

filled with national security information of incalculable value.

- 6 -

•

Until recently, the FBI had not begun to certify and accredit most

of its computer systems, including many classified systems.

•

Inadequate physical protections placed electronically stored

information at risk of compromise.

•

The FBI’s approach to system design has been deficient. It has

failed to ascertain the security requirements of the “owners” of

information on its systems and identify the threats and

vulnerabilities that must be countered.

•

Classified information stored on some of the FBI’s most widely

utilized systems was not adequately protected because computer

users lacked sufficient guidance about critical security features.

•

Some FBI inspectors had insufficient resources to perform

required audits. When audits were performed, audit logs

were reviewed sporadically, if at all.

According to the report, these findings resulted from the FBI’s lack of

attention to IT security in developing and managing computer

systems.

16

Additionally, the General Accounting Office (GAO) has issued

several reports and related testimony that highlight deficiencies with

the FBI’s IT. In June 2002, the Comptroller General provided the

following testimony before a subcommittee of the United States House

of Representatives Appropriations Committee:

Communications has been a longstanding problem for the

FBI. This problem has included antiquated computer

hardware and software, including the lack of a fully

functional e-mail system. These deficiencies serve to

significantly hamper the FBI’s ability to share important

and time sensitive information with the rest of the FBI

across other intelligence and law enforcement agencies.

We [the GAO] do not believe the FBI will be able to

successfully change its mission and effectively transform

itself without significantly upgrading its communications

16

Although the focus of our audit does not assess the FBI’s IT security

practices, the two prior reports mentioned above indicate that the FBI’s effective use

of IT must address information assurance as part of an overall IT governance model.

- 7 -

and information technology capabilities. This is critical,

and it will take time and money to successfully address.

17

In a review of the DOJ’s Campaign Finance Task Force, the GAO

reported in May 2002 that the FBI lacked an adequate information

system that could manage and interrelate the evidence that had been

gathered in relation to the Task Force’s investigations.

18

Also, as part

of a government-wide assessment of federal agencies, the GAO

reported in February

2002 that the FBI needed to fully establish the

management foundation that is necessary to

successfully develop,

implement, and maintain an enterprise architecture.

19

The deficiencies in IT management are not solely attributable to

the FBI itself, but are also attributable in part to DOJ actions. In

December 2000, the GAO issued a report on the Immigration and

Naturalization Service’s (INS) investment management capability.

20

This report stated that the DOJ was not guiding and overseeing the

INS’s IT investment management (ITIM) approach. The report

highlighted the DOJ’s responsibility, as required by the Clinger-Cohen

Act of 1996, to ensure that its components implement an effective

ITIM process. According to the report, the DOJ had not provided the

INS, or any other component, sufficient direction, guidance, and

oversight of ITIM activities. Further, the report stated:

While Justice [the Department of Justice] issued guidance

in January 2000 describing its high-level investment

management process, the guidance does not address the

need or requirements for Justice’s components to

implement an IT investment management process.

Specifically, this guidance does not instruct the

components to establish IT investment management

processes nor does it establish expectations for doing so.

Until Justice issues its policy and guidance and begins

monitoring its components’ progress, it has no assurance

17

This testimony, titled “FBI REORGANIZATION: Initial Steps Encouraging

but Broad Transformation Needed” (GAO-02-865T), was released on June 21, 2002.

18

This report, titled “CAMPAIGN FINANCE TASK FORCE: Problems and

Disagreements Initially Hampered Justice’s Investigation” (GAO/GGD-00-101BR),

was released on May 31, 2000.

19

This GAO report is discussed later in this report.

20

“INFORMATON TECHNOLOGY: INS Needs to Strengthen Its Investment

Management Capability” (GAO-01-146) was issued by the GAO in December 2000.

- 8 -

that it has the necessary investment management

processes in place to maximize the value of its IT

investments and manage the risks associated with the

investments.

The DOJ issued ITIM guidance in August 2001 and required the

components to develop an ITIM process by January 2002. This

guidance, and the FBI’s ITIM process, are further discussed later in

this introduction.

4. The FBI’s Current IT Investment Efforts

In a statement before the House Subcommittee on

Appropriations in March 2002, FBI Director Mueller stated: “Without

question, we all believe [information infrastructure] is the number one

problem confronting the FBI today, recognize that for a number of

reasons the situation developed over time, and know that in the future

a better approach to technology upgrades must be used.”

In the FBI Information Technology Upgrade Plan (FITUP),

prepared and submitted to Congress in September 2000, the Bureau

stated that a lack of funding was the cause for not making meaningful

upgrades to its IT infrastructure since 1994. Congress responded to

this concern by appropriating a total of approximately $2.2 billion for

FBI IT projects and systems for FYs 1997 to 2002.

21

The FBI received

$335.6 million of this amount in January 2002 from the Emergency

Supplemental Appropriations Act for information technology. The

following table summarizes the funds appropriated for FBI IT

investments since FY 1997.

21

This appropriation includes operation and maintenance costs of existing IT

systems, enhancements to existing IT systems, and funding for new IT projects. The

appropriation also includes personnel costs for managing the IT projects and

systems.

- 9 -

Funds Appropriated for FBI IT Investments Since FY 1997

Fiscal Year

Total IT Investments

(in millions)

2002

$714.0

2001

$352.8

2000

$293.0

1999

$332.0

1998

$241.2

1997

$309.2

Total

$2,242.2

Source: Exhibit 53s

22

prepared by the FBI

The FBI has several critical initiatives underway to upgrade its

infrastructure and investigation applications. Additionally, the FBI has

undertaken a major hiring initiative to recruit private sector IT experts

who can assist in designing and managing the sizable IT projects

recently funded by Congress. For example, the FBI’s last two Chief

Information Officers were hired from the private sector. Also, in

March 2002, the FBI announced the hiring of a project executive from

the private sector to manage Trilogy. Further, in June 2002, the FBI

announced the hiring of an executive from the private sector to

become the new Executive Assistant Director for Administration.

5. Trilogy: The FBI’s Largest IT Investment

Currently, the FBI’s largest IT project designed to improve IT

infrastructure and office automation is the Trilogy project, formerly

known as the FITUP. In September 2000, the FITUP was established

to enhance the investigative support for FBI agents. The FITUP noted

the following IT needs:

22

The Exhibit 53 for each fiscal year lists funds appropriated for major IT

projects. The FBI prepares the Exhibit 53 and submits it to the DOJ, which submits

it to the Office of Management and Budget (OMB). Total IT investments include

operation and maintenance costs of existing IT systems, enhancements to existing IT

systems, and funding for new IT projects. These investment costs also include

personnel costs associated with managing IT projects and systems.

- 10 -

•

getting all case files into electronic databases (since the ACS is

not consistently used);

•

making IT more user friendly for agents;

•

providing access to all databases via one search engine; and

•

providing reliable, high-speed flexible communications.

To address the above needs, the FITUP, renamed to Trilogy, is

intended to upgrade the FBI’s: (1) hardware and software – referred

to as the Information Presentation Component (IPC),

(2) communication networks – referred to as the Transportation

Network Component (TNC), and (3) five most important investigative

applications – referred to as the User Applications Component (UAC).

The IPC and TNC upgrades will provide the physical infrastructure

needed to run the applications from the UAC portion of Trilogy. The

UAC portion is intended to upgrade and consolidate five of the FBI’s

42 investigative applications. Because there are 37 other investigative

applications and approximately 160 non-investigative applications that

Trilogy will not address, Trilogy is only a starting point towards

upgrading the FBI’s entire IT infrastructure.

In November 2000, Congress appropriated $100.7 million for the

first year of the $379.8 million Trilogy project, which was to be funded

over a three-year period (from the date contractors were hired). The

$100.7 million was a combination of new program funding and a

re-direction of base resources. The FBI combined the IPC and TNC

portions for continuity when it requested contractor support, since

both encompass physical IT infrastructure enhancements. The

contractor for the IPC/TNC portions was hired in May 2001. As a

result, the originally scheduled completion date for these initiatives

was May 2004. A separate contractor was hired in June 2001 to

complete the UAC portion of Trilogy by June 2004.

After the terrorist attacks on September 11, 2001, the

importance of giving FBI agents and analysts the technological tools

necessary to perform their duties was heightened in the eyes of

Congress, the Attorney General, and the Director. Because the goal of

Trilogy is to address many of the technological needs of the FBI,

successful completion of the project in the shortest amount of time

possible was viewed as increasingly critical to the FBI’s fight against

terrorism. Rather than wait three years for the benefits of Trilogy,

Congress fully funded the FBI’s original request of $379.8 million and

- 11 -

provided an additional $78 million in January 2002 to speed up its

deployment.

23

With the supplemental funding, the FBI indicated to

Congress that it would complete the deployment of hardware

(including new desktop computers), networks, and software by

July 2002. Additionally, the FBI would seek to accelerate upgrades to

the five user applications. However, as discussed later in this report,

the FBI did not meet its July 2002 milestone and is not expecting to

complete the deployment of hardware, software, and networks until

March 2003.

Although we believe the FBI must have sufficient resources to

upgrade its technology through Trilogy and other projects, it must also

have the management processes in place to effectively utilize those

resources. With the recent influx of funding to the FBI, Congress

expects the FBI to make significant strides in upgrading its IT

infrastructure. But we believe the FBI will be successful in doing so

only if it has effective IT management control processes in place.

Later in this report, we provide an assessment of the FBI’s

management of Trilogy.

6. Framework for Assessing IT Investment Management

Several recent management reforms have required federal

agencies to improve their management processes for selecting and

managing IT investments. In particular, the Clinger-Cohen Act of

1996 requires the head of each agency to implement a process for

maximizing the value of the agency's IT investments and for assessing

and managing the risks of its acquisitions. A key goal of the

Clinger-Cohen Act is for agencies to have processes in place to ensure

that IT projects are being implemented at acceptable costs and within

reasonable time frames, and that the projects are contributing to

tangible, observable improvements in mission performance.

The Clinger-Cohen Act defines requirements for capital planning

and control of IT investments and mandates a select/control/evaluate

approach that federal agencies must follow. The following graphic

describes the fundamental phases of this IT investment approach.

23

The $78 million was part of the $745 million received from the Emergency

Supplemental Appropriations Act.

- 12 -

Fundamental Phases of the IT Investment Approach

Select

Phase

•

Screen

• Rank

• Select

Evaluate

Phase

•

Conduct

reviews

• Make adjustments

• Apply lessons

learned

How are you

ensuring

that projects

deliver

benefits?

?

?

How do you know

you have selected

the best projects?

?

Are the systems

delivering what

you expected?

Control

Phase

•

Monitor

progress

• Take

corrective

actions

DATA

Source: GAO

According to a GAO report, while almost all federal agencies

have created some type of IT investment management process, none

has implemented stable processes that address all three phases of the

select/control/evaluate approach.

24

One barrier to implementing

stable IT investment processes has been the lack of specific guidance

regarding what processes are required to build a stable, reliable IT

investment management organization. The select/control/evaluate

approach provides sound advice, but it does not provide a

comprehensive discussion of the organizational processes involved.

To address this concern, in May 2000 the GAO developed the

IT Investment Management Framework (Framework) to provide a

common methodology for discussing and assessing IT capital planning

and investment management practices at federal agencies. The

Framework enhances previous federal IT investment management

guidance by embedding the select/control/evaluate approach within a

framework that explicitly describes the organizational processes

required to carry out good IT investment management.

- 13 -

24

“Information Technology Investment Management: An Overview of GAO’s

Assessment Framework” (GAO/AIMD-00-155) was issued in May 2000.

The Framework, based on best practices of leading

organizations, is a hierarchical model comprising of five maturity

stages. These maturity stages represent steps toward achieving stable

and mature investment management processes. Each stage builds

upon the lower stages and enhances the organization's ability to

manage its investments. As agencies advance through these stages,

the agencies’ capability to effectively manage IT increases. The

following graphic describes the five maturity stages of the Framework.

The Five Maturity Stages of the ITIM Framework

Source: GAO

There is little awareness of investment

management techniques. IT management

processes are ad hoc, project-centric, and

have widely variable outcomes.

Repeatable investment control techniques are in

place and the key foundation capabilities have

been implemented.

Comprehensive IT investment portfolio selection

and control techniques are in place that

incorporate benefit and risk criteria linked to

mission goals and strategies.

Description

Investment benchmarking and IT-enabled

change management techniques are deployed

to strategically shape business outcomes.

Process evaluation techniques focus on

improving the performance and management

of the organization's IT investment portfolio.

Enterprise

and Strategic

Focus

Project-

Centric

Stage 4

Improving the

Investment Process

Stage 3

Developing a Complete

Investment Portfolio

Stage 2

Building the

Investment Foundation

Stage 1

Creating Investment

Awareness

Stage 5

Leveraging IT for

Strategic Outcomes

Maturity Stages

With the exception of the first stage, each maturity stage is

composed of critical processes that must be implemented and

institutionalized for the organization to satisfy the requirements of that

stage. These critical processes are further broken down into key

practices that describe the types of activities that an agency should be

engaged in to successfully implement each critical process. An

organization that has these critical processes in place is in a better

position to successfully invest in IT. The following graphic describes

the Framework’s five stages and associated critical processes.

- 14 -

The ITIM Framework’s Stages of Maturity with Critical

Processes

Investment Process Benchmarking

IT-Driven Strategic Business Change

Post-Implementation Reviews and Feedback

Portfolio Performance Evaluation and Im provement

Systems and Technology Succession Management

Authority Alignm ent of IT Inve stme nt Boards

P ortfolio S ele ction Criteria De finition

Investment Analysis

Portfolio Development

Portfolio Performance Oversight

IT Investm ent Board Operation

IT Asset Tracking

IT P roject Ove rsight

Business Needs Identification for IT Projects

Proposal Selection

IT Spending without Disciplined Investment

Processes

Stage 4

Improving the

Investment

Process

Stage 3

Developing

a Complete

Investm ent P ortfolio

Stage 2

Building the

Investment

Foundation

Stage 1

Creating

Investment

Awareness

Stage 5

Leveraging IT

for S trategic

O utcomes

Maturity Stages

Critical

Processes

Source: GAO

As established by the Framework, each critical process contains

five core elements that indicate whether the implementation and

institutionalization of a process can be effective and repeated. The

five core elements are:

• Purpose: This element is the primary reason for engaging in

the critical process and states the desired outcome for the

critical process.

•

Organizational commitment: This element comprises

management actions that ensure that the critical process is

established and will endure. Key practices typically involve

establishing organizational policies and engaging senior

management sponsorship.

•

Prerequisites: These elements are the conditions that must

exist within an organization to successfully implement a critical

process. These conditions typically involve allocating resources,

establishing organizational structures, and providing training.

- 15 -

•

Activities: These elements are the key practices necessary to

implement a critical process. An activity occurs over time and

has recognizable results. Key practices typically involve

establishing procedures, performing and tracking the work, and

taking corrective actions as necessary.

•

Evidence of performance: This element comprises artifacts,

documents, or other evidence that supports a contention that

the key practices within a critical process have been or are being

implemented. This core element typically consists of the

collection and verification of physical, documentary, or

testimonial evidence and often involves reviews by objective

parties.

With the exception of the “purpose” core element, each of the

other core elements contains key practices. The key practices are the

attributes and activities that contribute most to the effective

implementation and institutionalization of a critical process. The

following graphic summarizes the interrelationships of components in

an ITIM critical process.

- 16 -

Components of an ITIM Critical Process

Purpose

This is the primary reason for engaging in the critical process

and states the desired outcome for the critical process.

Prerequisites

These are the conditions that must

exist within an organization to

successfully implement a critical

process. This core element

typically involves allocating

resources, establishing

organizational structures, and

providing training.

Activities

These are the key practices

necessary to i mplement a critical

process. An activity occurs over time

and has recognizable results. Key

practices within this core element

typically involve establishing

procedures, performing and tracking

the work, and taking corrective

actions as necessary.

Evidence of

Performance

These are artifacts, documents, or

other evidence that support a

contention that the key practices

within a critical process have or are

being implemented. This core

element typically consists of the

collection and verification of

physical, documentary, or

testimonial evidence and typically

involves reviews by objective

parties.

Organizational Commitment

These are management actions that ensure that the critical

process is established and will endure. Key practices within

this core element typically involve establishing

organizational policies and engaging senior management

sponsorship.

Source: GAO

7. The DOJ’s ITIM Guidance

In August 2001, the DOJ’s Justice Management Division (JMD)

issued the Guide to the Department of Justice Information Technology

Investment Management Process (Guide). In response to various

regulations and guidelines issued in the last several years (including

the Clinger-Cohen Act, Executive Order 13011, and the

Office of Management and Budget (OMB) Circular A-130), the DOJ

issued the Guide to fulfill its obligation and responsibility to make

measurable improvements in mission performance and service delivery

to the public through the strategic application of IT.

The Guide uses the select/control/evaluate methodology to

implement the strategic and performance directives of the

Clinger-Cohen Act and other statutory provisions affecting IT

investments. The Guide is intended to promote a process that builds

on existing structures to provide maximum benefit across the entire

DOJ and with other federal agencies. This process allows the DOJ to

focus IT management on the strategic missions of the DOJ. Further, it

- 17 -

promotes an investment review process that drives budget formulation

and execution for information systems, and restructures the way the

DOJ performs its functions before investing in IT. In addition, this

process provides the methods, structures, disciplines, and

management framework that govern the way IT is deployed

throughout the DOJ. The Guide applies to all IT projects from all DOJ

components.

The Guide requires each component to:

•

designate a component Chief Information Officer consistent with

the DOJ’s ITIM policy;

•

establish an Executive Review Board that will approve the entire

component IT portfolio and oversee the decisions made about

specific investments; and

•

establish a component ITIM process that incorporates the DOJ’s

ITIM process, but is customized to function within the

component’s unique environment.

Further, by January 2002 each component was required to

submit to the DOJ an ITIM plan incorporating the above stipulations.

8. The FBI’s Recent Efforts to Implement an ITIM Process

In an effort to improve its IT investment management practices

and comply with DOJ and other statutory regulations, the FBI

developed the “ITIM Model and Transition Plan” (Plan) with support

from a contractor. The initial draft of the Plan was completed and

submitted to JMD in January 2002. The FBI has retained this

contractor to assist in the ongoing implementation of the ITIM process.

The FBI estimates total costs for developing its ITIM process will be in

excess of $4 million through FY 2003.

The purpose of the Plan is to establish and define the FBI’s

Stage Two

25

methodology and build the foundation for enhanced IT

investment management. It identifies the gaps between the FBI’s

current IT investment processes and the required IT management

practices for Stage Two maturity.

25

“Stage Two” refers to Stage Two of the Framework, Building the IT

Investment Foundation.

- 18 -

The following excerpts from the FBI’s Plan provide an overview

of how the FBI’s select, control, and evaluate processes for IT

investment management are intended to operate upon

implementation.

26

Select

In the Select phase, potential projects will be initiated by

the project sponsor via the development of a preliminary

feasibility analysis (concept paper), followed by the

development of a more-robust business case analyses

(OMB Exhibit 300). The project proposal package will be

submitted to the Technical Review Board

27

to be assessed

for any technical risks and then submitted to the Project

Oversight Committee

28

for a business review. The Project

Oversight Committee will assemble the multiple requests

and prioritize these requests against predefined selection

criteria. A “candidate” fiscal project portfolio will then be

developed and presented to the Executive Review Board

29

for final evaluation and approval, and ultimately for

submission to the fiscal budget process.

Control

In the Control phase, the current fiscal year IT portfolio

will be tracked by the functional project management

office and individual project teams. Monthly status reports

will be created and presented to the Project Oversight

Committee, who will work to mitigate any project related

risks. Projects with exceptions to the baseline plans will be

subsequently presented to the Executive Review Board for

26

See Appendices 2 and 3, respectively, for flowcharts on the Plan’s control

and evaluate processes.

27

According to the Plan, the Technical Review Board must be established to

review each proposed ITIM initiative for enterprise architecture compliance, IT

security compliance, and other technical risks.

28

According to the Plan, the Project Oversight Committee must be established

to perform the program management and oversight duties of the ITIM process, such

as making recommendations to the Executive Review Board on selecting IT proposals

and disposing of IT projects.

29

According to the Plan, the Executive Review Board must be established to

make the final IT investment decisions.

- 19 -

decisions about budget, scope, timeline and/or projected

outcomes. During the control phase, a project will be able

to receive approval to: proceed “as is,” proceed with

modified funding levels and/or modified functionality, or be

terminated.

Evaluate

In the Evaluate phase, IT investments that are in the

operations and maintenance mode will be monitored by

the Executive Review Board to ensure that expected

benefits are being realized. Periodic program reviews will

be conducted, wherein each IT investment will be

evaluated against predefined performance metrics and

criteria. Based on the reviews, decisions will be made

about: future phases of existing projects; and the current

policies and procedures governing the entire IT investment

management, the systems development life-cycle, and

other related processes. Advocacy arguments (to modify

existing management practices and procedures) are also

constructed during this phase, if applicable.

JMD officially approved the FBI’s Plan in May 2002, although

officials from the IRD told us that in February 2002 they received

verbal approval to initiate their ITIM process.

30

The May 2002

approval letter states that the FBI ITIM process conforms to the

guidelines defined by the GAO, OMB, and DOJ. Further, it states that

the Plan is clear and comprehensive in its statement of the ITIM policy

and its definition of organizational roles, responsibilities, and

deliverables. Additional JMD comments, as well as our own

independent assessment of the Plan, are discussed later in this report.

The FBI started its ITIM process in February 2002 by appointing

the three oversight review boards discussed above (the Technical

Review Board, the Project Oversight Committee, and the Executive

Review Board). Also, in February 2002 the FBI held training seminars

for each division to introduce the concepts of the Plan. In March 2002,

the FBI began pilot testing the select phase of the Plan for FY 2004

proposed IT project enhancements. In May 2002, the pilot test of the

30

JMD officials told us that the delay in providing written approval of the FBI’s

ITIM process was because JMD did not have a Chief Information Officer early in

2002.

- 20 -

select phase was completed and the ITIM contractor issued the, “Post

Implementation Review: FBI ITIM Pilot.”

The Plan recognizes that as the FBI’s ITIM process moves

through the maturity stages, other key components of IT

infrastructure must evolve to optimize the IT investment function.

These components include an IT strategic plan, an enterprise

architecture framework, and project management. According to the

Framework, an effective IT function will include these components and

mature IT investment management processes are dependent on the

components being in place.

- 21 -

OIG

FINDINGS AND RECOMMENDATIONS

1. The FBI’s Management of IT Investments

The FBI is not effectively selecting, controlling, and

evaluating its IT investments because it has not fully

implemented any of the critical processes necessary for

successful IT investment management. In the past, the

FBI has not given sufficient attention to information

technology investment management. As a result, the FBI

continues to spend hundreds of millions of dollars on IT

projects without having adequate selection and project

management controls in place to ensure that IT projects

will meet intended goals. However, since the FBI

developed its ITIM Model and Transition Plan in

January 2002, it has focused more management attention

in this area and has made progress towards attaining a

basic IT investment management foundation. Much of the

progress has been in the “select” phase of the Plan, which

was pilot tested in the Spring of 2002.

The ability of the FBI to completely implement the

“control” and “evaluate” phases of the Plan, and achieve

mature IT investment processes that can lead to enhanced

mission performance, will require the FBI to increase its

efforts in: (1) fully developing and documenting its new

ITIM process; (2) requiring more input and participation

from ITIM managers and users; and (3) further developing

its project management and enterprise architecture

functions. While the FBI recognizes many of these needs

and has taken initial steps to address the needs, further

action in these areas is needed to ensure that IT projects

are developed within cost and schedule requirements, and

meet performance expectations. The Trilogy project

provides an example of how the non-implementation of

fundamental IT investment management practices can put

a project at risk of not delivering, within cost and schedule

requirements, what was promised.

A. The FBI’s Progress Toward Attaining a Basic IT

Investment Management Foundation

Although the FBI made measurable progress in improving its IT

investment capability since it initiated a new ITIM process in early

- 22 -

2002, the FBI still lacks a complete foundation to build its IT

investment maturity processes, and therefore is still in Stage One

maturity.

31

In the past, the FBI has not given sufficient management

attention to IT investments. Because of the lack of management

attention in the past, the FBI failed to implement the critical processes

necessary to build an IT investment foundation. These critical

processes include: (1) IT investment review board operation, (2) IT

project oversight, (3) IT system and project identification and tracking,

(4) business needs identification for IT projects, and (5) IT proposal

selection.

(1) Importance of Attaining a Basic IT Investment

Management Foundation

The primary purpose for attaining a basic IT investment

management capability (Stage Two maturity) is to build the foundation

for repeatable, successful IT project-level investment control and

selection processes. Effective control processes over IT projects

ensure that deviations from cost and schedule baselines can be

identified and corrected. Selection processes ensure that the FBI has

an effective methodology for approving only IT projects that are

consistent with its needs and goals. According to the Framework, an

organization can only achieve Stage Two maturity if it fully implements

the following five critical processes:

1. defining investment review board operations,

2. developing a basic process for selecting new IT

proposals,

3. developing project-level investment control processes,

4. identifying IT projects and systems, and

5. identifying the business needs for each IT project.

To implement these critical processes, the FBI must execute a

total of 38 key practices as defined in the Framework, or have

alternative practices in place that are designed to achieve the same

outcome.

31

Stage One maturity is the lowest level of maturity designated by the GAO

ITIM Framework. According to the Framework, an organization is in Stage One

maturity when it has not fully implemented the five critical processes associated with

Stage Two maturity.

- 23 -

At the start of our audit in January 2002, FBI officials told us

that the Bureau was in the process of developing its new ITIM process.

Although its ITIM process was still in the development stages, FBI

officials told us that the FBI was executing certain key practices from

Stage Two of the Framework. Additionally, the FBI officials said in

March 2002 that they would pilot test ITIM processes pertaining to the

selection of new IT proposals for the FY 2004 budget cycle. Further,

the Plan establishes the FBI’s goal to fully attain Stage Two maturity

for the FY 2005 budget cycle that starts in March of 2003, thereby

establishing the foundation for enhanced investment capability.

(2) Summary of the FBI’s Progress Toward Attaining Stage

Two Maturity

Based on the FBI’s responses to the self-assessment

32

(and our

validation of those responses), the FBI did not yet have in place any of

the five critical processes associated with Stage Two maturity.

However, since the FBI began pilot testing the select phase of its Plan

in March 2002, it has made progress towards implementing the 38 key

practices comprising the five critical processes - particularly in the area

of selecting new proposals for IT projects. Specifically, at the

beginning of our audit in January 2002, the FBI was only executing

4 of the 38 required key practices; however, as of June 2002, the FBI

was executing 14 of the key practices. The following table provides a

summary of the FBI’s progress toward implementing the key practices

required for each critical process.

32

To facilitate our assessment of the FBI’s IT investment maturity, the FBI

completed a self-assessment regarding the key practices from the Framework that it

was executing, or planning to execute, upon implementation of its new ITIM process.

- 24 -

FBI Progress Toward Attaining Stage Two Maturity

Critical

Process

Status of

Implementing

Critical

Process

Total Key

Practices

Required

Key

Practices

Executed

Prior to

March

2002

Key

Practices

Executed

as of

June

2002

1. IT Investment

Board

Operation

Not Implemented

6

0

2

2. IT Project

Oversight Not Implemented

11

1

2

3. IT Project

Identification

Not Implemented

7

1

2

4. Business

Needs

Identification

for IT Projects

Not Implemented

8

2

3

5. Proposal

Selection

Not Yet

Implemented,

but Substantial

Progress Made

6

0

5

Total

38

4

14

Source: OIG analyses

For the remainder of section A of this finding, we provide

detailed narratives of the FBI’s progress toward implementing each of

the five critical processes. We also provide specific recommendations

for expediting implementation of the critical processes and establishing

more timely Stage Two maturity.

Each critical process contains core elements that provide the

common framework for the process. For example, the organizational

commitment element addresses the management actions that ensure

the critical process is established and will endure; the prerequisites

element addresses the conditions that must exist within an

organization to successfully implement a critical process; and the

activities element consists of the key practices necessary to implement

a critical process. The key practices are the tasks within a core

- 25 -

element that must be performed by an organization to effectively

implement and institutionalize a critical process.

(3) Critical Process #1: IT Investment Review Board Operation

Depending on its size, structure, and culture, an organization

may have more than one IT investment review board. The purpose of

such boards is to ensure that basic policies for selecting, controlling,

and evaluating IT investments are developed, institutionalized, and

consistently followed throughout the organization. To establish a fully

functioning investment review board, the FBI must execute the

following six key practices:

1. create an IT investment process guide containing policies

and procedures to direct board operations;

2. require executives and line managers to support and

carry out board decisions;

3. allocate adequate resources for operating each board;

4. define board membership, policies and procedures, roles and

responsibilities;

5. create and define board membership to integrate both IT and

business knowledge; and

6. require the IT investment boards to follow the written

policies and procedures as defined in the process guide.

The following table summarizes the FBI’s progress toward

implementing fully functioning investment review boards.

- 26 -

FBI Progress Toward Implementing Fully Functioning

Investment Review Boards (Critical Process #1)

Key Practice

Key Practice

Execution

Status Prior to

March 2002

Key Practice

Execution

Status as of

June 2002

Organizational Commitment 1. An

organization-specific IT investment

process guide is created to direct each

board’s operations.

Not Executed Executed

Organizational Commitment 2.

Organization executives and line

managers support and carry out IT

investment board decisions.

Not Executed Not Executed

Prerequisite 1. Adequate resources are

provided for operating each IT

investment board.

Not Executed Not Executed

Prerequisite 2. Board members

understand the investment board’s

policies and procedures and exhibit core

competencies in using the IT investment

approach via training, education, or

experience.

Not Executed Not Executed

Activity 1. Each IT investment board is

created and defined with board

membership integrating both IT and

business knowledge.

Not Executed Executed

Activity 2. Each IT investment board

operates according to written policies and

procedures in the organization-specific

IT investment process guide.

Not Executed Not Executed

Source: OIG analyses

a. The FBI Has Executed Two of the Six Key Practices

Associated with IT Investment Board Operation

We determined that the FBI executed two of the six key

practices associated with implementing this critical process.

Specifically, the FBI created an IT investment process guide containing

policies and procedures to direct board operations (Organizational

Commitment 1), and it created and defined three investment review

boards integrating both IT and business knowledge (Activity 1).

- 27 -

Regarding the IT investment process guide (Organizational

Commitment 1), in January 2002 the FBI issued its IT Investment

Model and Transition Plan

33

containing required guide elements

prescribed by the Framework including:

•

specifics about the roles of key people within the FBI investment

process;

•

an outline of the significant events and decision points within the

processes;

•

an identification of the external and environmental factors that

will influence the processes; and

•

the manner in which IT investment-related processes will be

coordinated with other organizational plans and processes.

Regarding the investment review boards (Activity 1), in

June 2002 the Director approved board charters for each of the three

investment review boards (the Executive Review Board, the

Project Oversight Committee, and the Technical Review Board) that

defined board membership and the responsibilities of board members.

•

The Executive Review Board is comprised of the FBI Director (as

Chairperson), the Chief Information Officer, the FBI’s four

Executive Assistant Directors (EADs),

34

a Special Agent in

Charge committee member, the Assistant Director of the Finance

Division, and the Strategic Planning Manager.

This Board’s primary responsibility will be to evaluate and

approve projects in the candidate fiscal project portfolios and

forward approved projects to the fiscal budget process. This

Board will also determine whether problematic projects should

proceed “as is,” proceed with modified funding levels and/or

modified functionality, or be terminated.

•

The Project Oversight Committee includes: the Chief

Information Officer (as Chairperson), the Assistant Director from

33

The Plan was issued in draft form because it is the intent of the FBI to

modify and supplement the Plan as the ITIM process is being pilot tested.

34

The EADs are for: (1) Criminal Investigations, (2) Counterterrorism and

Counterintelligence, (3) Law Enforcement Services, and (4) Administration.

- 28 -

each division, a member from the Office of General Counsel, the

Chief Contracting Officer, and the Strategic Planning Manager.

Once the Technical Review Board completes its assessment, the

Project Review Board then performs a business review of the

proposed projects, prioritizes these proposals against predefined

selection criteria, and develops a “candidate” fiscal project

portfolio for presentation to the Executive Review Board. The

committee also reviews monthly status reports for ongoing

projects to mitigate project related risks. Projects with

exceptions to baseline plans will be presented to the Executive

Review Board for corrective action.

•

The Technical Review Board is comprised of: the Section Chief,

Information Resources Management Office (as Chairperson); the

Assistant Director of IRD; the IRD’s section chiefs; and

representatives from the Laboratory Division, CJIS Division, and

Security Division. This board’s primary responsibility will be to

assess technical risks for proposed projects.

The boards actually began functioning as early as March 2002, in

conjunction with the FBI’s pilot testing of ITIM processes pertaining to

the selection of new IT proposals for the FY 2004 budget cycle.

Although board membership consists mostly of FBI managers who do

not have extensive IT knowledge,

35

the use of subject matter experts

and reliance on the Enterprise Architecture Technical Committee

36

can

compensate for a lack of IT knowledge.

b. The FBI Must Execute Four of the Six Key Practices

Associated with IT Investment Board Operation

Although progress has been made, the FBI does not have fully

functioning IT investment boards because it still must execute four of

the six key practices associated with this critical process. Specifically,

the FBI must ensure that:

35

Based on our interviews with FBI managers from the IRD, CJIS, and

Inspection Divisions, most of the members on the investment boards are former agents

with no specialized expertise, training, or competencies in IT.

36

The Enterprise Architecture Technical Committee was created to provide

technical expertise to the Technical Review Board. Members of this committee are

comprised of IT specialists familiar with enterprise architecture, configuration

management, and quality assurance.

- 29 -

•

organization executives and line managers support and carry out

IT investment board decisions (Organizational Commitment 2);

•

adequate resources are provided for operating each IT

investment board (Prerequisite 1);

•

board members understand the investment board’s policies and

procedures and exhibit core competencies in using the IT

investment approach via training, education, or experience

(Prerequisite 2); and

•

each IT investment board operates according to written policies

and procedures contained in the investment process guide

(Activity 2).

Regarding Organizational Commitment 2 and Activity 2, the

approved charters for the investment review boards have been in

effect since June 2002. Consequently, the FBI did not have sufficient

data for us to assess whether managers and support staff effectively

carried out board decisions and whether the boards operated according

to the written policies and procedures contained in the Plan and board

charters.

Regarding Prerequisites 1 and 2, in our judgment the FBI did not

adequately plan sufficient time to ensure the IT investment boards

operated effectively. Specifically, the FBI did not provide ample time

between the initial draft of its Plan (January 25, 2002) and the

March 2002 pilot testing of the select phase to adequately prepare and

train IT board members. The DOJ originally instructed each

component to begin developing an ITIM process in January 2001.

37

In

June 2001, the DOJ required each component to complete and submit

to JMD an ITIM process and transition plan by the end of 2001.

38

The

DOJ also required each component to initiate the ITIM process for the

FY 2004 budget cycle, which for the FBI began in March 2002.

Consequently, the FBI had only one full month between the issuance

of the Plan in late January 2002 and the initiation of the select phase

of its ITIM process in early March 2002.

37

This instruction originated from DOJ Order 2880.1A, policy on Information

Technology Investment Management, issued in January 2001.

38

This instruction originated from a DOJ memorandum dated

June 28, 2001. This memorandum required each component to have an ITIM

transition plan that will allow implementation for the FY 2004 budget cycle.

- 30 -

The ITIM Program Office Manager told us that the former FBI

Chief Financial Officer would not approve the use of a contractor to

assist in the development of the ITIM process earlier in the year.

According to the former Chief Financial Officer, she had concerns that

federal contracting regulations prohibited the FBI from using a

contractor to perform a service that involves budget planning.

However, following her transfer to another division in December 2001,

the Information Resources Management Section received authorization

to hire a contractor to assist with the development and implementation

of the ITIM process.

We believe that without an ITIM contractor the FBI still had the

opportunity to begin planning its ITIM process (including the training

of board members) early in 2001. In fact, had the FBI better

coordinated other ongoing efforts to develop processes that

complement IT investment management, the FBI could have made

significant strides in initiating its ITIM process during 2001 without

expending additional resources. As discussed in section B of this

finding, the FBI did not sufficiently incorporate (a) its enterprise

architecture function (which was under development in 2001) and

(b) the Project Management Process (issued in draft form in

October 2001) into the development of its ITIM process. Enterprise

architecture and project management not only complement the ITIM

process, but also facilitate the maturation of ITIM. As discussed in

section B of this finding, the FBI did not effectively utilize its internal

resources when it developed its ITIM process through the use of a

contractor because the FBI did not adequately consider the

complementary, and potentially duplicative efforts that were already

underway.

Not providing ample time resulted in inadequate training of

board members and minimal preparation time to develop IT proposals.

For example, Technical Review Board members had only 3 business

days to review over 50 IT proposals prior to their first board meeting.

FBI officials recognized these implementation issues in the Post-

Implementation Review of the select phase pilot test.

In preparing board members for their duties, the FBI has thus

far only provided one overview training session for board members

and other users in the ITIM process. Additionally, while FBI officials

have told us more ITIM training will be forthcoming, they have not

provided us with any specific training plans for the future. Further,

members of the Technical Review Board told us that board members,

especially the Assistant Directors and EADs, do not have extensive

- 31 -

knowledge in managing IT and must rely heavily on knowledgeable

staff and other subject matter experts.

For the ITIM process to become institutionalized, the FBI must

have a better training program. According to the Framework, board

members should understand the board’s policies, roles, rules, and

activities and be capable of carrying out their responsibilities

competently. Education and training for members is needed in areas

such as economic evaluation techniques, capital budgeting methods,

and performance measurement strategies.

The FBI’s Post-

Implementation Review of the select phase pilot testing recommends

“role-specific” training sessions for the following ITIM roles: (1) ITIM

Liaison representatives,

39

(2) Executive Review Board members,

(3) Program Oversight Review Board members, (4) Technical Review

Board members, and (5) ITIM stakeholders. It further recommends

continuation of the overview training sessions previously provided,

plus training for ITIM specific tools, such as the concept paper

(containing the preliminary feasibility analysis), the OMB Exhibit 300

(containing the business case analyses), and IT proposal summaries.

FBI officials told us that time constraints were the main cause for

not executing the four key practices identified above. As a result,

there was insufficient time to introduce ITIM concepts to board

members and other ITIM users. As mentioned above, the DOJ

required each component to develop and begin implementation of an

ITIM process for the FY 2004 budget cycle, which for the FBI begins in

March 2002. Although FBI officials were aware of the requirement to

initiate and adopt an ITIM process in January 2001, it was not until

December 2001 that it began to develop its ITIM process. Had the FBI

initiated more timely action to develop its ITIM process, it would have

had significantly more time to prepare and train ITIM board members

and other users. Without sufficient training and allocation of time to

perform required tasks, the investment review boards cannot carry out

their responsibilities to effectively select, control and evaluate projects.

39

The FBI’s ITIM process defines the ITIM Liaison Representative as an

individual from a particular division/business unit that facilitates workflow

and communications between that division/business unit and the ITIM

program office.

- 32 -

c. Recommendations

We recommend that the Director of the FBI:

1. Require the ITIM Program Office to plan for and take more timely

action to allow board members and other ITIM users to execute

assigned responsibilities competently (Prerequisite 1).

2. Ensure that all members of IT investment boards receive sufficient

education and training to execute assigned responsibilities

effectively. We suggest that for each of the investment boards the

FBI: (a) identify the core competencies required of members in

using the IT investment approach, and (b) develop appropriate

education and training development plans to ensure members

acquire the required core competencies (Prerequisite 2).

(4) Critical Process #2: IT Project Oversight

The purpose of this critical process is to ensure that the FBI’s

investment review boards and project development teams provide

effective oversight for its IT projects throughout all phases of the

project life-cycle. IT investment boards generally review each

project’s progress toward predicted cost and schedule expectations as

well as anticipated benefits and risk exposure. The board members

also employ early warning systems that enable them to take corrective

actions at the first signs of cost, schedule, and performance slippages.

Individual project development teams are responsible for meeting

project milestones within the expected cost and schedule parameters.

Effective project oversight requires, among other things:

•

having written policies and procedures for project management;

•

developing and maintaining an approved project management

plan for each project;

•

having written policies and procedures for oversight of IT

projects;

•

making up-to-date cost and schedule data for projects available

to the investment review boards;

- 33 -

•

reviewing each project’s performance by comparing actual cost

and schedule data to expectations regularly; and

•

ensuring that corrective actions for each under-performing

project are defined, implemented, and tracked until the desired

outcome is achieved.

We concluded that the FBI is not effectively overseeing its

ongoing IT projects. While the FBI maintained project management

guidance and had three IT investment review boards in operation since

March 2002, these activities have not adequately supported the FBI’s

IT project oversight function. Our testing of the key practices

associated with this critical process indicates that the FBI is executing

only two out of the eleven key practices required to implement this

critical process. The following table summarizes FBI progress toward

implementing IT project oversight.

- 34 -

FBI Progress Toward Implementing IT Project Oversight

(Critical Process #2)

Key Practice

Key Practice

Execution

Status Prior to

March 2002

Key Practice

Execution

Status as of

June 2002

Organizational Commitment 1. The

organization has written policies and

procedures for project management. Executed

Executed

Organizational Commitment 2. The

organization has written policies and

procedures for management oversight

of IT projects.

Not Executed Not Executed

Prerequisite 1. Adequate resources

are provided to assist the boards in

overseeing IT projects.

Not Executed Not Executed

Prerequisite 2. Each IT project has

and maintains an approved project

management plan that includes cost

and schedule controls.

Not Executed Not Executed

Prerequisite 3. An IT investment

review board is operating.

Not Executed Executed

Prerequisite 4. Information from the

IT asset inventory is used by the IT

investment board as applicable.

Not Executed Not Executed

Activity 1. Each project's up-to-date

cost and schedule data are provided to

the appropriate IT investment board. Not Executed Not Executed

Activity 2. Using established criteria,

the IT investment board oversees each

IT project's performance regularly by

comparing actual cost and schedule

data to expectations.

Not Executed Not Executed

Activity 3. The IT investment board

performs special reviews of projects

that have not met predetermined

performance standards.

Not Executed Not Executed

Activity 4. Appropriate corrective

actions for each under-performing

project are defined, documented, and

agreed to by the IT investment board

and the project manager.

Not Executed Not Executed

Activity 5. Corrective actions are

implemented and tracked until the

desired outcome is achieved.

Not Executed Not Executed

Source: OIG analyses

- 35 -