United States Department of Justice

Office of the Inspector General

Audit Division

A

UDIT

R

EPORT

F

EDERAL

B

UREAU OF

I

NVESTIGATION

’

S

M

ANAGEMENT OF

I

NFORMATION

T

ECHNOLOGY

I

NVESTMENTS

D

ECEMBER

2002

03-09

FEDERAL BUREAU OF INVESTIGATION’S MANAGEMENT

OF INFORMATION TECHNOLOGY INVESTMENTS

EXECUTIVE SUMMARY

Following the September 11, 2001, terrorist attacks, the Attorney

General and the Director of the Federal Bureau of Investigation (FBI)

made clear that prevention of terrorism is the top priority of the

Department of Justice (DOJ) and the FBI. Effective use of information

technology (IT) is crucial to the FBI’s ability to meet this priority as well

as its other critical responsibilities.

However, reviews conducted by the Office of the Inspector

General (OIG) and the General Accounting Office (GAO) have found

major weaknesses associated with the FBI’s IT. The FBI has listed

upgrading its information technology as one of its top ten highest

priorities. In June 2002 Congressional testimony, the FBI

acknowledged that its IT infrastructure is severely outdated.

Because of the importance of the FBI’s management of its IT

systems, we performed this audit to: (1) determine whether the FBI

was effectively managing its IT investments; and (2) assess the FBI’s

IT-related strategic planning and performance measurement activities.

1

We also examined the FBI’s efforts to develop enterprise architecture

2

and project management capabilities.

In this audit, we conducted approximately 85 interviews with

70 officials from the FBI, DOJ, GAO, and the Office of Management and

Budget (OMB). The FBI officials interviewed were from the Director’s

office, Information Resources Division, Criminal Justice Information

Services Division, Laboratory Division, Inspection Division, and Finance

1

During our audit fieldwork, we initiated work relating to a third objective: to

determine if the FBI has implemented prior information technology related

recommendations directed toward improving information technology. We will issue a

separate report on this objective.

2

Enterprise architecture is the organization-wide blueprint that defines an

entity’s functions and systems, including IT systems. It provides a comprehensive

view (through models, narratives, and diagrams) of the interrelationships of an

organization’s operations and structures and how these structures align with the

organization’s mission. The Clinger-Cohen Act of 1996 recognizes the

interrelationship between enterprise architecture and IT investment management by

requiring federal agencies to develop an enterprise architecture.

- i -

Division. Additionally, OIG auditors and analysts traveled to FBI

laboratory facilities in Quantico, VA, and five FBI field offices to

conduct interviews and assess the FBI’s implementation of IT

initiatives. We also reviewed more than 200 documents, including the

FBI’s IT management policies and procedures, project management

guidance, strategic and program plans, IT project proposals and

management plans, budget documentation, organizational structures,

Congressional testimony, and prior OIG and GAO reports.

1. Summary of Audit Findings

We concluded that the FBI has not effectively managed its IT

investments because it has not fully implemented the management

processes associated with successful IT investments. The foundation

for sound IT investment management (ITIM) includes the following

fundamental elements:

•

defining and developing IT investment boards;

•

following a disciplined process of tracking and overseeing each

project’s cost and schedule milestones over time;

•

identifying existing IT systems and projects;

•

identifying the business needs for each IT project; and

•

using defined processes to select new IT project proposals.

The FBI failed to implement these critical processes. We found

that the FBI does not have fully functioning IT investment boards that

are engaged in all phases of IT investment management. The FBI was

not following a disciplined process of tracking and overseeing each

project’s cost and schedule milestones. The FBI failed to document a

complete inventory of existing IT systems and projects, and did not

consistently identify the business needs for each IT project. The FBI

did not have a fully established process for selecting new IT project

proposals that considered both existing IT projects and new projects.

Because the FBI has not fully implemented the critical processes

associated with effective IT investment management, the FBI

continues to spend hundreds of millions of dollars on IT projects

without adequate assurance that these projects will meet their

intended goals.

- ii -

We concluded that these shortcomings primarily resulted from

the FBI not devoting sufficient management attention in the past to IT

investment management.

However, FBI management has recognized that its past methods

to manage IT projects have been deficient, and the FBI recently has

committed to changing those practices. In January 2002, the FBI

developed a conceptual model for selecting, controlling, and evaluating

IT investments. The model seeks to define a process that will promote

a Bureau-wide perspective on IT investment management, so that only

IT projects with the best probability of improving mission performance

are selected. Further, the process is intended to provide the methods,

structures, disciplines, and management framework that governs the

way IT projects are controlled and evaluated.

In addition to developing a conceptual model for a new ITIM

process, in early 2002 the FBI began a pilot test of the new process for

the selection of IT proposals. We found that the FBI made

improvements during the pilot testing of the new selection process.

Pursuant to the new process, the FBI created three IT investment

review boards that reviewed IT proposals for technical compliance and

“mission fit.” These boards, comprised of the FBI Director, FBI

executives and IT managers, selected new IT proposals that will be

considered for inclusion in the Fiscal Year (FY) 2004 budget request.

While the FBI has made efforts to improve its IT investment

management practices, the FBI must take further actions to ensure

that it can implement the fundamental processes necessary to build an

IT investment foundation, as well as the more mature processes

associated with highly effective IT investment management. These

actions include:

•

fully developing and documenting its new IT investment

management process – which is necessary to completely

implement the activities defined in the FBI’s conceptual model;

•

requiring increased participation from IT program managers and

users – which is necessary to ensure senior management

acceptance and foster understanding and institutionalization of

the ITIM process; and

•

further developing the FBI’s project management and enterprise

architecture functions – which is necessary to execute the

- iii -

control and evaluate components of the ITIM process as well as

advance its investment management capability.

Our audit also reviewed the FBI’s management of Trilogy, the

FBI’s largest and most critical IT project. We found that the lack of

critical IT investment management processes contributed to missed

milestones and led to uncertainties about cost, schedule, and technical

goals. Specifically, despite $78 million in additional funding, the FBI

missed its July 2002 milestone date for completing the physical IT

infrastructure upgrades to field offices, including new computer

hardware and networks.

3

FBI officials stated that they are not

expecting the physical infrastructure components of Trilogy to be

completed until March 2003. In addition, the user application

component of Trilogy, recognized by FBI officials as the most

important aspect of the project in terms of improving agent

performance, is at high risk of not being completed within the funding

levels appropriated by Congress. In our judgment, the management

problems associated with Trilogy demonstrate the FBI’s urgent need

for enhanced IT investment management.

We also concluded that the FBI’s IT strategic planning and IT

performance measurement are inadequate. We found that the FBI's

strategic plan does not include goals for IT investment management,

and the FBI’s strategic plan and performance plan are not consistent

with the DOJ’s annual performance plan.

The remainder of this executive summary provides more

background and details on our audit findings and recommendations to

help improve the FBI’s management of its IT investments.

2. Background

The Clinger-Cohen Act of 1996 requires each federal agency to

implement a process for maximizing the value of its IT investments.

This process is intended to ensure that IT projects are being

implemented at acceptable costs and within reasonable time frames,

and that the projects are contributing to enhanced mission

performance. Specifically, the Clinger-Cohen Act requires federal

agencies to: (1) develop an enterprise architecture framework, and

3

With the $78 million in additional funding, Trilogy’s total appropriation was

$458 million as of June 2002.

- iv -

(2) follow a “select/control/evaluate” approach to managing IT

investments.

In May 2000, the GAO developed the IT Investment

Management Framework (Framework) to provide a common

methodology for assessing IT capital planning and investment

management practices at federal agencies. The Framework specifically

describes the organizational processes required to carry out sound IT

investment management.

The Framework, based on best practices of leading

organizations, is a hierarchical model comprised of five maturity

stages. These maturity stages represent steps toward achieving stable

and mature investment management processes. As agencies advance

through these stages, their capability to effectively manage IT

increases. With the exception of the first stage, each maturity stage is

comprised of critical processes that must be implemented and

institutionalized for the agency to satisfy the requirements of that

stage. These critical processes are further broken down into key

practices an agency should perform to successfully implement each

critical process.

An agency using these critical processes is in a better position to

successfully invest in IT and use its IT investments to achieve its

priorities. Conversely, an agency that does not have these critical

processes in place is at high risk that its IT projects will fail to support

the achievement of priorities.

To determine whether the FBI was effectively managing its IT

investments, we utilized the Framework because it is: (1) a

standardized tool for internal and external evaluations of an agency’s

IT investment management process; (2) a consistent and

understandable mechanism for reporting the results of these

assessments; and (3) a road map agencies can use for improving their

IT investment management process.

In addition, the Government Performance and Results Act of

1993 (Results Act) requires strategic planning and performance

measurement throughout the federal government. The Results Act

seeks to improve the effectiveness, efficiency, and accountability of

federal programs by requiring federal agencies to establish goals for

program performance and measurement. The Results Act requires

agencies to prepare a strategic plan, annual performance plan, and

annual performance report.

- v -

While IT strategic planning is a function somewhat independent

of IT investment management, these two functions are interrelated

and complementary. The DOJ has recognized the importance of

integrating strategic planning with IT management. In July 2002, the

DOJ released its IT Strategic Plan that included a strategic initiative to

establish and improve investment management processes.

3. The FBI’s Management of IT Investments

Our audit found that the FBI has not established an IT

investment foundation and therefore is in Stage One maturity

according to the ITIM Framework. Stage One maturity is characterized

by inconsistent, unstructured, and unpredictable investment

processes. Our observations of the FBI’s IT investment processes

found that the FBI’s actual processes are consistent with these

Stage One deficiencies.

The critical processes necessary to establish an IT investment

foundation include: (1) defining investment review board operations,

(2) developing project-level investment control processes,

(3) identifying IT projects and systems, (4) identifying the business

needs for each IT project, and (5) developing a basic process for

selecting new IT proposals.

We found that the FBI failed to implement these critical

processes. The FBI did not have a fully established investment review

board operation because the FBI did not provide adequate resources

for operating the IT investment boards. Additionally, we found

insufficient evidence to demonstrate that: (1) organization executives

and line managers supported and carried out IT investment board

decisions and (2) board members understood the investment board’s

policies and procedures and exhibited core competencies in using the

IT investment approach via training, education, or experience.

Specifically, the FBI did not provide ample time to adequately prepare

and train IT board members prior to initiating the pilot test of its

recently developed ITIM process. This resulted in inadequate training

of board members and minimal preparation time to develop IT

proposals. For example, Technical Review Board members had only

three business days to review over 50 IT proposals prior to their first

board meeting.

Additionally, we found that the FBI is not effectively overseeing

its IT projects. For example, while the FBI has issued project

management guidance, the guidance is not being followed on a

- vi -

consistent basis. Depending on whom we talked to, we obtained

different answers as to which document represented the FBI’s official

project management guidance.

Without effective oversight of IT projects, FBI officials do not

have adequate assurance that IT projects are being developed on

schedule and within established budgets. According to a former Chief

Information Officer at the FBI, the lack of effective oversight of IT

projects has prevented IT project managers from being held

accountable for cost and schedule overruns and the ultimate

performance of projects. Senior FBI officials also told us that the

Bureau’s budget formulation process focuses only on the acquisition

costs for IT projects and not the full life-cycle costs, especially

operations and maintenance costs.

We also found that the FBI’s investment review boards are not

aware of all the IT projects and resources for which the boards are

responsible. FBI Divisions maintained some version of an IT inventory

for the projects and systems under their jurisdiction, and there was no

centralized office responsible for maintaining a uniform listing Bureau-

wide. FBI managers told us they were in the process of developing an

IT asset inventory, but at the time of our audit they were unable to

provide an estimated date for completing the inventory.

FBI personnel told us that staff shortages are the primary cause

for the incomplete IT asset inventory. In our judgment, staff

shortages may be a contributing factor, but the lack of centralized

management over IT investments was the significant reason for this

problem. Until June 2002, the FBI did not have a centralized project

management office to assist the investment boards in overseeing IT

projects. The FBI maintained three separate division-level project

management offices to manage IT projects.

We also determined that the FBI did not have a fully established

process for selecting IT proposals. FBI officials told us that, prior to

March 2002, individual divisions determined IT needs in a “stovepipe,”

without knowledge of the business needs and priorities of the Bureau

as a whole. The FBI did not have a clearly designated official to

manage the proposal selection process. According to Information

Resources Management Section personnel, the Finance Division

managed the IT selection process. However, according to Finance

Division personnel, the Information Resources Management office was

responsible for managing the proposal selection process.

- vii -

Without a comprehensive proposal selection process that

includes adequate resources and training, the FBI cannot ensure that it

is selecting the best IT projects that meet mission-critical needs.

Because the FBI did not fully implement any of the critical

processes associated with Stage Two, the FBI continues to spend

hundreds of millions of dollars on IT projects without having adequate

selection and project management controls in place to ensure that IT

projects will deliver their intended benefits.

The FBI began pilot testing the select phase of its new ITIM

process in March 2002, and since then has made measurable progress

towards implementing the key practices that comprise the critical

processes – particularly in the area of selecting new proposals for IT

projects. Specifically, at the beginning of our audit in January 2002,

the FBI only was executing 4 of the 38 required key practices;

however, as of June 2002, the FBI was executing 14 of the key

practices.

With the pilot testing of its new ITIM process, the FBI created an

IT investment process guide containing policies and procedures to

direct board operations, and created and defined three investment

review boards integrating both IT and business knowledge.

Additionally, the FBI has designated an official responsible for

managing the IT project and system identification process and

ensuring that the inventory meets the needs of the investment

management process. Further, during the test pilot of the ITIM

process, the board reviews of IT project proposals provided assurance

that business needs were clearly identified and defined. Also during

the test pilot, we determined that FBI IT investment board members

analyzed and prioritized new IT proposals according to established

selection criteria for the FY 2004 budget cycle.

Despite the progress made, full implementation of the ITIM

process will require the FBI to (1) fully develop and document its new

ITIM process; (2) require more input and participation from IT

managers and users; and (3) further develop its project management

and enterprise architecture functions. Completion of the initial steps

taken by the FBI will ensure that IT projects are developed within cost

and schedule requirements, and meet performance expectations. The

Trilogy project provides an example of how the non-implementation of

fundamental IT investment management practices can put a project at

risk of not delivering what was promised, within cost and schedule

requirements.

- viii -

4. Trilogy

We also performed a case study of the FBI’s implementation of

its Trilogy project. We selected Trilogy because it is the FBI’s largest

ongoing IT project and is considered vital to the FBI’s ability to

perform its mission. Trilogy is intended to upgrade the FBI’s:

(1) hardware and software – referred to as the Information

Presentation Component (IPC), (2) communication networks – referred

to as the Transportation Network Component (TNC), and (3) five most

important investigative applications – referred to as the User

Applications Component (UAC). The IPC and TNC upgrades will

provide the physical infrastructure needed to run the applications from

the UAC portion. The UAC portion is intended to upgrade and

consolidate five of the FBI’s 42 investigative applications. Because of

the 37 other investigative applications and approximately 160 non-

investigative applications that Trilogy will not cover, Trilogy is only a

starting point towards upgrading the FBI’s entire IT infrastructure.

According to the FBI, Trilogy is not designed to provide the FBI with

state-of-the-art IT; it is intended to provide the foundation so that the

FBI can eventually attain state-of-the-art IT.

In November 2000, Congress appropriated $100.7 million for the

first year of the $379.8 million Trilogy project, which was to be funded

over a three-year period (from the date contractors were hired). The

$100.7 million was a combination of new program funding and a

re-direction of base resources. When the FBI requested contractor

support for Trilogy, it combined the IPC and TNC portions for

continuity as both encompass physical IT infrastructure enhancements.

The contractor for the IPC/TNC portions was hired in May 2001, and

the originally scheduled completion date for these components was

May 2004. A different contractor was hired in June 2001 to complete

the UAC portion of Trilogy by June 2004.

After the terrorist attacks on September 11, 2001, the urgency

of completing Trilogy increased, and the FBI explored options to

accelerate the deployment of all three components of Trilogy. The FBI

informed Congress in February 2002 that, with an additional

$70 million, the FBI could accelerate the deployment of Trilogy. This

acceleration would include completion of the IPC/TNC phase by

July 2002 and rapid deployment of the most critical analytical tools

included as part of the UAC phase.

- ix -

In January 2002, Congress supplemented Trilogy’s FY 2002

budget with $78 million

4

to expedite the deployment of all three

components. This supplemental appropriation increased the total

funding of Trilogy from approximately $380 million to $458 million.

Even with these additional funds, the FBI missed its July 2002

milestone date for completing the IPC and TNC phases. FBI officials

stated that they are not expecting these components of Trilogy to be

completed until March 2003. In addition, the user application

component of Trilogy, recognized by FBI officials as the most

important aspect of the project in terms of improving agent

performance, is at high risk of not being completed within the funding

levels appropriated by Congress. Further, despite receiving an

additional $78 million from Congress in January 2002, FBI managers

have acknowledged to us that the last phase of UAC will not be

completed any sooner than originally planned (in June 2004).

In terms of a cost baseline, FBI officials told us that the rapid

procurement and deployment of Trilogy has prevented the project

managers from performing earned value management,

5

as promised

to Congress. While FBI officials were confident they know how much

money has been spent on Trilogy to date, and how much funding has

been committed, they have less assurance as to whether Trilogy is on

budget, over budget, or under budget.

A schedule baseline for Trilogy has never been well-established.

First, FBI officials said they would complete IPC/TNC deployment in

May 2004. Then, they said it could be finished in June 2003. Next,

they said it would be finished by December 2002. After receiving

$78 million of supplemental funding, they said it would be done by

July 2002. Then, they said they could not make the July 2002

deadline and moved it to October 2002. As of June 2002, FBI officials

have said deployment will probably not be complete until March 2003.

Also as of June 2002, the FBI was still in the process of building a

comprehensive schedule of Trilogy milestones.

Regarding the technical requirements for Trilogy, we were told

that some aspects of Trilogy as submitted to Congress did not turn out

to be technically feasible. For example, FBI officials told us that the

4

The $78 million is comprised of the $70 million that FBI requested for

acceleration, plus $8 million for contractor support.

5

Earned value management is a project monitoring method that compares

the value of products and services received with funds that have been expended.

- x -

thin-client strategy was not pursued because it was found that this

type of network could not be achieved given the technical

requirements of the FBI.

6

Another example is web-enablement of the

Automated Case Support (ACS) system, which was also discontinued

when it was realized that it would require more resources than

anticipated.

7

Had a more rigorous proposal selection process been in

place to require sufficient documentation of the technical requirements

and risks of the project, the expending of time and resources on thin-

client technology and web-enablement of ACS may have been

minimized.

Another technical issue involves the development of the UAC

portion of Trilogy. Because the UAC portion is focused on making

significant changes to, or possibly complete replacements of, five of

the FBI’s investigative systems, documentation for the exact

configuration of these systems is critical to designing the requirements

for UAC. According to a senior FBI official, the FBI must know what it

has before it can define the right solution to fix the problem. Lack of

documentation for the configuration of these five investigative systems

has caused the FBI to engage in a process of reverse engineering,

which is trying to determine the structure and components of the

systems after deployment. Because the FBI has to perform reverse

engineering on the FBI’s five investigative systems, there are

limitations as to how rapidly UAC can be developed and deployed.

Our observations at five FBI field offices indicated that

deployment of the IT physical infrastructure was still ongoing as of

June 2002. For two field offices, additional installation work remained

to be completed, and for four field offices hundreds of desktop

computers still remained to be delivered. A lack of clear

communication between FBI Headquarters and the field offices

contributed to the confusion over the number of desktop computers to

be delivered and shortages of fiber optic cable. Additionally contractor

maintenance support for the Trilogy architecture was inefficient,

resulting in agents being without computers for weeks at a time.

Improvements in agent and support personnel training, procurement

of trouble-shooting equipment for the Trilogy architecture, and timely

6

According to the FBI, a thin-client strategy would utilize application software

that is run from the server computer, and consequently permit desktop computers to

function with few hardware resources such as processors and memory.

7

Web-enablement refers to the ability of the software application to interface

with the Internet through a browser, thereby extending information access.

- xi -

completion of FBI unique macros for Microsoft Word will enhance user

utilization of the Trilogy architecture.

The new Trilogy project executive, hired in March 2002, has

taken a different approach to managing Trilogy. She has emphasized

the importance of having more structured oversight of the project.

She has been developing a comprehensive schedule for all three

components. Additionally, she has indicated that there are limitations

to how fast Trilogy can be deployed, without risking the security of the

system. In our judgment, while these actions taken since March 2002

represent positive changes to Trilogy’s project management function,

the project’s completion time, final cost, and ultimate performance

remain uncertain. Also, we concluded that for the Trilogy project

management function to be effective, it must include oversight from IT

investment review boards to provide much needed monitoring.

5. FBI’s IT Strategic Planning and Performance Measurement

We also assessed the FBI’s IT strategic planning and

performance measurement. We found that the FBI’s strategic plan

does not include IT investment management goals and the FBI’s

strategic plan and performance plan are not consistent with the DOJ’s

annual performance plan. Also, as of the end of June 2002, the FBI

did not have a current strategic plan dedicated to IT. Instead,

individual FBI divisions had program plans that included the use of IT

within particular programs.

This occurred because the FBI has not updated its strategic plan

since 1998, and its performance plan does not include the same

strategic objectives, goals, and strategies relating to IT as does the

DOJ's annual performance plan. We believe that the FBI will have

difficulty improving its IT investment management process without

incorporating it into the strategic plan. Additionally, without adequate

strategic planning and performance measurements, there is a

heightened risk that the FBI may not be appropriately allocating

resources to meet the DOJ’s strategic priorities.

In our judgment, the FBI must change the division-specific IT

focus and implement a Bureau-wide IT strategic plan. The purpose of

the FBI’s ITIM process is to move away from the decentralized IT focus

to a centralized one. As a result, we recommend that the FBI update

its IT strategic plan and performance plans to (1) fully integrate these

plans with the FBI’s ITIM process; and (2) include those performance

goals and indicators defined in the DOJ’s IT Strategic Plan.

- xii -

6. OIG Recommendations

In this report, we make 30 recommendations that focus on

specific and immediate steps the FBI should take to help improve its IT

investment management. These recommendations include:

•

Ensure that the FBI continues its efforts to establish a

comprehensive enterprise architecture that is integrated with the

ITIM process.

•

Require the ITIM Program Office to plan for and allocate

sufficient time for IT investment review board members and

other ITIM users to execute assigned responsibilities

competently.

•

Ensure that members of IT investment boards and other ITIM

users receive sufficient training to execute assigned

responsibilities effectively.

•

Ensure that official project management guidance is used for all

FBI IT projects through management oversight from the IT

investment review boards.

•

Ensure that each IT project has a project management plan,

approved by the IT investment review boards, that includes cost

and schedule controls.

•

Ensure that a complete IT asset inventory is developed, and

information from the IT asset inventory is made available to, and

used by, the IT investment review boards as necessary.

•

Ensure that the FBI develops written policies and procedures for

identifying the business needs (and the associated users) of each

IT project.

•

Ensure that identified users participate in project management

throughout a project's life-cycle.

•

Ensure that the policies and procedures of the ITIM process are

expanded, documented, and made available to ITIM users.

•

Ensure that the ITIM Program Office and the ITIM contractor

incorporate the input from various ITIM users through

- xiii -

working group sessions as the ITIM process is being further

developed and refined.

•

Ensure that the FBI develops and implements a specific plan

detailing how and when it will integrate the ITIM process with a

system development life-cycle methodology.

7. Conclusion

The underlying practices we assessed are fundamental to any

proje