United States Department of Justice
Office of the Inspector General
Audit Division
A
UDIT
R
EPORT
F
EDERAL
B
UREAU OF
I
NVESTIGATION
S
M
ANAGEMENT OF
I
NFORMATION
T
ECHNOLOGY
I
NVESTMENTS
D
ECEMBER
2002
03-09
pg_0002
FEDERAL BUREAU OF INVESTIGATION’S MANAGEMENT
OF INFORMATION TECHNOLOGY INVESTMENTS
EXECUTIVE SUMMARY
Following the September 11, 2001, terrorist attacks, the Attorney
General and the Director of the Federal Bureau of Investigation (FBI)
made clear that prevention of terrorism is the top priority of the
Department of Justice (DOJ) and the FBI. Effective use of information
technology (IT) is crucial to the FBI’s ability to meet this priority as well
as its other critical responsibilities.
However, reviews conducted by the Office of the Inspector
General (OIG) and the General Accounting Office (GAO) have found
major weaknesses associated with the FBI’s IT. The FBI has listed
upgrading its information technology as one of its top ten highest
priorities. In June 2002 Congressional testimony, the FBI
acknowledged that its IT infrastructure is severely outdated.
Because of the importance of the FBI’s management of its IT
systems, we performed this audit to: (1) determine whether the FBI
was effectively managing its IT investments; and (2) assess the FBI’s
IT-related strategic planning and performance measurement activities.
1
We also examined the FBI’s efforts to develop enterprise architecture
2
and project management capabilities.
In this audit, we conducted approximately 85 interviews with
70 officials from the FBI, DOJ, GAO, and the Office of Management and
Budget (OMB). The FBI officials interviewed were from the Director’s
office, Information Resources Division, Criminal Justice Information
Services Division, Laboratory Division, Inspection Division, and Finance
1
During our audit fieldwork, we initiated work relating to a third objective: to
determine if the FBI has implemented prior information technology related
recommendations directed toward improving information technology. We will issue a
separate report on this objective.
2
Enterprise architecture is the organization-wide blueprint that defines an
entity’s functions and systems, including IT systems. It provides a comprehensive
view (through models, narratives, and diagrams) of the interrelationships of an
organization’s operations and structures and how these structures align with the
organization’s mission. The Clinger-Cohen Act of 1996 recognizes the
interrelationship between enterprise architecture and IT investment management by
requiring federal agencies to develop an enterprise architecture.
- i -
pg_0003
Division. Additionally, OIG auditors and analysts traveled to FBI
laboratory facilities in Quantico, VA, and five FBI field offices to
conduct interviews and assess the FBI’s implementation of IT
initiatives. We also reviewed more than 200 documents, including the
FBI’s IT management policies and procedures, project management
guidance, strategic and program plans, IT project proposals and
management plans, budget documentation, organizational structures,
Congressional testimony, and prior OIG and GAO reports.
1. Summary of Audit Findings
We concluded that the FBI has not effectively managed its IT
investments because it has not fully implemented the management
processes associated with successful IT investments. The foundation
for sound IT investment management (ITIM) includes the following
fundamental elements:
defining and developing IT investment boards;
following a disciplined process of tracking and overseeing each
project’s cost and schedule milestones over time;
identifying existing IT systems and projects;
identifying the business needs for each IT project; and
using defined processes to select new IT project proposals.
The FBI failed to implement these critical processes. We found
that the FBI does not have fully functioning IT investment boards that
are engaged in all phases of IT investment management. The FBI was
not following a disciplined process of tracking and overseeing each
project’s cost and schedule milestones. The FBI failed to document a
complete inventory of existing IT systems and projects, and did not
consistently identify the business needs for each IT project. The FBI
did not have a fully established process for selecting new IT project
proposals that considered both existing IT projects and new projects.
Because the FBI has not fully implemented the critical processes
associated with effective IT investment management, the FBI
continues to spend hundreds of millions of dollars on IT projects
without adequate assurance that these projects will meet their
intended goals.
- ii -
pg_0004
We concluded that these shortcomings primarily resulted from
the FBI not devoting sufficient management attention in the past to IT
investment management.
However, FBI management has recognized that its past methods
to manage IT projects have been deficient, and the FBI recently has
committed to changing those practices. In January 2002, the FBI
developed a conceptual model for selecting, controlling, and evaluating
IT investments. The model seeks to define a process that will promote
a Bureau-wide perspective on IT investment management, so that only
IT projects with the best probability of improving mission performance
are selected. Further, the process is intended to provide the methods,
structures, disciplines, and management framework that governs the
way IT projects are controlled and evaluated.
In addition to developing a conceptual model for a new ITIM
process, in early 2002 the FBI began a pilot test of the new process for
the selection of IT proposals. We found that the FBI made
improvements during the pilot testing of the new selection process.
Pursuant to the new process, the FBI created three IT investment
review boards that reviewed IT proposals for technical compliance and
“mission fit.” These boards, comprised of the FBI Director, FBI
executives and IT managers, selected new IT proposals that will be
considered for inclusion in the Fiscal Year (FY) 2004 budget request.
While the FBI has made efforts to improve its IT investment
management practices, the FBI must take further actions to ensure
that it can implement the fundamental processes necessary to build an
IT investment foundation, as well as the more mature processes
associated with highly effective IT investment management. These
actions include:
fully developing and documenting its new IT investment
management process – which is necessary to completely
implement the activities defined in the FBI’s conceptual model;
requiring increased participation from IT program managers and
users – which is necessary to ensure senior management
acceptance and foster understanding and institutionalization of
the ITIM process; and
further developing the FBI’s project management and enterprise
architecture functions – which is necessary to execute the
- iii -
pg_0005
control and evaluate components of the ITIM process as well as
advance its investment management capability.
Our audit also reviewed the FBI’s management of Trilogy, the
FBI’s largest and most critical IT project. We found that the lack of
critical IT investment management processes contributed to missed
milestones and led to uncertainties about cost, schedule, and technical
goals. Specifically, despite $78 million in additional funding, the FBI
missed its July 2002 milestone date for completing the physical IT
infrastructure upgrades to field offices, including new computer
hardware and networks.
3
FBI officials stated that they are not
expecting the physical infrastructure components of Trilogy to be
completed until March 2003. In addition, the user application
component of Trilogy, recognized by FBI officials as the most
important aspect of the project in terms of improving agent
performance, is at high risk of not being completed within the funding
levels appropriated by Congress. In our judgment, the management
problems associated with Trilogy demonstrate the FBI’s urgent need
for enhanced IT investment management.
We also concluded that the FBI’s IT strategic planning and IT
performance measurement are inadequate. We found that the FBI's
strategic plan does not include goals for IT investment management,
and the FBI’s strategic plan and performance plan are not consistent
with the DOJ’s annual performance plan.
The remainder of this executive summary provides more
background and details on our audit findings and recommendations to
help improve the FBI’s management of its IT investments.
2. Background
The Clinger-Cohen Act of 1996 requires each federal agency to
implement a process for maximizing the value of its IT investments.
This process is intended to ensure that IT projects are being
implemented at acceptable costs and within reasonable time frames,
and that the projects are contributing to enhanced mission
performance. Specifically, the Clinger-Cohen Act requires federal
agencies to: (1) develop an enterprise architecture framework, and
3
With the $78 million in additional funding, Trilogy’s total appropriation was
$458 million as of June 2002.
- iv -
pg_0006
(2) follow a “select/control/evaluate” approach to managing IT
investments.
In May 2000, the GAO developed the IT Investment
Management Framework (Framework) to provide a common
methodology for assessing IT capital planning and investment
management practices at federal agencies. The Framework specifically
describes the organizational processes required to carry out sound IT
investment management.
The Framework, based on best practices of leading
organizations, is a hierarchical model comprised of five maturity
stages. These maturity stages represent steps toward achieving stable
and mature investment management processes. As agencies advance
through these stages, their capability to effectively manage IT
increases. With the exception of the first stage, each maturity stage is
comprised of critical processes that must be implemented and
institutionalized for the agency to satisfy the requirements of that
stage. These critical processes are further broken down into key
practices an agency should perform to successfully implement each
critical process.
An agency using these critical processes is in a better position to
successfully invest in IT and use its IT investments to achieve its
priorities. Conversely, an agency that does not have these critical
processes in place is at high risk that its IT projects will fail to support
the achievement of priorities.
To determine whether the FBI was effectively managing its IT
investments, we utilized the Framework because it is: (1) a
standardized tool for internal and external evaluations of an agency’s
IT investment management process; (2) a consistent and
understandable mechanism for reporting the results of these
assessments; and (3) a road map agencies can use for improving their
IT investment management process.
In addition, the Government Performance and Results Act of
1993 (Results Act) requires strategic planning and performance
measurement throughout the federal government. The Results Act
seeks to improve the effectiveness, efficiency, and accountability of
federal programs by requiring federal agencies to establish goals for
program performance and measurement. The Results Act requires
agencies to prepare a strategic plan, annual performance plan, and
annual performance report.
- v -
pg_0007
While IT strategic planning is a function somewhat independent
of IT investment management, these two functions are interrelated
and complementary. The DOJ has recognized the importance of
integrating strategic planning with IT management. In July 2002, the
DOJ released its IT Strategic Plan that included a strategic initiative to
establish and improve investment management processes.
3. The FBI’s Management of IT Investments
Our audit found that the FBI has not established an IT
investment foundation and therefore is in Stage One maturity
according to the ITIM Framework. Stage One maturity is characterized
by inconsistent, unstructured, and unpredictable investment
processes. Our observations of the FBI’s IT investment processes
found that the FBI’s actual processes are consistent with these
Stage One deficiencies.
The critical processes necessary to establish an IT investment
foundation include: (1) defining investment review board operations,
(2) developing project-level investment control processes,
(3) identifying IT projects and systems, (4) identifying the business
needs for each IT project, and (5) developing a basic process for
selecting new IT proposals.
We found that the FBI failed to implement these critical
processes. The FBI did not have a fully established investment review
board operation because the FBI did not provide adequate resources
for operating the IT investment boards. Additionally, we found
insufficient evidence to demonstrate that: (1) organization executives
and line managers supported and carried out IT investment board
decisions and (2) board members understood the investment board’s
policies and procedures and exhibited core competencies in using the
IT investment approach via training, education, or experience.
Specifically, the FBI did not provide ample time to adequately prepare
and train IT board members prior to initiating the pilot test of its
recently developed ITIM process. This resulted in inadequate training
of board members and minimal preparation time to develop IT
proposals. For example, Technical Review Board members had only
three business days to review over 50 IT proposals prior to their first
board meeting.
Additionally, we found that the FBI is not effectively overseeing
its IT projects. For example, while the FBI has issued project
management guidance, the guidance is not being followed on a
- vi -
pg_0008
consistent basis. Depending on whom we talked to, we obtained
different answers as to which document represented the FBI’s official
project management guidance.
Without effective oversight of IT projects, FBI officials do not
have adequate assurance that IT projects are being developed on
schedule and within established budgets. According to a former Chief
Information Officer at the FBI, the lack of effective oversight of IT
projects has prevented IT project managers from being held
accountable for cost and schedule overruns and the ultimate
performance of projects. Senior FBI officials also told us that the
Bureau’s budget formulation process focuses only on the acquisition
costs for IT projects and not the full life-cycle costs, especially
operations and maintenance costs.
We also found that the FBI’s investment review boards are not
aware of all the IT projects and resources for which the boards are
responsible. FBI Divisions maintained some version of an IT inventory
for the projects and systems under their jurisdiction, and there was no
centralized office responsible for maintaining a uniform listing Bureau-
wide. FBI managers told us they were in the process of developing an
IT asset inventory, but at the time of our audit they were unable to
provide an estimated date for completing the inventory.
FBI personnel told us that staff shortages are the primary cause
for the incomplete IT asset inventory. In our judgment, staff
shortages may be a contributing factor, but the lack of centralized
management over IT investments was the significant reason for this
problem. Until June 2002, the FBI did not have a centralized project
management office to assist the investment boards in overseeing IT
projects. The FBI maintained three separate division-level project
management offices to manage IT projects.
We also determined that the FBI did not have a fully established
process for selecting IT proposals. FBI officials told us that, prior to
March 2002, individual divisions determined IT needs in a “stovepipe,”
without knowledge of the business needs and priorities of the Bureau
as a whole. The FBI did not have a clearly designated official to
manage the proposal selection process. According to Information
Resources Management Section personnel, the Finance Division
managed the IT selection process. However, according to Finance
Division personnel, the Information Resources Management office was
responsible for managing the proposal selection process.
- vii -
pg_0009
Without a comprehensive proposal selection process that
includes adequate resources and training, the FBI cannot ensure that it
is selecting the best IT projects that meet mission-critical needs.
Because the FBI did not fully implement any of the critical
processes associated with Stage Two, the FBI continues to spend
hundreds of millions of dollars on IT projects without having adequate
selection and project management controls in place to ensure that IT
projects will deliver their intended benefits.
The FBI began pilot testing the select phase of its new ITIM
process in March 2002, and since then has made measurable progress
towards implementing the key practices that comprise the critical
processes – particularly in the area of selecting new proposals for IT
projects. Specifically, at the beginning of our audit in January 2002,
the FBI only was executing 4 of the 38 required key practices;
however, as of June 2002, the FBI was executing 14 of the key
practices.
With the pilot testing of its new ITIM process, the FBI created an
IT investment process guide containing policies and procedures to
direct board operations, and created and defined three investment
review boards integrating both IT and business knowledge.
Additionally, the FBI has designated an official responsible for
managing the IT project and system identification process and
ensuring that the inventory meets the needs of the investment
management process. Further, during the test pilot of the ITIM
process, the board reviews of IT project proposals provided assurance
that business needs were clearly identified and defined. Also during
the test pilot, we determined that FBI IT investment board members
analyzed and prioritized new IT proposals according to established
selection criteria for the FY 2004 budget cycle.
Despite the progress made, full implementation of the ITIM
process will require the FBI to (1) fully develop and document its new
ITIM process; (2) require more input and participation from IT
managers and users; and (3) further develop its project management
and enterprise architecture functions. Completion of the initial steps
taken by the FBI will ensure that IT projects are developed within cost
and schedule requirements, and meet performance expectations. The
Trilogy project provides an example of how the non-implementation of
fundamental IT investment management practices can put a project at
risk of not delivering what was promised, within cost and schedule
requirements.
- viii -
pg_0010
4. Trilogy
We also performed a case study of the FBI’s implementation of
its Trilogy project. We selected Trilogy because it is the FBI’s largest
ongoing IT project and is considered vital to the FBI’s ability to
perform its mission. Trilogy is intended to upgrade the FBI’s:
(1) hardware and software – referred to as the Information
Presentation Component (IPC), (2) communication networks – referred
to as the Transportation Network Component (TNC), and (3) five most
important investigative applications – referred to as the User
Applications Component (UAC). The IPC and TNC upgrades will
provide the physical infrastructure needed to run the applications from
the UAC portion. The UAC portion is intended to upgrade and
consolidate five of the FBI’s 42 investigative applications. Because of
the 37 other investigative applications and approximately 160 non-
investigative applications that Trilogy will not cover, Trilogy is only a
starting point towards upgrading the FBI’s entire IT infrastructure.
According to the FBI, Trilogy is not designed to provide the FBI with
state-of-the-art IT; it is intended to provide the foundation so that the
FBI can eventually attain state-of-the-art IT.
In November 2000, Congress appropriated $100.7 million for the
first year of the $379.8 million Trilogy project, which was to be funded
over a three-year period (from the date contractors were hired). The
$100.7 million was a combination of new program funding and a
re-direction of base resources. When the FBI requested contractor
support for Trilogy, it combined the IPC and TNC portions for
continuity as both encompass physical IT infrastructure enhancements.
The contractor for the IPC/TNC portions was hired in May 2001, and
the originally scheduled completion date for these components was
May 2004. A different contractor was hired in June 2001 to complete
the UAC portion of Trilogy by June 2004.
After the terrorist attacks on September 11, 2001, the urgency
of completing Trilogy increased, and the FBI explored options to
accelerate the deployment of all three components of Trilogy. The FBI
informed Congress in February 2002 that, with an additional
$70 million, the FBI could accelerate the deployment of Trilogy. This
acceleration would include completion of the IPC/TNC phase by
July 2002 and rapid deployment of the most critical analytical tools
included as part of the UAC phase.
- ix -
pg_0011
In January 2002, Congress supplemented Trilogy’s FY 2002
budget with $78 million
4
to expedite the deployment of all three
components. This supplemental appropriation increased the total
funding of Trilogy from approximately $380 million to $458 million.
Even with these additional funds, the FBI missed its July 2002
milestone date for completing the IPC and TNC phases. FBI officials
stated that they are not expecting these components of Trilogy to be
completed until March 2003. In addition, the user application
component of Trilogy, recognized by FBI officials as the most
important aspect of the project in terms of improving agent
performance, is at high risk of not being completed within the funding
levels appropriated by Congress. Further, despite receiving an
additional $78 million from Congress in January 2002, FBI managers
have acknowledged to us that the last phase of UAC will not be
completed any sooner than originally planned (in June 2004).
In terms of a cost baseline, FBI officials told us that the rapid
procurement and deployment of Trilogy has prevented the project
managers from performing earned value management,
5
as promised
to Congress. While FBI officials were confident they know how much
money has been spent on Trilogy to date, and how much funding has
been committed, they have less assurance as to whether Trilogy is on
budget, over budget, or under budget.
A schedule baseline for Trilogy has never been well-established.
First, FBI officials said they would complete IPC/TNC deployment in
May 2004. Then, they said it could be finished in June 2003. Next,
they said it would be finished by December 2002. After receiving
$78 million of supplemental funding, they said it would be done by
July 2002. Then, they said they could not make the July 2002
deadline and moved it to October 2002. As of June 2002, FBI officials
have said deployment will probably not be complete until March 2003.
Also as of June 2002, the FBI was still in the process of building a
comprehensive schedule of Trilogy milestones.
Regarding the technical requirements for Trilogy, we were told
that some aspects of Trilogy as submitted to Congress did not turn out
to be technically feasible. For example, FBI officials told us that the
4
The $78 million is comprised of the $70 million that FBI requested for
acceleration, plus $8 million for contractor support.
5
Earned value management is a project monitoring method that compares
the value of products and services received with funds that have been expended.
- x -
pg_0012
thin-client strategy was not pursued because it was found that this
type of network could not be achieved given the technical
requirements of the FBI.
6
Another example is web-enablement of the
Automated Case Support (ACS) system, which was also discontinued
when it was realized that it would require more resources than
anticipated.
7
Had a more rigorous proposal selection process been in
place to require sufficient documentation of the technical requirements
and risks of the project, the expending of time and resources on thin-
client technology and web-enablement of ACS may have been
minimized.
Another technical issue involves the development of the UAC
portion of Trilogy. Because the UAC portion is focused on making
significant changes to, or possibly complete replacements of, five of
the FBI’s investigative systems, documentation for the exact
configuration of these systems is critical to designing the requirements
for UAC. According to a senior FBI official, the FBI must know what it
has before it can define the right solution to fix the problem. Lack of
documentation for the configuration of these five investigative systems
has caused the FBI to engage in a process of reverse engineering,
which is trying to determine the structure and components of the
systems after deployment. Because the FBI has to perform reverse
engineering on the FBI’s five investigative systems, there are
limitations as to how rapidly UAC can be developed and deployed.
Our observations at five FBI field offices indicated that
deployment of the IT physical infrastructure was still ongoing as of
June 2002. For two field offices, additional installation work remained
to be completed, and for four field offices hundreds of desktop
computers still remained to be delivered. A lack of clear
communication between FBI Headquarters and the field offices
contributed to the confusion over the number of desktop computers to
be delivered and shortages of fiber optic cable. Additionally contractor
maintenance support for the Trilogy architecture was inefficient,
resulting in agents being without computers for weeks at a time.
Improvements in agent and support personnel training, procurement
of trouble-shooting equipment for the Trilogy architecture, and timely
6
According to the FBI, a thin-client strategy would utilize application software
that is run from the server computer, and consequently permit desktop computers to
function with few hardware resources such as processors and memory.
7
Web-enablement refers to the ability of the software application to interface
with the Internet through a browser, thereby extending information access.
- xi -
pg_0013
completion of FBI unique macros for Microsoft Word will enhance user
utilization of the Trilogy architecture.
The new Trilogy project executive, hired in March 2002, has
taken a different approach to managing Trilogy. She has emphasized
the importance of having more structured oversight of the project.
She has been developing a comprehensive schedule for all three
components. Additionally, she has indicated that there are limitations
to how fast Trilogy can be deployed, without risking the security of the
system. In our judgment, while these actions taken since March 2002
represent positive changes to Trilogy’s project management function,
the project’s completion time, final cost, and ultimate performance
remain uncertain. Also, we concluded that for the Trilogy project
management function to be effective, it must include oversight from IT
investment review boards to provide much needed monitoring.
5. FBI’s IT Strategic Planning and Performance Measurement
We also assessed the FBI’s IT strategic planning and
performance measurement. We found that the FBI’s strategic plan
does not include IT investment management goals and the FBI’s
strategic plan and performance plan are not consistent with the DOJ’s
annual performance plan. Also, as of the end of June 2002, the FBI
did not have a current strategic plan dedicated to IT. Instead,
individual FBI divisions had program plans that included the use of IT
within particular programs.
This occurred because the FBI has not updated its strategic plan
since 1998, and its performance plan does not include the same
strategic objectives, goals, and strategies relating to IT as does the
DOJ's annual performance plan. We believe that the FBI will have
difficulty improving its IT investment management process without
incorporating it into the strategic plan. Additionally, without adequate
strategic planning and performance measurements, there is a
heightened risk that the FBI may not be appropriately allocating
resources to meet the DOJ’s strategic priorities.
In our judgment, the FBI must change the division-specific IT
focus and implement a Bureau-wide IT strategic plan. The purpose of
the FBI’s ITIM process is to move away from the decentralized IT focus
to a centralized one. As a result, we recommend that the FBI update
its IT strategic plan and performance plans to (1) fully integrate these
plans with the FBI’s ITIM process; and (2) include those performance
goals and indicators defined in the DOJ’s IT Strategic Plan.
- xii -
pg_0014
6. OIG Recommendations
In this report, we make 30 recommendations that focus on
specific and immediate steps the FBI should take to help improve its IT
investment management. These recommendations include:
Ensure that the FBI continues its efforts to establish a
comprehensive enterprise architecture that is integrated with the
ITIM process.
Require the ITIM Program Office to plan for and allocate
sufficient time for IT investment review board members and
other ITIM users to execute assigned responsibilities
competently.
Ensure that members of IT investment boards and other ITIM
users receive sufficient training to execute assigned
responsibilities effectively.
Ensure that official project management guidance is used for all
FBI IT projects through management oversight from the IT
investment review boards.
Ensure that each IT project has a project management plan,
approved by the IT investment review boards, that includes cost
and schedule controls.
Ensure that a complete IT asset inventory is developed, and
information from the IT asset inventory is made available to, and
used by, the IT investment review boards as necessary.
Ensure that the FBI develops written policies and procedures for
identifying the business needs (and the associated users) of each
IT project.
Ensure that identified users participate in project management
throughout a project's life-cycle.
Ensure that the policies and procedures of the ITIM process are
expanded, documented, and made available to ITIM users.
Ensure that the ITIM Program Office and the ITIM contractor
incorporate the input from various ITIM users through
- xiii -
pg_0015
working group sessions as the ITIM process is being further
developed and refined.
Ensure that the FBI develops and implements a specific plan
detailing how and when it will integrate the ITIM process with a
system development life-cycle methodology.
7. Conclusion
The underlying practices we assessed are fundamental to any
project management endeavor. However, the FBI has not executed
the majority of these tasks to select and manage its IT resources. For
example, organizational policies were not clearly established to ensure
that critical IT investment policies endure. Additionally, there were no
clearly defined, uniform procedures for project management, tracking
project performance, and taking corrective actions as necessary. Prior
to the development of its ITIM process in early 2002, the FBI did not
give sufficient attention to IT investment management. Since the FBI
developed its ITIM process in early 2002, it has focused more
management attention in this area and has made progress towards
attaining a basic IT investment management foundation. Despite the
progress, the FBI did not fully implement any of the critical processes
necessary to build an IT investment foundation. As a result, the FBI
continues to spend hundreds of millions of dollars on IT projects
without having adequate selection and project management controls in
place to ensure that IT projects will deliver their intended benefits.
- xiv -
pg_0016
TABLE OF CONTENTS
INTRODUCTION .............................................................................1
1. Background ..........................................................................1
2. The FBI’s Management of IT Infrastructure ................................2
3.
Prior Reports on the FBI’s IT and DOJ Oversight of
Components’ IT .....................................................................4
4.
The FBI’s Current IT Investment Efforts ....................................9
5. Trilogy: The FBI’s Largest IT Investment................................ 10
6. Framework for Assessing IT Investment Management ............... 12
7. The DOJ’s ITIM Guidance ...................................................... 17
8. The FBI’s Recent Efforts to Implement an ITIM Process ............. 18
OIG FINDINGS AND RECOMMENDATIONS ....................................... 22
1. The FBI’s Management of IT Investments................................ 22
A. The FBI’s Progress Toward Attaining a Basic IT
Investment Management Foundation................................. 22
B. The FBI’s Ability to Improve its IT Investment
Practices ....................................................................... 60
C. Trilogy Case Study .......................................................... 86
2. The FBI’s IT Strategic Planning and Performance
Measurement .................................................................... 114
A. Background on Strategic Planning ................................... 114
B. Strategic Planning’s Relationship to the ITIM Process......... 116
C. Results of our Assessment of the FBI’s IT Strategic
Planning and Performance Measurement.......................... 117
D. Summary .................................................................... 118
E. Recommendation ......................................................... 118
STATEMENT ON COMPLIANCE WITH LAWS AND REGULATIONS ........ 119
STATEMENT ON MANAGEMENT CONTROLS .................................... 120
APPENDIX 1: OBJECTIVES, SCOPE, AND METHODOLOGY................ 121
pg_0017
APPENDIX 2: FLOWCHART OF FBI’S ITIM CONTROL PHASE ............. 125
APPENDIX 3: FLOWCHART OF FBI’S ITIM EVALUATE PHASE ............ 126
APPENDIX 4: JMD’S ASSESSMENT OF THE FBI’S ITIM
PROCESS ............................................................. 127
APPENDIX 5: GAO’S FIVE STAGES OF ENTERPRISE
ARCHITECTURE MATURITY...................................... 133
APPENDIX 6: FBI’S ENTERPRISE ARCHITECTURE MATURITY
SURVEY ............................................................... 135
APPENDIX 7: FBI
S RESPONSE TO THE DRAFT REPORT .................. 136
APPENDIX 8: OIG, AUDIT DIVISION ANALYSES AND
SUMMARY OF ACTIONS NECESSARY TO
TO CLOSE REPORT ................................................ 153
pg_0018
INTRODUCTION
1. Background
The Federal Bureau of Investigation (FBI or Bureau) is the
principal investigative arm of the Department of Justice (DOJ). To
execute its responsibilities, the FBI’s Headquarters in Washington, D.C.
provides program direction and support services to 56 field offices,
approximately 400 satellite offices known as resident agencies and
more than 40 foreign liaison posts.
As of June 2002, the FBI had over 11,000 Special Agents and
over 16,000 other employees who performed professional,
administrative, technical, clerical, craft, trade, or maintenance
operations. The FBI’s budget authority increased 31 percent from
$3.339 billion in FY 2001 to nearly $4.371 billion in FY 2002.
8
Of this
budget authority, $714 million was allocated to information technology
(IT) projects in FY 2002 compared to $353 million in FY 2001.
The terrorist attacks of September 11, 2001, prompted the
Attorney General to make counterterrorism the DOJ’s highest priority.
The DOJ reflected these new priorities in its Strategic Plan for Fiscal
Years 2001 – 2006, which was issued in November 2001. In the
Strategic Plan, the Attorney General recognized that the fight against
terrorism requires the DOJ “to improve the integrity and security of its
computer systems and make more effective use of information
technology.”
In response to the DOJ’s new priorities following September 11,
2001, the FBI proposed fundamental changes in its strategic priorities
and business practices. In May 2002, the Director of the FBI
announced a major reorganization that dedicates more resources to
the prevention of terrorism.
9
Although the core missions of the FBI
remain intact, the proposed changes would transform the Bureau’s
role from reactive to preventive. To accomplish this transition, FBI
officials have repeatedly told Congress that new and improved IT is
required to support a redesigned and refocused FBI. In testimony
8
These figures were taken from the DOJ’s website (
www.usdoj.gov
). They
include a $745 million Counterterrorism Supplemental for FY 2002 and exclude
Federal Retiree and Health Benefit Costs.
9
This reorganization was approved by Congress on July 31, 2002.
- 1 -
pg_0019
before the Senate Judiciary Committee on June 6, 2002, the Director
released the FBI’s top ten priorities in the post-September 11 era, with
the number one priority being protecting the United States from
terrorist attacks. Number ten on the list of priorities is upgrading
technology to successfully perform the FBI’s mission. Clearly, the
FBI’s future ability to prevent terrorism and other crimes depends on
modern information technology and effective management of
technology.
2. The FBI’s Management of IT Infrastructure
The FBI has three divisions that manage major IT projects: the
Information Resources Division (IRD), the Criminal Justice Information
Services Division (CJIS), and the Laboratory Division. As discussed
below, the FBI is attempting to centralize the management of IT,
rather than manage IT within divisions.
The IRD provides the day-to-day support services to manage the
information systems of the FBI. The IRD’s responsibilities include
management of all hardware, software, and IT peripheral equipment
located at the FBI’s Headquarters, field offices, and other offsite
locations.
The IRD has been restructured in recent years to increase the
oversight and jurisdiction of the Chief Information Officer. Until
November 2001, the Chief Information Officer of the FBI was the
Assistant Director of IRD who reported to the Director. However, to
give the Chief Information Officer greater authority over the entire
FBI, the Chief Information Officer was moved out of IRD and into the
Director’s office, pursuant to a restructuring approved by Congress on
November 30, 2001. Additionally, to support the Chief Information
Officer, the Information Resources Management Section
10
was moved
out of IRD and into the Chief Information Officer’s office, following
another restructuring in February 2002. Also, in February 2002, the IT
Investment Management Program Office was formed (within the
Information Resources Management Section) and was staffed with one
individual whose responsibility was to manage the FBI’s IT investment
management program. Based on these actions, the FBI recognizes
that centralizing the management of IT requires a Chief Information
Officer to have Bureau-wide oversight and jurisdiction, rather than be
isolated within a division.
10
The Information Resources Management Section is responsible for
managing IT investments and enterprise architecture.
- 2 -
pg_0020
The CJIS Division uses several significant IT systems to manage
and disseminate relevant criminal justice information to the FBI and
other law enforcement agencies. For example, the
National Crime Information Center 2000 is a nationwide information
system that supports federal, state, and local law enforcement
agencies. Additionally, the CJIS Division is responsible for
managing
the Integrated Automated Fingerprint Identification System and the
National Incident-Based Reporting System. To support the
management of these systems, the CJIS Division maintains a
Contract Administration Office, which provides quality assurance,
configuration management, and project management support services
necessary to manage these and other systems under its jurisdiction.
The Laboratory Division manages several forensic computer
systems that provide forensic and technical services to law
enforcement agencies. A significant system includes the Combined
DNA Index System (CODIS), which provides software and support
services to state and local laboratories to establish databases of
criminals, unsolved crime scenes, and missing persons. A component
of CODIS, the National DNA Index System, shares DNA profiles from
convicted offenders and crime scenes to laboratories throughout the
United States. To manage these systems, the Laboratory Division
maintains its own project management office.
The FBI has recognized that its IT infrastructure was significantly
outdated and did not effectively support user needs. Although recent
upgrades have changed these numbers, as of September 2000, over
13,000 desktop computers were 4 to 8 years old and could not run
basic software packages, some communication networks were up to
12 years old and were obsolete, and multiple user-applications existed
that were neither web-enabled
11
nor user-friendly.
12
On June 6, 2002,
the Director stated to the Senate Judiciary Committee:
You’ve heard me talk about the necessity for upgrading our
technology. And upgrading our technology means not just
getting the computers on board, the hard drives. It means
everybody from top to bottom becoming facile with the
11
Web-enablement refers to the ability of the software application to interface
with the Internet through a browser, thereby extending information access.
12
According to FBI officials, the FBI acknowledged these needs to Congress in
the late 1990s, in addition to the technology upgrade plan prepared in September
2000.
- 3 -
pg_0021
computer, understanding the computer and understanding
how technology can assist us to do our jobs better. And
that is somewhat of a transformation for an organization
such as the FBI, which is years behind where it should be,
in terms of having the technological infrastructure.
3. Prior Reports on the FBI’s IT and DOJ Oversight of
Components’ IT
Reports issued by the Office of the Inspector General (OIG) over
the past 12 years have highlighted many IT inefficiencies at the FBI.
In 1990, the OIG issued a report entitled, “The FBI’s Automatic Data
Processing General Controls.” This report found
11 major internal control weaknesses, many of which are still
applicable today. Specifically the report stated that:
the FBI’s phased implementation of its 10-year Long Range
Automation Strategy, scheduled for completion in 1990, was
severely behind schedule and may not be accomplished;
the FBI’s Information Resources Management program was
fragmented and ineffective, and the FBI’s Information Resources
Management official did not have effective organization-wide
authority;
the FBI had not developed and implemented a data architecture;
the FBI had not adequately involved top management in FBI
Headquarters or the field offices in systems development
through an Executive Review Committee; and
the FBI’s major mainframe investigative systems were labor
intensive, complex, untimely, and non-user friendly and few
Special Agents used these systems.
Regarding the first weakness, the FBI’s IT infrastructure is still
severely outdated, as we previously mentioned. Regarding the second
weakness, the FBI has recently restructured the IRD and Information
Resources Management Section to reduce the fragmented
management structure that existed among the three divisions
responsible for managing IT. Regarding the third weakness, as
discussed later in the report, the FBI is still developing an enterprise
architecture framework, which includes the technical or data
architecture. Regarding the fourth weakness, as discussed later in the
- 4 -
pg_0022
report, the FBI did not have formally established IT investment review
boards or committees until March 2002. Regarding the fifth weakness,
the FBI’s major investigative systems remain labor intensive, complex,
non-user friendly, and many Special Agents still do not use these
systems.
The OIG’s July 1999 special report on the handling of intelligence
information related to the DOJ’s campaign finance task force
13
stated
that FBI personnel were not well versed in the Automated Case
Support (ACS) system
14
and other databases. Additionally, a
November 1999 report on the death of a federal inmate, Kenneth
Michael Trentadue, noted deficiencies in uploading key evidence into
the ACS.
A March 2002 report entitled, “An Investigation of the Belated
Production of Documents in the Oklahoma City Bombing Case,”
analyzed the causes for the belated production of many documents in
the Oklahoma City bombing case. This report concluded that the ACS
system is extraordinarily difficult to use, has significant deficiencies,
and is not the vehicle for moving the FBI into the 21
st
century. The
report noted that inefficiencies and complexities with the ACS
combined with the lack of a true information management system
were contributing factors in the FBI’s failure to provide hundreds of
investigative documents to the defendants in the Oklahoma City
Bombing Case. These reports illustrate that the FBI has not given
sufficient attention to correcting its deficiencies in information
management and the ACS.
In May 2002, pursuant to the FY 2002 Government Information
Security Reform Act, the OIG issued a report on the FBI’s
administrative and investigative mainframe systems. This report
identified continued vulnerabilities with management, operational, and
technical controls. Significant vulnerabilities were noted in the
following areas:
13
The report, “Handling of FBI Intelligence Information Related to the Justice
Department’s Campaign Finance Investigation,” was issued in July 1999.
14
The ACS is the FBI’s primary investigative computer application that
uploads and stores case files electronically.
- 5 -
pg_0023
security policies, procedures, standards, and guidelines;
physical controls;
system and network backup and restoration controls;
password management;
logon management;
account integrity management;
system auditing management; and
system patches.
The report stated that these vulnerabilities occurred because the
DOJ and FBI security management had not enforced compliance with
existing security policies, developed a complete set of policies to
effectively secure the administrative and investigative mainframes, or
held FBI personnel responsible for timely correction of recurring
findings. Further, the report indicated that FBI management has been
slow to correct identified weaknesses and implement corrective action.
Therefore, many of these deficiencies repeat year after year in
subsequent audits.
In March 2002, the Commission for the Review of FBI Security
Programs issued a report titled, “A Review of FBI Security Programs.”
This Commission, chaired by former FBI Director William H. Webster,
was established to investigate the espionage of a FBI Supervisory
Special Agent, Robert Hanssen.
15
The report identified a wide range of
problems affecting the FBI’s computer systems and information
security policies, including the following:
• Classified information had been moved into systems not
properly accredited for its protection.
15
According to the report, over a period of 22 years, Robert Hanssen gave
the Soviet Union and Russia vast quantities of documents and computer diskettes
filled with national security information of incalculable value.
- 6 -
pg_0024
Until recently, the FBI had not begun to certify and accredit most
of its computer systems, including many classified systems.
Inadequate physical protections placed electronically stored
information at risk of compromise.
The FBI’s approach to system design has been deficient. It has
failed to ascertain the security requirements of the “owners” of
information on its systems and identify the threats and
vulnerabilities that must be countered.
Classified information stored on some of the FBI’s most widely
utilized systems was not adequately protected because computer
users lacked sufficient guidance about critical security features.
Some FBI inspectors had insufficient resources to perform
required audits. When audits were performed, audit logs
were reviewed sporadically, if at all.
According to the report, these findings resulted from the FBI’s lack of
attention to IT security in developing and managing computer
systems.
16
Additionally, the General Accounting Office (GAO) has issued
several reports and related testimony that highlight deficiencies with
the FBI’s IT. In June 2002, the Comptroller General provided the
following testimony before a subcommittee of the United States House
of Representatives Appropriations Committee:
Communications has been a longstanding problem for the
FBI. This problem has included antiquated computer
hardware and software, including the lack of a fully
functional e-mail system. These deficiencies serve to
significantly hamper the FBI’s ability to share important
and time sensitive information with the rest of the FBI
across other intelligence and law enforcement agencies.
We [the GAO] do not believe the FBI will be able to
successfully change its mission and effectively transform
itself without significantly upgrading its communications
16
Although the focus of our audit does not assess the FBI’s IT security
practices, the two prior reports mentioned above indicate that the FBI’s effective use
of IT must address information assurance as part of an overall IT governance model.
- 7 -
pg_0025
and information technology capabilities. This is critical,
and it will take time and money to successfully address.
17
In a review of the DOJ’s Campaign Finance Task Force, the GAO
reported in May 2002 that the FBI lacked an adequate information
system that could manage and interrelate the evidence that had been
gathered in relation to the Task Force’s investigations.
18
Also, as part
of a government-wide assessment of federal agencies, the GAO
reported in February
2002 that the FBI needed to fully establish the
management foundation that is necessary to
successfully develop,
implement, and maintain an enterprise architecture.
19
The deficiencies in IT management are not solely attributable to
the FBI itself, but are also attributable in part to DOJ actions. In
December 2000, the GAO issued a report on the Immigration and
Naturalization Service’s (INS) investment management capability.
20
This report stated that the DOJ was not guiding and overseeing the
INS’s IT investment management (ITIM) approach. The report
highlighted the DOJ’s responsibility, as required by the Clinger-Cohen
Act of 1996, to ensure that its components implement an effective
ITIM process. According to the report, the DOJ had not provided the
INS, or any other component, sufficient direction, guidance, and
oversight of ITIM activities. Further, the report stated:
While Justice [the Department of Justice] issued guidance
in January 2000 describing its high-level investment
management process, the guidance does not address the
need or requirements for Justice’s components to
implement an IT investment management process.
Specifically, this guidance does not instruct the
components to establish IT investment management
processes nor does it establish expectations for doing so.
Until Justice issues its policy and guidance and begins
monitoring its components’ progress, it has no assurance
17
This testimony, titled “FBI REORGANIZATION: Initial Steps Encouraging
but Broad Transformation Needed” (GAO-02-865T), was released on June 21, 2002.
18
This report, titled “CAMPAIGN FINANCE TASK FORCE: Problems and
Disagreements Initially Hampered Justice’s Investigation” (GAO/GGD-00-101BR),
was released on May 31, 2000.
19
This GAO report is discussed later in this report.
20
“INFORMATON TECHNOLOGY: INS Needs to Strengthen Its Investment
Management Capability” (GAO-01-146) was issued by the GAO in December 2000.
- 8 -
pg_0026
that it has the necessary investment management
processes in place to maximize the value of its IT
investments and manage the risks associated with the
investments.
The DOJ issued ITIM guidance in August 2001 and required the
components to develop an ITIM process by January 2002. This
guidance, and the FBI’s ITIM process, are further discussed later in
this introduction.
4. The FBI’s Current IT Investment Efforts
In a statement before the House Subcommittee on
Appropriations in March 2002, FBI Director Mueller stated: “Without
question, we all believe [information infrastructure] is the number one
problem confronting the FBI today, recognize that for a number of
reasons the situation developed over time, and know that in the future
a better approach to technology upgrades must be used.”
In the FBI Information Technology Upgrade Plan (FITUP),
prepared and submitted to Congress in September 2000, the Bureau
stated that a lack of funding was the cause for not making meaningful
upgrades to its IT infrastructure since 1994. Congress responded to
this concern by appropriating a total of approximately $2.2 billion for
FBI IT projects and systems for FYs 1997 to 2002.
21
The FBI received
$335.6 million of this amount in January 2002 from the Emergency
Supplemental Appropriations Act for information technology. The
following table summarizes the funds appropriated for FBI IT
investments since FY 1997.
21
This appropriation includes operation and maintenance costs of existing IT
systems, enhancements to existing IT systems, and funding for new IT projects. The
appropriation also includes personnel costs for managing the IT projects and
systems.
- 9 -
pg_0027
Funds Appropriated for FBI IT Investments Since FY 1997
Fiscal Year
Total IT Investments
(in millions)
2002
$714.0
2001
$352.8
2000
$293.0
1999
$332.0
1998
$241.2
1997
$309.2
Total
$2,242.2
Source: Exhibit 53s
22
prepared by the FBI
The FBI has several critical initiatives underway to upgrade its
infrastructure and investigation applications. Additionally, the FBI has
undertaken a major hiring initiative to recruit private sector IT experts
who can assist in designing and managing the sizable IT projects
recently funded by Congress. For example, the FBI’s last two Chief
Information Officers were hired from the private sector. Also, in
March 2002, the FBI announced the hiring of a project executive from
the private sector to manage Trilogy. Further, in June 2002, the FBI
announced the hiring of an executive from the private sector to
become the new Executive Assistant Director for Administration.
5. Trilogy: The FBI’s Largest IT Investment
Currently, the FBI’s largest IT project designed to improve IT
infrastructure and office automation is the Trilogy project, formerly
known as the FITUP. In September 2000, the FITUP was established
to enhance the investigative support for FBI agents. The FITUP noted
the following IT needs:
22
The Exhibit 53 for each fiscal year lists funds appropriated for major IT
projects. The FBI prepares the Exhibit 53 and submits it to the DOJ, which submits
it to the Office of Management and Budget (OMB). Total IT investments include
operation and maintenance costs of existing IT systems, enhancements to existing IT
systems, and funding for new IT projects. These investment costs also include
personnel costs associated with managing IT projects and systems.
- 10 -
pg_0028
getting all case files into electronic databases (since the ACS is
not consistently used);
making IT more user friendly for agents;
providing access to all databases via one search engine; and
providing reliable, high-speed flexible communications.
To address the above needs, the FITUP, renamed to Trilogy, is
intended to upgrade the FBI’s: (1) hardware and software – referred
to as the Information Presentation Component (IPC),
(2) communication networks – referred to as the Transportation
Network Component (TNC), and (3) five most important investigative
applications – referred to as the User Applications Component (UAC).
The IPC and TNC upgrades will provide the physical infrastructure
needed to run the applications from the UAC portion of Trilogy. The
UAC portion is intended to upgrade and consolidate five of the FBI’s
42 investigative applications. Because there are 37 other investigative
applications and approximately 160 non-investigative applications that
Trilogy will not address, Trilogy is only a starting point towards
upgrading the FBI’s entire IT infrastructure.
In November 2000, Congress appropriated $100.7 million for the
first year of the $379.8 million Trilogy project, which was to be funded
over a three-year period (from the date contractors were hired). The
$100.7 million was a combination of new program funding and a
re-direction of base resources. The FBI combined the IPC and TNC
portions for continuity when it requested contractor support, since
both encompass physical IT infrastructure enhancements. The
contractor for the IPC/TNC portions was hired in May 2001. As a
result, the originally scheduled completion date for these initiatives
was May 2004. A separate contractor was hired in June 2001 to
complete the UAC portion of Trilogy by June 2004.
After the terrorist attacks on September 11, 2001, the
importance of giving FBI agents and analysts the technological tools
necessary to perform their duties was heightened in the eyes of
Congress, the Attorney General, and the Director. Because the goal of
Trilogy is to address many of the technological needs of the FBI,
successful completion of the project in the shortest amount of time
possible was viewed as increasingly critical to the FBI’s fight against
terrorism. Rather than wait three years for the benefits of Trilogy,
Congress fully funded the FBI’s original request of $379.8 million and
- 11 -
pg_0029
provided an additional $78 million in January 2002 to speed up its
deployment.
23
With the supplemental funding, the FBI indicated to
Congress that it would complete the deployment of hardware
(including new desktop computers), networks, and software by
July 2002. Additionally, the FBI would seek to accelerate upgrades to
the five user applications. However, as discussed later in this report,
the FBI did not meet its July 2002 milestone and is not expecting to
complete the deployment of hardware, software, and networks until
March 2003.
Although we believe the FBI must have sufficient resources to
upgrade its technology through Trilogy and other projects, it must also
have the management processes in place to effectively utilize those
resources. With the recent influx of funding to the FBI, Congress
expects the FBI to make significant strides in upgrading its IT
infrastructure. But we believe the FBI will be successful in doing so
only if it has effective IT management control processes in place.
Later in this report, we provide an assessment of the FBI’s
management of Trilogy.
6. Framework for Assessing IT Investment Management
Several recent management reforms have required federal
agencies to improve their management processes for selecting and
managing IT investments. In particular, the Clinger-Cohen Act of
1996 requires the head of each agency to implement a process for
maximizing the value of the agency's IT investments and for assessing
and managing the risks of its acquisitions. A key goal of the
Clinger-Cohen Act is for agencies to have processes in place to ensure
that IT projects are being implemented at acceptable costs and within
reasonable time frames, and that the projects are contributing to
tangible, observable improvements in mission performance.
The Clinger-Cohen Act defines requirements for capital planning
and control of IT investments and mandates a select/control/evaluate
approach that federal agencies must follow. The following graphic
describes the fundamental phases of this IT investment approach.
23
The $78 million was part of the $745 million received from the Emergency
Supplemental Appropriations Act.
- 12 -
pg_0030
Fundamental Phases of the IT Investment Approach
Select
Phase
Screen
• Rank
• Select
Evaluate
Phase
Conduct
reviews
• Make adjustments
• Apply lessons
learned
How are you
ensuring
that projects
deliver
benefits?
?
?
How do you know
you have selected
the best projects?
?
Are the systems
delivering what
you expected?
Control
Phase
Monitor
progress
• Take
corrective
actions
DATA
Source: GAO
According to a GAO report, while almost all federal agencies
have created some type of IT investment management process, none
has implemented stable processes that address all three phases of the
select/control/evaluate approach.
24
One barrier to implementing
stable IT investment processes has been the lack of specific guidance
regarding what processes are required to build a stable, reliable IT
investment management organization. The select/control/evaluate
approach provides sound advice, but it does not provide a
comprehensive discussion of the organizational processes involved.
To address this concern, in May 2000 the GAO developed the
IT Investment Management Framework (Framework) to provide a
common methodology for discussing and assessing IT capital planning
and investment management practices at federal agencies. The
Framework enhances previous federal IT investment management
guidance by embedding the select/control/evaluate approach within a
framework that explicitly describes the organizational processes
required to carry out good IT investment management.
- 13 -
24
“Information Technology Investment Management: An Overview of GAO’s
Assessment Framework” (GAO/AIMD-00-155) was issued in May 2000.
pg_0031
The Framework, based on best practices of leading
organizations, is a hierarchical model comprising of five maturity
stages. These maturity stages represent steps toward achieving stable
and mature investment management processes. Each stage builds
upon the lower stages and enhances the organization's ability to
manage its investments. As agencies advance through these stages,
the agencies’ capability to effectively manage IT increases. The
following graphic describes the five maturity stages of the Framework.
The Five Maturity Stages of the ITIM Framework
Source: GAO
There is little awareness of investment
management techniques. IT management
processes are ad hoc, project-centric, and
have widely variable outcomes.
Repeatable investment control techniques are in
place and the key foundation capabilities have
been implemented.
Comprehensive IT investment portfolio selection
and control techniques are in place that
incorporate benefit and risk criteria linked to
mission goals and strategies.
Description
Investment benchmarking and IT-enabled
change management techniques are deployed
to strategically shape business outcomes.
Process evaluation techniques focus on
improving the performance and management
of the organization's IT investment portfolio.
Enterprise
and Strategic
Focus
Project-
Centric
Stage 4
Improving the
Investment Process
Stage 3
Developing a Complete
Investment Portfolio
Stage 2
Building the
Investment Foundation
Stage 1
Creating Investment
Awareness
Stage 5
Leveraging IT for
Strategic Outcomes
Maturity Stages
With the exception of the first stage, each maturity stage is
composed of critical processes that must be implemented and
institutionalized for the organization to satisfy the requirements of that
stage. These critical processes are further broken down into key
practices that describe the types of activities that an agency should be
engaged in to successfully implement each critical process. An
organization that has these critical processes in place is in a better
position to successfully invest in IT. The following graphic describes
the Framework’s five stages and associated critical processes.
- 14 -
pg_0032
The ITIM Framework’s Stages of Maturity with Critical
Processes
Investment Process Benchmarking
IT-Driven Strategic Business Change
Post-Implementation Reviews and Feedback
Portfolio Performance Evaluation and Im provement
Systems and Technology Succession Management
Authority Alignm ent of IT Inve stme nt Boards
P ortfolio S ele ction Criteria De finition
Investment Analysis
Portfolio Development
Portfolio Performance Oversight
IT Investm ent Board Operation
IT Asset Tracking
IT P roject Ove rsight
Business Needs Identification for IT Projects
Proposal Selection
IT Spending without Disciplined Investment
Processes
Stage 4
Improving the
Investment
Process
Stage 3
Developing
a Complete
Investm ent P ortfolio
Stage 2
Building the
Investment
Foundation
Stage 1
Creating
Investment
Awareness
Stage 5
Leveraging IT
for S trategic
O utcomes
Maturity Stages
Critical
Processes
Source: GAO
As established by the Framework, each critical process contains
five core elements that indicate whether the implementation and
institutionalization of a process can be effective and repeated. The
five core elements are:
• Purpose: This element is the primary reason for engaging in
the critical process and states the desired outcome for the
critical process.
Organizational commitment: This element comprises
management actions that ensure that the critical process is
established and will endure. Key practices typically involve
establishing organizational policies and engaging senior
management sponsorship.
Prerequisites: These elements are the conditions that must
exist within an organization to successfully implement a critical
process. These conditions typically involve allocating resources,
establishing organizational structures, and providing training.
- 15 -
pg_0033
Activities: These elements are the key practices necessary to
implement a critical process. An activity occurs over time and
has recognizable results. Key practices typically involve
establishing procedures, performing and tracking the work, and
taking corrective actions as necessary.
Evidence of performance: This element comprises artifacts,
documents, or other evidence that supports a contention that
the key practices within a critical process have been or are being
implemented. This core element typically consists of the
collection and verification of physical, documentary, or
testimonial evidence and often involves reviews by objective
parties.
With the exception of the “purpose” core element, each of the
other core elements contains key practices. The key practices are the
attributes and activities that contribute most to the effective
implementation and institutionalization of a critical process. The
following graphic summarizes the interrelationships of components in
an ITIM critical process.
- 16 -
pg_0034
Components of an ITIM Critical Process
Purpose
This is the primary reason for engaging in the critical process
and states the desired outcome for the critical process.
Prerequisites
These are the conditions that must
exist within an organization to
successfully implement a critical
process. This core element
typically involves allocating
resources, establishing
organizational structures, and
providing training.
Activities
These are the key practices
necessary to i mplement a critical
process. An activity occurs over time
and has recognizable results. Key
practices within this core element
typically involve establishing
procedures, performing and tracking
the work, and taking corrective
actions as necessary.
Evidence of
Performance
These are artifacts, documents, or
other evidence that support a
contention that the key practices
within a critical process have or are
being implemented. This core
element typically consists of the
collection and verification of
physical, documentary, or
testimonial evidence and typically
involves reviews by objective
parties.
Organizational Commitment
These are management actions that ensure that the critical
process is established and will endure. Key practices within
this core element typically involve establishing
organizational policies and engaging senior management
sponsorship.
Source: GAO
7. The DOJ’s ITIM Guidance
In August 2001, the DOJ’s Justice Management Division (JMD)
issued the Guide to the Department of Justice Information Technology
Investment Management Process (Guide). In response to various
regulations and guidelines issued in the last several years (including
the Clinger-Cohen Act, Executive Order 13011, and the
Office of Management and Budget (OMB) Circular A-130), the DOJ
issued the Guide to fulfill its obligation and responsibility to make
measurable improvements in mission performance and service delivery
to the public through the strategic application of IT.
The Guide uses the select/control/evaluate methodology to
implement the strategic and performance directives of the
Clinger-Cohen Act and other statutory provisions affecting IT
investments. The Guide is intended to promote a process that builds
on existing structures to provide maximum benefit across the entire
DOJ and with other federal agencies. This process allows the DOJ to
focus IT management on the strategic missions of the DOJ. Further, it
- 17 -
pg_0035
promotes an investment review process that drives budget formulation
and execution for information systems, and restructures the way the
DOJ performs its functions before investing in IT. In addition, this
process provides the methods, structures, disciplines, and
management framework that govern the way IT is deployed
throughout the DOJ. The Guide applies to all IT projects from all DOJ
components.
The Guide requires each component to:
designate a component Chief Information Officer consistent with
the DOJ’s ITIM policy;
establish an Executive Review Board that will approve the entire
component IT portfolio and oversee the decisions made about
specific investments; and
establish a component ITIM process that incorporates the DOJ’s
ITIM process, but is customized to function within the
component’s unique environment.
Further, by January 2002 each component was required to
submit to the DOJ an ITIM plan incorporating the above stipulations.
8. The FBI’s Recent Efforts to Implement an ITIM Process
In an effort to improve its IT investment management practices
and comply with DOJ and other statutory regulations, the FBI
developed the “ITIM Model and Transition Plan” (Plan) with support
from a contractor. The initial draft of the Plan was completed and
submitted to JMD in January 2002. The FBI has retained this
contractor to assist in the ongoing implementation of the ITIM process.
The FBI estimates total costs for developing its ITIM process will be in
excess of $4 million through FY 2003.
The purpose of the Plan is to establish and define the FBI’s
Stage Two
25
methodology and build the foundation for enhanced IT
investment management. It identifies the gaps between the FBI’s
current IT investment processes and the required IT management
practices for Stage Two maturity.
25
“Stage Two” refers to Stage Two of the Framework, Building the IT
Investment Foundation.
- 18 -
pg_0036
The following excerpts from the FBI’s Plan provide an overview
of how the FBI’s select, control, and evaluate processes for IT
investment management are intended to operate upon
implementation.
26
Select
In the Select phase, potential projects will be initiated by
the project sponsor via the development of a preliminary
feasibility analysis (concept paper), followed by the
development of a more-robust business case analyses
(OMB Exhibit 300). The project proposal package will be
submitted to the Technical Review Board
27
to be assessed
for any technical risks and then submitted to the Project
Oversight Committee
28
for a business review. The Project
Oversight Committee will assemble the multiple requests
and prioritize these requests against predefined selection
criteria. A “candidate” fiscal project portfolio will then be
developed and presented to the Executive Review Board
29
for final evaluation and approval, and ultimately for
submission to the fiscal budget process.
Control
In the Control phase, the current fiscal year IT portfolio
will be tracked by the functional project management
office and individual project teams. Monthly status reports
will be created and presented to the Project Oversight
Committee, who will work to mitigate any project related
risks. Projects with exceptions to the baseline plans will be
subsequently presented to the Executive Review Board for
26
See Appendices 2 and 3, respectively, for flowcharts on the Plan’s control
and evaluate processes.
27
According to the Plan, the Technical Review Board must be established to
review each proposed ITIM initiative for enterprise architecture compliance, IT
security compliance, and other technical risks.
28
According to the Plan, the Project Oversight Committee must be established
to perform the program management and oversight duties of the ITIM process, such
as making recommendations to the Executive Review Board on selecting IT proposals
and disposing of IT projects.
29
According to the Plan, the Executive Review Board must be established to
make the final IT investment decisions.
- 19 -
pg_0037
decisions about budget, scope, timeline and/or projected
outcomes. During the control phase, a project will be able
to receive approval to: proceed “as is,” proceed with
modified funding levels and/or modified functionality, or be
terminated.
Evaluate
In the Evaluate phase, IT investments that are in the
operations and maintenance mode will be monitored by
the Executive Review Board to ensure that expected
benefits are being realized. Periodic program reviews will
be conducted, wherein each IT investment will be
evaluated against predefined performance metrics and
criteria. Based on the reviews, decisions will be made
about: future phases of existing projects; and the current
policies and procedures governing the entire IT investment
management, the systems development life-cycle, and
other related processes. Advocacy arguments (to modify
existing management practices and procedures) are also
constructed during this phase, if applicable.
JMD officially approved the FBI’s Plan in May 2002, although
officials from the IRD told us that in February 2002 they received
verbal approval to initiate their ITIM process.
30
The May 2002
approval letter states that the FBI ITIM process conforms to the
guidelines defined by the GAO, OMB, and DOJ. Further, it states that
the Plan is clear and comprehensive in its statement of the ITIM policy
and its definition of organizational roles, responsibilities, and
deliverables. Additional JMD comments, as well as our own
independent assessment of the Plan, are discussed later in this report.
The FBI started its ITIM process in February 2002 by appointing
the three oversight review boards discussed above (the Technical
Review Board, the Project Oversight Committee, and the Executive
Review Board). Also, in February 2002 the FBI held training seminars
for each division to introduce the concepts of the Plan. In March 2002,
the FBI began pilot testing the select phase of the Plan for FY 2004
proposed IT project enhancements. In May 2002, the pilot test of the
30
JMD officials told us that the delay in providing written approval of the FBI’s
ITIM process was because JMD did not have a Chief Information Officer early in
2002.
- 20 -
pg_0038
select phase was completed and the ITIM contractor issued the, “Post
Implementation Review: FBI ITIM Pilot.”
The Plan recognizes that as the FBI’s ITIM process moves
through the maturity stages, other key components of IT
infrastructure must evolve to optimize the IT investment function.
These components include an IT strategic plan, an enterprise
architecture framework, and project management. According to the
Framework, an effective IT function will include these components and
mature IT investment management processes are dependent on the
components being in place.
- 21 -
pg_0039
OIG
FINDINGS AND RECOMMENDATIONS
1. The FBI’s Management of IT Investments
The FBI is not effectively selecting, controlling, and
evaluating its IT investments because it has not fully
implemented any of the critical processes necessary for
successful IT investment management. In the past, the
FBI has not given sufficient attention to information
technology investment management. As a result, the FBI
continues to spend hundreds of millions of dollars on IT
projects without having adequate selection and project
management controls in place to ensure that IT projects
will meet intended goals. However, since the FBI
developed its ITIM Model and Transition Plan in
January 2002, it has focused more management attention
in this area and has made progress towards attaining a
basic IT investment management foundation. Much of the
progress has been in the “select” phase of the Plan, which
was pilot tested in the Spring of 2002.
The ability of the FBI to completely implement the
“control” and “evaluate” phases of the Plan, and achieve
mature IT investment processes that can lead to enhanced
mission performance, will require the FBI to increase its
efforts in: (1) fully developing and documenting its new
ITIM process; (2) requiring more input and participation
from ITIM managers and users; and (3) further developing
its project management and enterprise architecture
functions. While the FBI recognizes many of these needs
and has taken initial steps to address the needs, further
action in these areas is needed to ensure that IT projects
are developed within cost and schedule requirements, and
meet performance expectations. The Trilogy project
provides an example of how the non-implementation of
fundamental IT investment management practices can put
a project at risk of not delivering, within cost and schedule
requirements, what was promised.
A. The FBI’s Progress Toward Attaining a Basic IT
Investment Management Foundation
Although the FBI made measurable progress in improving its IT
investment capability since it initiated a new ITIM process in early
- 22 -
pg_0040
2002, the FBI still lacks a complete foundation to build its IT
investment maturity processes, and therefore is still in Stage One
maturity.
31
In the past, the FBI has not given sufficient management
attention to IT investments. Because of the lack of management
attention in the past, the FBI failed to implement the critical processes
necessary to build an IT investment foundation. These critical
processes include: (1) IT investment review board operation, (2) IT
project oversight, (3) IT system and project identification and tracking,
(4) business needs identification for IT projects, and (5) IT proposal
selection.
(1) Importance of Attaining a Basic IT Investment
Management Foundation
The primary purpose for attaining a basic IT investment
management capability (Stage Two maturity) is to build the foundation
for repeatable, successful IT project-level investment control and
selection processes. Effective control processes over IT projects
ensure that deviations from cost and schedule baselines can be
identified and corrected. Selection processes ensure that the FBI has
an effective methodology for approving only IT projects that are
consistent with its needs and goals. According to the Framework, an
organization can only achieve Stage Two maturity if it fully implements
the following five critical processes:
1. defining investment review board operations,
2. developing a basic process for selecting new IT
proposals,
3. developing project-level investment control processes,
4. identifying IT projects and systems, and
5. identifying the business needs for each IT project.
To implement these critical processes, the FBI must execute a
total of 38 key practices as defined in the Framework, or have
alternative practices in place that are designed to achieve the same
outcome.
31
Stage One maturity is the lowest level of maturity designated by the GAO
ITIM Framework. According to the Framework, an organization is in Stage One
maturity when it has not fully implemented the five critical processes associated with
Stage Two maturity.
- 23 -
pg_0041
At the start of our audit in January 2002, FBI officials told us
that the Bureau was in the process of developing its new ITIM process.
Although its ITIM process was still in the development stages, FBI
officials told us that the FBI was executing certain key practices from
Stage Two of the Framework. Additionally, the FBI officials said in
March 2002 that they would pilot test ITIM processes pertaining to the
selection of new IT proposals for the FY 2004 budget cycle. Further,
the Plan establishes the FBI’s goal to fully attain Stage Two maturity
for the FY 2005 budget cycle that starts in March of 2003, thereby
establishing the foundation for enhanced investment capability.
(2) Summary of the FBI’s Progress Toward Attaining Stage
Two Maturity
Based on the FBI’s responses to the self-assessment
32
(and our
validation of those responses), the FBI did not yet have in place any of
the five critical processes associated with Stage Two maturity.
However, since the FBI began pilot testing the select phase of its Plan
in March 2002, it has made progress towards implementing the 38 key
practices comprising the five critical processes - particularly in the area
of selecting new proposals for IT projects. Specifically, at the
beginning of our audit in January 2002, the FBI was only executing
4 of the 38 required key practices; however, as of June 2002, the FBI
was executing 14 of the key practices. The following table provides a
summary of the FBI’s progress toward implementing the key practices
required for each critical process.
32
To facilitate our assessment of the FBI’s IT investment maturity, the FBI
completed a self-assessment regarding the key practices from the Framework that it
was executing, or planning to execute, upon implementation of its new ITIM process.
- 24 -
pg_0042
FBI Progress Toward Attaining Stage Two Maturity
Critical
Process
Status of
Implementing
Critical
Process
Total Key
Practices
Required
Key
Practices
Executed
Prior to
March
2002
Key
Practices
Executed
as of
June
2002
1. IT Investment
Board
Operation
Not Implemented
6
0
2
2. IT Project
Oversight Not Implemented
11
1
2
3. IT Project
Identification
Not Implemented
7
1
2
4. Business
Needs
Identification
for IT Projects
Not Implemented
8
2
3
5. Proposal
Selection
Not Yet
Implemented,
but Substantial
Progress Made
6
0
5
Total
38
4
14
Source: OIG analyses
For the remainder of section A of this finding, we provide
detailed narratives of the FBI’s progress toward implementing each of
the five critical processes. We also provide specific recommendations
for expediting implementation of the critical processes and establishing
more timely Stage Two maturity.
Each critical process contains core elements that provide the
common framework for the process. For example, the organizational
commitment element addresses the management actions that ensure
the critical process is established and will endure; the prerequisites
element addresses the conditions that must exist within an
organization to successfully implement a critical process; and the
activities element consists of the key practices necessary to implement
a critical process. The key practices are the tasks within a core
- 25 -
pg_0043
element that must be performed by an organization to effectively
implement and institutionalize a critical process.
(3) Critical Process 1: IT Investment Review Board Operation
Depending on its size, structure, and culture, an organization
may have more than one IT investment review board. The purpose of
such boards is to ensure that basic policies for selecting, controlling,
and evaluating IT investments are developed, institutionalized, and
consistently followed throughout the organization. To establish a fully
functioning investment review board, the FBI must execute the
following six key practices:
1. create an IT investment process guide containing policies
and procedures to direct board operations;
2. require executives and line managers to support and
carry out board decisions;
3. allocate adequate resources for operating each board;
4. define board membership, policies and procedures, roles and
responsibilities;
5. create and define board membership to integrate both IT and
business knowledge; and
6. require the IT investment boards to follow the written
policies and procedures as defined in the process guide.
The following table summarizes the FBI’s progress toward
implementing fully functioning investment review boards.
- 26 -
pg_0044
FBI Progress Toward Implementing Fully Functioning
Investment Review Boards (Critical Process 1)
Key Practice
Key Practice
Execution
Status Prior to
March 2002
Key Practice
Execution
Status as of
June 2002
Organizational Commitment 1. An
organization-specific IT investment
process guide is created to direct each
board’s operations.
Not Executed Executed
Organizational Commitment 2.
Organization executives and line
managers support and carry out IT
investment board decisions.
Not Executed Not Executed
Prerequisite 1. Adequate resources are
provided for operating each IT
investment board.
Not Executed Not Executed
Prerequisite 2. Board members
understand the investment board’s
policies and procedures and exhibit core
competencies in using the IT investment
approach via training, education, or
experience.
Not Executed Not Executed
Activity 1. Each IT investment board is
created and defined with board
membership integrating both IT and
business knowledge.
Not Executed Executed
Activity 2. Each IT investment board
operates according to written policies and
procedures in the organization-specific
IT investment process guide.
Not Executed Not Executed
Source: OIG analyses
a. The FBI Has Executed Two of the Six Key Practices
Associated with IT Investment Board Operation
We determined that the FBI executed two of the six key
practices associated with implementing this critical process.
Specifically, the FBI created an IT investment process guide containing
policies and procedures to direct board operations (Organizational
Commitment 1), and it created and defined three investment review
boards integrating both IT and business knowledge (Activity 1).
- 27 -
pg_0045
Regarding the IT investment process guide (Organizational
Commitment 1), in January 2002 the FBI issued its IT Investment
Model and Transition Plan
33
containing required guide elements
prescribed by the Framework including:
specifics about the roles of key people within the FBI investment
process;
an outline of the significant events and decision points within the
processes;
an identification of the external and environmental factors that
will influence the processes; and
the manner in which IT investment-related processes will be
coordinated with other organizational plans and processes.
Regarding the investment review boards (Activity 1), in
June 2002 the Director approved board charters for each of the three
investment review boards (the Executive Review Board, the
Project Oversight Committee, and the Technical Review Board) that
defined board membership and the responsibilities of board members.
The Executive Review Board is comprised of the FBI Director (as
Chairperson), the Chief Information Officer, the FBI’s four
Executive Assistant Directors (EADs),
34
a Special Agent in
Charge committee member, the Assistant Director of the Finance
Division, and the Strategic Planning Manager.
This Board’s primary responsibility will be to evaluate and
approve projects in the candidate fiscal project portfolios and
forward approved projects to the fiscal budget process. This
Board will also determine whether problematic projects should
proceed “as is,” proceed with modified funding levels and/or
modified functionality, or be terminated.
The Project Oversight Committee includes: the Chief
Information Officer (as Chairperson), the Assistant Director from
33
The Plan was issued in draft form because it is the intent of the FBI to
modify and supplement the Plan as the ITIM process is being pilot tested.
34
The EADs are for: (1) Criminal Investigations, (2) Counterterrorism and
Counterintelligence, (3) Law Enforcement Services, and (4) Administration.
- 28 -
pg_0046
each division, a member from the Office of General Counsel, the
Chief Contracting Officer, and the Strategic Planning Manager.
Once the Technical Review Board completes its assessment, the
Project Review Board then performs a business review of the
proposed projects, prioritizes these proposals against predefined
selection criteria, and develops a “candidate” fiscal project
portfolio for presentation to the Executive Review Board. The
committee also reviews monthly status reports for ongoing
projects to mitigate project related risks. Projects with
exceptions to baseline plans will be presented to the Executive
Review Board for corrective action.
The Technical Review Board is comprised of: the Section Chief,
Information Resources Management Office (as Chairperson); the
Assistant Director of IRD; the IRD’s section chiefs; and
representatives from the Laboratory Division, CJIS Division, and
Security Division. This board’s primary responsibility will be to
assess technical risks for proposed projects.
The boards actually began functioning as early as March 2002, in
conjunction with the FBI’s pilot testing of ITIM processes pertaining to
the selection of new IT proposals for the FY 2004 budget cycle.
Although board membership consists mostly of FBI managers who do
not have extensive IT knowledge,
35
the use of subject matter experts
and reliance on the Enterprise Architecture Technical Committee
36
can
compensate for a lack of IT knowledge.
b. The FBI Must Execute Four of the Six Key Practices
Associated with IT Investment Board Operation
Although progress has been made, the FBI does not have fully
functioning IT investment boards because it still must execute four of
the six key practices associated with this critical process. Specifically,
the FBI must ensure that:
35
Based on our interviews with FBI managers from the IRD, CJIS, and
Inspection Divisions, most of the members on the investment boards are former agents
with no specialized expertise, training, or competencies in IT.
36
The Enterprise Architecture Technical Committee was created to provide
technical expertise to the Technical Review Board. Members of this committee are
comprised of IT specialists familiar with enterprise architecture, configuration
management, and quality assurance.
- 29 -
pg_0047
organization executives and line managers support and carry out
IT investment board decisions (Organizational Commitment 2);
adequate resources are provided for operating each IT
investment board (Prerequisite 1);
board members understand the investment board’s policies and
procedures and exhibit core competencies in using the IT
investment approach via training, education, or experience
(Prerequisite 2); and
each IT investment board operates according to written policies
and procedures contained in the investment process guide
(Activity 2).
Regarding Organizational Commitment 2 and Activity 2, the
approved charters for the investment review boards have been in
effect since June 2002. Consequently, the FBI did not have sufficient
data for us to assess whether managers and support staff effectively
carried out board decisions and whether the boards operated according
to the written policies and procedures contained in the Plan and board
charters.
Regarding Prerequisites 1 and 2, in our judgment the FBI did not
adequately plan sufficient time to ensure the IT investment boards
operated effectively. Specifically, the FBI did not provide ample time
between the initial draft of its Plan (January 25, 2002) and the
March 2002 pilot testing of the select phase to adequately prepare and
train IT board members. The DOJ originally instructed each
component to begin developing an ITIM process in January 2001.
37
In
June 2001, the DOJ required each component to complete and submit
to JMD an ITIM process and transition plan by the end of 2001.
38
The
DOJ also required each component to initiate the ITIM process for the
FY 2004 budget cycle, which for the FBI began in March 2002.
Consequently, the FBI had only one full month between the issuance
of the Plan in late January 2002 and the initiation of the select phase
of its ITIM process in early March 2002.
37
This instruction originated from DOJ Order 2880.1A, policy on Information
Technology Investment Management, issued in January 2001.
38
This instruction originated from a DOJ memorandum dated
June 28, 2001. This memorandum required each component to have an ITIM
transition plan that will allow implementation for the FY 2004 budget cycle.
- 30 -
pg_0048
The ITIM Program Office Manager told us that the former FBI
Chief Financial Officer would not approve the use of a contractor to
assist in the development of the ITIM process earlier in the year.
According to the former Chief Financial Officer, she had concerns that
federal contracting regulations prohibited the FBI from using a
contractor to perform a service that involves budget planning.
However, following her transfer to another division in December 2001,
the Information Resources Management Section received authorization
to hire a contractor to assist with the development and implementation
of the ITIM process.
We believe that without an ITIM contractor the FBI still had the
opportunity to begin planning its ITIM process (including the training
of board members) early in 2001. In fact, had the FBI better
coordinated other ongoing efforts to develop processes that
complement IT investment management, the FBI could have made
significant strides in initiating its ITIM process during 2001 without
expending additional resources. As discussed in section B of this
finding, the FBI did not sufficiently incorporate (a) its enterprise
architecture function (which was under development in 2001) and
(b) the Project Management Process (issued in draft form in
October 2001) into the development of its ITIM process. Enterprise
architecture and project management not only complement the ITIM
process, but also facilitate the maturation of ITIM. As discussed in
section B of this finding, the FBI did not effectively utilize its internal
resources when it developed its ITIM process through the use of a
contractor because the FBI did not adequately consider the
complementary, and potentially duplicative efforts that were already
underway.
Not providing ample time resulted in inadequate training of
board members and minimal preparation time to develop IT proposals.
For example, Technical Review Board members had only 3 business
days to review over 50 IT proposals prior to their first board meeting.
FBI officials recognized these implementation issues in the Post-
Implementation Review of the select phase pilot test.
In preparing board members for their duties, the FBI has thus
far only provided one overview training session for board members
and other users in the ITIM process. Additionally, while FBI officials
have told us more ITIM training will be forthcoming, they have not
provided us with any specific training plans for the future. Further,
members of the Technical Review Board told us that board members,
especially the Assistant Directors and EADs, do not have extensive
- 31 -
pg_0049
knowledge in managing IT and must rely heavily on knowledgeable
staff and other subject matter experts.
For the ITIM process to become institutionalized, the FBI must
have a better training program. According to the Framework, board
members should understand the board’s policies, roles, rules, and
activities and be capable of carrying out their responsibilities
competently. Education and training for members is needed in areas
such as economic evaluation techniques, capital budgeting methods,
and performance measurement strategies.
The FBI’s Post-
Implementation Review of the select phase pilot testing recommends
“role-specific” training sessions for the following ITIM roles: (1) ITIM
Liaison representatives,
39
(2) Executive Review Board members,
(3) Program Oversight Review Board members, (4) Technical Review
Board members, and (5) ITIM stakeholders. It further recommends
continuation of the overview training sessions previously provided,
plus training for ITIM specific tools, such as the concept paper
(containing the preliminary feasibility analysis), the OMB Exhibit 300
(containing the business case analyses), and IT proposal summaries.
FBI officials told us that time constraints were the main cause for
not executing the four key practices identified above. As a result,
there was insufficient time to introduce ITIM concepts to board
members and other ITIM users. As mentioned above, the DOJ
required each component to develop and begin implementation of an
ITIM process for the FY 2004 budget cycle, which for the FBI begins in
March 2002. Although FBI officials were aware of the requirement to
initiate and adopt an ITIM process in January 2001, it was not until
December 2001 that it began to develop its ITIM process. Had the FBI
initiated more timely action to develop its ITIM process, it would have
had significantly more time to prepare and train ITIM board members
and other users. Without sufficient training and allocation of time to
perform required tasks, the investment review boards cannot carry out
their responsibilities to effectively select, control and evaluate projects.
39
The FBI’s ITIM process defines the ITIM Liaison Representative as an
individual from a particular division/business unit that facilitates workflow
and communications between that division/business unit and the ITIM
program office.
- 32 -
pg_0050
c. Recommendations
We recommend that the Director of the FBI:
1. Require the ITIM Program Office to plan for and take more timely
action to allow board members and other ITIM users to execute
assigned responsibilities competently (Prerequisite 1).
2. Ensure that all members of IT investment boards receive sufficient
education and training to execute assigned responsibilities
effectively. We suggest that for each of the investment boards the
FBI: (a) identify the core competencies required of members in
using the IT investment approach, and (b) develop appropriate
education and training development plans to ensure members
acquire the required core competencies (Prerequisite 2).
(4) Critical Process 2: IT Project Oversight
The purpose of this critical process is to ensure that the FBI’s
investment review boards and project development teams provide
effective oversight for its IT projects throughout all phases of the
project life-cycle. IT investment boards generally review each
project’s progress toward predicted cost and schedule expectations as
well as anticipated benefits and risk exposure. The board members
also employ early warning systems that enable them to take corrective
actions at the first signs of cost, schedule, and performance slippages.
Individual project development teams are responsible for meeting
project milestones within the expected cost and schedule parameters.
Effective project oversight requires, among other things:
having written policies and procedures for project management;
developing and maintaining an approved project management
plan for each project;
having written policies and procedures for oversight of IT
projects;
making up-to-date cost and schedule data for projects available
to the investment review boards;
- 33 -
pg_0051
reviewing each project’s performance by comparing actual cost
and schedule data to expectations regularly; and
ensuring that corrective actions for each under-performing
project are defined, implemented, and tracked until the desired
outcome is achieved.
We concluded that the FBI is not effectively overseeing its
ongoing IT projects. While the FBI maintained project management
guidance and had three IT investment review boards in operation since
March 2002, these activities have not adequately supported the FBI’s
IT project oversight function. Our testing of the key practices
associated with this critical process indicates that the FBI is executing
only two out of the eleven key practices required to implement this
critical process. The following table summarizes FBI progress toward
implementing IT project oversight.
- 34 -
pg_0052
FBI Progress Toward Implementing IT Project Oversight
(Critical Process 2)
Key Practice
Key Practice
Execution
Status Prior to
March 2002
Key Practice
Execution
Status as of
June 2002
Organizational Commitment 1. The
organization has written policies and
procedures for project management. Executed
Executed
Organizational Commitment 2. The
organization has written policies and
procedures for management oversight
of IT projects.
Not Executed Not Executed
Prerequisite 1. Adequate resources
are provided to assist the boards in
overseeing IT projects.
Not Executed Not Executed
Prerequisite 2. Each IT project has
and maintains an approved project
management plan that includes cost
and schedule controls.
Not Executed Not Executed
Prerequisite 3. An IT investment
review board is operating.
Not Executed Executed
Prerequisite 4. Information from the
IT asset inventory is used by the IT
investment board as applicable.
Not Executed Not Executed
Activity 1. Each project's up-to-date
cost and schedule data are provided to
the appropriate IT investment board. Not Executed Not Executed
Activity 2. Using established criteria,
the IT investment board oversees each
IT project's performance regularly by
comparing actual cost and schedule
data to expectations.
Not Executed Not Executed
Activity 3. The IT investment board
performs special reviews of projects
that have not met predetermined
performance standards.
Not Executed Not Executed
Activity 4. Appropriate corrective
actions for each under-performing
project are defined, documented, and
agreed to by the IT investment board
and the project manager.
Not Executed Not Executed
Activity 5. Corrective actions are
implemented and tracked until the
desired outcome is achieved.
Not Executed Not Executed
Source: OIG analyses
- 35 -
pg_0053
a. The FBI Has Executed Two of the Eleven Key Practices
Associated with IT Project Oversight
While the FBI has project management guidance (and is
therefore executing the key practice relating to the existence of project
management methodology), the guidance is not being followed on a
consistent basis. In fact, depending on whom we talked to, we
obtained different answers as to which document represented the FBI’s
official project management guidance.
For example, although IRD managers were aware that the DOJ’s
System Development Life-Cycle is the FBI’s official project
management methodology, they acknowledged that it is not
consistently applied. Laboratory Division management officials told us
that they do not follow the DOJ’s System Development Life-Cycle
methodology, but rather have adopted their own project management
system based on one used at the Department of Defense because it
better meets their needs. CJIS Division management officials told us
that although its Contract Administration Office is responsible for
project management functions, they were not following any specific
project methodology.
Other FBI personnel from the Information Resources
Management Section told us the Project Management Process,
developed by the FBI’s Inspection Division, was the FBI’s project
management guidance. However, Inspection Division personnel
indicated to us that the Project Management Process was still pending
approval from the Director, as of June 2002. As a result, there
appeared to be confusion among FBI officials as to what the official
project management guidance was. As of June 2002, the Project
Management Process had not been approved, nor was it being used to
manage IT projects.
As previously discussed in the prior report section pertaining to
the investment review board critical process, the FBI established three
IT investment review boards in March 2002 (the Executive Review
Board, the Project Oversight Committee, and the Technical Review
Board). Although the investment review boards are operating, the
boards have not yet been involved in project oversight. As the ITIM
process continues to evolve, project oversight by these boards should
increase accordingly.
- 36 -
pg_0054
b. The FBI Must Execute Nine of the Eleven Key Practices
Associated with IT Project Oversight
Based on our analyses, the FBI does not have effective IT
project oversight because it has not yet executed nine out of the
eleven key practices associated with this critical process. Specifically,
the FBI must ensure that:
written policies and procedures are developed for management
oversight of IT projects (Organizational Commitment 2);
adequate resources are provided to assist the investment boards
in overseeing IT projects (Prerequisite 1);
an approved project management plan is prepared for each IT
project that includes cost and schedule controls (Prerequisite 2);
information from the IT asset inventory is used by the IT
investment boards as applicable (Prerequisite 4);
each project's up-to-date cost and schedule data are provided to
the
appropriate IT investment board (Activity 1);
using established criteria, the IT investment boards oversee each
IT project's performance regularly by comparing actual cost and
schedule data to expectations (Activity 2);
the IT investment boards perform special reviews of projects
that have not met predetermined performance standards
(Activity 3);
appropriate corrective actions for each under-performing project
are defined, documented, and agreed to by the IT investment
boards and the project manager (Activity 4); and
corrective actions are implemented and tracked until the desired
outcome is achieved (Activity 5).
Regarding Organizational Commitment 2, the FBI has not
developed written policies and procedures for management oversight
of IT projects. While the Plan provides a conceptual basis for board
oversight of IT projects and the board charters define the boards’
responsibilities, the FBI does not have the specific policies and
procedures in place for overseeing and controlling projects. FBI
- 37 -
pg_0055
officials have acknowledged to us that the Plan was never intended to
represent the complete and final policies and procedures for
management oversight of IT projects. The Plan states that it is a fluid
document that will need to be modified and supplemented as the pilot
test is performed. As a result, FBI officials recognize that additional
policies and procedures must be developed. As of June 2002, FBI
officials have told us they are in the process of developing these
specific policies and procedures for the control phase of the ITIM pilot
test.
Regarding Prerequisite 1 (providing adequate resources to the
boards), we concluded that this key practice has not been executed
because as of June 2002, the FBI did not have a functioning project
management office to assist the boards in overseeing IT projects. The
Plan calls for a functioning project management office to assist the
boards, especially the Project Oversight Committee, and consequently
is a necessary resource for IT project oversight. As of June 2002, the
FBI has not yet utilized its project management function to assist the
Project Oversight Committee in IT investment decision-making.
The functioning project management office represents a critical
resource to the Project Oversight Committee and thus to IT project
oversight. In our judgment, the functioning project management
office needs to have jurisdiction over IT projects throughout the
Bureau, rather than limit its responsibilities to division-specific
projects. Until June 2002, the FBI lacked a functioning project
management office that had jurisdiction over IT projects throughout
the Bureau. Rather than having a centralized project management
office, independent of individual divisions, the FBI maintained three
separate division-level project management offices to manage IT
projects. These three separate project management functions were
maintained in the IRD, CJIS, and Laboratory Divisions, contributing to
inefficiencies in project coordination and the risk of “stove piping”
projects. Because of its importance in supporting the ITIM process,
the subject of establishing and maintaining a centralized project
management office is further discussed later in this report.
Regarding Prerequisite 2, we determined that each IT project
does not have an approved project management plan that includes
cost and schedule controls. Personnel from the IRD project
management office told us that generally IT projects with high visibility
have project management plans that include cost and schedule
controls. However, other lower visibility projects have less rigid
controls in place. This condition developed because the IRD project
- 38 -
pg_0056
management office did not uniformly enforce the development of
project management plans by all IT project managers. In our
judgment, projects under the IRD’s discretion have not been
adequately controlled. Although personnel from the CJIS and
Laboratory Divisions indicated that IT projects under their respective
divisions did have management plans with cost and schedule controls,
without a functioning board that approves and monitors these project
management plans FBI managers have no assurance that IT projects
are effectively managed in accordance with uniform standards.
Regarding Prerequisite 4, the FBI has not yet developed an IT
asset inventory; consequently, the FBI’s investment review boards are
not aware of all the IT projects and resources for which the boards are
responsible. FBI managers told us they were in the process of
developing an IT asset inventory. However, at the time of our audit
they were unable to provide an estimated date for completing the
inventory. Unless the investment review board members are fully
cognizant of the IT projects and resources for which they are
responsible, the boards cannot exercise effective oversight of ongoing
IT projects. Additional details pertaining to the FBI’s plans to finalize
the IT inventory are provided later in this report.
Finally, since the IT investment review boards were not involved
in overseeing IT projects as of June 2002, we concluded that none of
the five remaining key practices activities have been executed. These
five key practices are the basic activities that investment review
boards must implement to effectively oversee IT projects during the
control phase. The FBI provided us documentation indicating that the
Project Oversight Committee (the primary IT investment review board
responsible for overseeing IT projects) met in June 2002 to discuss the
FBI’s intent to pilot test the control phase of the Plan by September
2002. The documentation stated that the FBI was still working on
designing the specific procedures associated with the control phase,
including integrating the ITIM process with the project management
office. Additionally, the FBI has only provided us with summary
information on when and how the control phase of the ITIM process
will be rolled out. The information lacks specific details needed to
effectively implement this critical process.
FBI personnel told us that the lack of established IT investment
review boards (prior to March 2002) was the main cause for ineffective
project oversight. Additionally, they stated that the control phase of
the ITIM process would be pilot tested by September 2002. However,
the FBI has not been able to provide us with a specific timeline as to:
- 39 -
pg_0057
(1) how the pilot test will be executed, and (2) details as to how the
ITIM process will interface with a project management methodology.
These issues are further discussed in Section B of this finding.
Without effective oversight of IT projects, FBI officials do not
have adequate assurance that IT projects are being developed on
schedule and within established budgets. As described in the following
paragraphs, the lack of effective IT project oversight has contributed
to the FBI’s problems in managing IT projects, including a lack of
accountability for cost and schedule overruns, a lack of consideration
for full life-cycle costs, and lost credibility with Congress.
According to a former Chief Information Officer at the FBI, the
lack of effective oversight of IT projects (as a result of not having IT
investment review boards and a centralized project management
office) have prevented IT project managers from being held
accountable for cost and schedule overruns and the ultimate
performance of projects. For example, the former Chief Information
Officer told us that the CJIS Division completed the Integrated
Automated Fingerprint Identification System and the National Crime
Information Center 2000 years behind schedule and millions of dollars
over budget. He also told us that management changes in the
CJIS Division have not occurred, despite these overruns.
Senior FBI officials also told us that the Bureau’s budget
formulation process focuses only on the acquisition costs for IT
projects and not the full life-cycle costs, especially operations and
maintenance costs. For example, an assessment performed by the
FBI’s Inspection Division on the Trilogy project
40
noted that the life-
cycle cost estimate is inadequate and only focuses on the term of the
contract, not the life of the project. FBI personnel told us that a lack
of consideration for full project costs is not limited to Trilogy, but also
applies to other IT projects. Without accountability for significant
deviations from project baselines, there is a lack of incentives for
project managers to adequately control and evaluate projects.
According to FBI officials, the FBI’s inability to effectively
complete IT projects within budget and schedule reduced the FBI’s
credibility in the eyes of Congress. The lack of credibility contributed
to delays in the FBI receiving Congressional funding to upgrade its IT
infrastructure. This subject, along with how Trilogy may be adversely
affected because of uncertainties in determining projected costs and
40
The Trilogy project is discussed in greater detail in section C of this finding.
- 40 -
pg_0058
scheduled completion dates for project milestones, is further discussed
in section C of this finding.
c. Recommendations
We recommend that the Director of the FBI ensure:
3. Official project management guidance is consistently followed by all
FBI IT project managers.
4. Written policies and procedures are developed for management
oversight of IT projects for use by the investment review boards
(Organizational Commitment 2).
5. IT Investment Review Boards are supported by a centralized
project management office that operates in accordance with ITIM
policies and procedures (Prerequisite 1).
6. Each IT project has a project management plan, approved by the
Project Oversight Committee, that includes cost and schedule
controls (Prerequisite 2).
7. Information being developed in the IT asset inventory is made
available to, and used by, the boards (Prerequisite 4).
8. Execution of the five key practices consisting of the activities
necessary for the investment review boards to maintain effective
oversight of IT projects during the critical control phase. These
five key practices consist of:
Providing each project's up-to-date cost and schedule data to the
appropriate IT investment board (Activity 1).
Establishing criteria for the boards to review each IT project’s
performance by comparing actual cost and schedule data to
expectations (Activity 2).
Performing special reviews of projects that have not met
predetermined performance standards (Activity 3).
Defining, documenting, and agreeing to corrective actions for
each under-performing project by the appropriate IT investment
board and the project manager (Activity 4).
- 41 -
pg_0059
Tracking and implementing corrective actions until the desired
outcome is achieved (Activity 5).
(5) Critical Process 3: IT Project and System Identification
For the FBI to make effective IT investment decisions, it must
have at its disposal information about existing IT investments as well
as the proposed investments being considered. The purpose of this
critical process is to provide the IT investment boards the information
required to fully evaluate the impacts and opportunities created by
both the proposed and current IT investments. The key practices of
this process require the FBI to identify and track the IT projects and
systems within the organization to create a comprehensive inventory.
According to the Framework, effective identification of IT projects and
systems requires:
identifying specific information about each IT project and system
in an inventory, according to written procedures;
updating information in the inventory as changes to projects and
systems occur;
making information from the inventory available to users as
needed; and
assigning responsibility for managing the IT system identification
process.
While the FBI has taken steps to identify its IT projects and
systems in an IT asset inventory, it still does not have a complete IT
asset inventory that is being using by the IT investment review boards
for investment management purposes. As part of an enterprise
architecture data repository, the FBI is developing a comprehensive
inventory of its IT projects and systems. In addition, FBI officials have
told us that the enterprise architecture office is primarily responsible
for developing and maintaining the data repository. However, the data
repository has not been completed, nor have board members used its
contents during the select phase of the ITIM process that took place
during the Spring of 2002. The FBI’s enterprise architecture function
is further discussed in section B of this finding. The following table
summarizes the key practice ratings for the IT project and system
identification critical process.
- 42 -
pg_0060
FBI Progress Toward Identifying IT Projects and Systems
(Critical Process 3)
Key Practice
Key Practice
Execution
Status Prior to
March 2002
Key Practice
Execution
Status as of
June 2002
Organizational Commitment 1.
The organization has written policies
and procedures for identifying its IT
projects and systems and collecting an
inventory that includes information
about the IT projects and systems
that is relevant to the investment
management process.
Executed
Executed
Organizational Commitment 2.
An official is assigned responsibility
for managing the IT project and
system identification process and
ensuring the inventory meets the
needs of the investment management
process.
Not Executed Executed
Prerequisite 1. Adequate resources
are provided for identifying IT projects
and systems and collecting relevant
information into an inventory.
Not Executed Not Executed
Activity 1. The organization's IT
projects and systems are identified
and specific information about these
projects is collected in an inventory. Not Executed Not Executed
Activity 2. Changes to IT projects
and systems are identified and
changed information is collected in the
inventory.
Not Executed Not Executed
Activity 3. Information from the
inventory is available on demand to
decision-makers and other affected
parties.
Not Executed Not Executed
Activity 4. The IT project and system
inventory and its information records
are maintained to contribute to future
investment selections and
assessments.
Not Executed Not Executed
Source: OIG analyses
- 43 -
pg_0061
a. The FBI has Executed Two of the Seven Key Practices
Associated With Identifying IT Projects and Systems
Based on our analyses, we determined that the FBI has executed
two of the seven key practices associated with this critical process.
Specifically, the FBI has developed written policies and procedures for
identifying its IT projects and systems in an inventory that includes
information relevant to the investment management process
(Organizational Commitment 1). Additionally, the FBI has designated
an official responsible for managing the IT project and system
identification process and ensuring that the inventory meets the needs
of the investment management process (Organizational
Commitment 2).
Regarding Organizational Commitment 1, we determined that
the FBI has developed adequate written policies and procedures for:
(a) identifying its IT projects and systems and (b) collecting
information relevant to the investment management process on each
project and system. Prior to December 2001, the FBI did not have
written policies and procedures for identifying IT projects and systems.
The FBI did, however, provide us with an electronic communication
dated December 3, 2001 from the enterprise architecture staff that
was distributed Bureau-wide requesting management from each
division to provide information on its IT systems. The information
obtained from the divisions is used by the enterprise architecture staff
to develop the data repository of IT systems.
Regarding Organizational Commitment 2, the FBI has designated
the Chief Architect of the enterprise architecture office with
responsibility for managing the IT project and system identification
process and ensuring that the inventory, when completed, meets the
needs of the investment management process and ITIM managers and
users. The Chief Architect currently reports to the Information
Resource Management Section Chief, who reports to the Chief
Information Officer.
b. The FBI Must Execute Five of the Seven Key Practices
Associated with Identifying IT Projects and Systems
Although the FBI has made recent progress in identifying IT
projects and systems, the FBI does not have a comprehensive IT
project and system identification process because it still has not
executed five out of the seven key practices associated with this
critical process. Specifically, the FBI must ensure that:
- 44 -
pg_0062
adequate resources are provided for identifying IT projects and
systems and collecting relevant information into an inventory
(Prerequisite 1);
the organization's IT projects and systems are identified and
specific information about these projects and systems is
collected in an inventory (Activity 1);
changes to IT projects and systems are identified and changed
information is collected in the inventory (Activity 2);
information from the inventory is available on demand to
decision-makers and other affected parties (Activity 3); and
the IT project and system inventory and its information records
are maintained to contribute to future investment selections and
assessments (Activity 4).
Regarding Prerequisite 1, FBI managers told us that the FBI has
not allocated adequate resources to ensure timely and successful
completion of the IT project and system identification critical process.
FBI managers from the Information Resources Management Section
told us that they do not have sufficient staffing to support the ITIM
process, including the enterprise architecture function. The enterprise
architecture office within the Information Resources Management
Section plays a key role in the ITIM process as it assists the Technical
Review Board and maintains the data repository information on IT
systems and projects. Further, personnel who we interviewed from
the enterprise architecture office told us that limited staffing was a
factor in not having the data repository completed.
41
Regarding the remaining four key practices, none of those
practices can be executed until the FBI completes the creation of its IT
asset inventory. More importantly, the IT asset inventory will have
little value to the FBI if it is not used when making IT investment
decisions. Prior attempts at compiling an inventory of IT projects were
used to satisfy Congressional and DOJ requests, rather than to assist
the IT investment management process. For example, the FBI
41
Our judgments regarding staffing issues within the enterprise architecture
office are discussed in more detail later in this report.
- 45 -
pg_0063
prepared a partial list of its information technology projects to comply
with a Congressional request in August 2000.
FBI officials informed us that they anticipate the investment
review boards will use the completed inventories to contribute to
future investment selections and assessments. The Plan states that
the FBI must establish a complete IT portfolio set as the ITIM process
matures. Further, FBI personnel told us that the enterprise
architecture data repository, when complete, will be available to
decision-makers and other ITIM users via the FBI’s Intranet.
However, we have not been provided with a specific timeframe for
when the FBI expects to have a completed inventory.
FBI personnel told us that the primary cause of not having a
completed IT asset inventory and actively using it in the ITIM process
is because of staffing shortages. While that may be a contributing
factor, we concluded that the lack of centralized management over IT
investments was also a limiting factor. As a result, certain divisions
maintained some version of an IT inventory for the projects and
systems under their jurisdiction, and there was no centralized office
responsible for maintaining a uniform listing Bureau-wide.
Without a complete IT asset inventory in the ITIM process, FBI
management and board members do not have adequate assurance
that accurate, timely, and complete information on existing IT projects
and systems is available to them. As a result, there is a risk that new
IT proposals selected overlap with one of the 200 or so existing FBI
applications. While the recently established review boards helped to
mitigate this risk for the FY 2004 budget selection process, we believe
that an IT asset inventory must be used by the boards to optimize the
use of the FBI’s resources.
c. Recommendations
We recommend that the Director of the FBI:
9. Establish a deadline for completing the creation of the FBI IT
inventory and ensure progress toward completion is monitored
(Activity 1).
- 46 -
pg_0064
10. Implement processes to ensure:
a. subsequent changes to IT projects and systems are identified
and documented in the inventory (Activity 2);
b. information from the inventory is available on demand to
decision-makers and other affected parties (Activity 3); and
c. the IT project and system inventory and its information
records are maintained to contribute to future investment
selections and assessments (Activity 4).
(6) Critical Process 4: Business Needs Identification
This critical process establishes the mechanism for identifying
the business needs and the associated users that drive each IT
project. This critical process links the organization’s business
objectives with its IT strategy and creates the partnership between the
users and the IT providers. According to the Framework, effective
identification of business needs requires:
defining the organization’s business needs and goals;
identifying users who will participate throughout the life-cycle of
each project;
defining business needs for each IT project; and
training IT staff in business needs identification.
While the FBI has made progress in identifying business needs
for IT projects, it has not yet executed all the key practices necessary
to implement this critical process. Prior to pilot testing the select
phase of its ITIM process in March 2002, the FBI had been identifying
users for each IT project in the Exhibit 300.
42
Since pilot testing the
select phase of the ITIM process beginning in March 2002, the FBI has
used a concept paper along with the Exhibit 300 to identify and define
business needs. In addition, the FBI has defined its general business
needs and goals in its strategic plan, which is further discussed later in
this report. However, as previously mentioned, the FBI has not
42
An Exhibit 300 is a capital asset plan that must be prepared for major
projects and is submitted to the DOJ and OMB.
- 47 -
pg_0065
identified all of its IT projects in an asset inventory; consequently,
progress in implementing this critical process is contingent upon
completing the FBI IT inventory. Also, we were not provided evidence
indicating that identified users participate in project management
throughout a project's life-cycle. The following table summarizes the
key practice ratings for the business needs identification critical
process.
FBI Progress Toward Identifying its Business Needs
(Critical Process 4)
Key Practice
Key Practice
Execution
Status Prior to
March 2002
Key Practice
Execution
Status as of
June 2002
Organizational Commitment 1. The
organization has written policies and
procedures for identifying the business needs
(and the associated users) of each IT project. Not Executed Not Executed
Prerequisite 1. Adequate resources are
provided for identifying business needs and
associated users.
Not Executed Not Executed
Prerequisite 2. The organization has defined
business needs or stated mission goals.
Executed
Executed
Prerequisite 3. IT staff are trained in
business needs identification.
Not Executed Not Executed
Prerequisite 4. All IT projects are identified
in the IT asset inventory.
Not Executed Not Executed
Activity 1. The business needs for each IT
project are clearly identified and defined. Not Executed Executed
Activity 2. Specific users are identified for
each IT project.
Executed
Executed
Activity 3. Identified users participate in
project management throughout a project's
life-cycle.
Not Executed Not Executed
Source: OIG analyses
a. The FBI has Executed Three of the Eight Key Practices
Required to Identify its Business Needs and Associated
Users
We determined that the FBI has executed three of the eight key
practices associated with this critical process. Specifically, the FBI has
defined its business needs or stated mission goals (Prerequisite 2); the
business needs for identified IT projects are clearly identified and
- 48 -
pg_0066
defined (Activity 1); and specific users are identified for each IT
project (Activity 2).
Regarding Prerequisite 2, we determined that the FBI has
defined business needs or stated mission goals. The FBI has stated
mission goals in its strategic plan. The FBI’s strategic plan has not
been updated since 1998, but the Director has revised the priorities of
the Bureau since the terrorist attacks on September 11, 2001.
Further, the FBI is currently in the process of developing an enterprise
architecture framework, which will link the FBI’s strategic plan to its
business needs.
Regarding Activity 1, we determined that the business needs for
each IT project are clearly identified and defined in the Exhibit 300.
Prior to the initiation of the ITIM pilot test in March 2002, the FBI did
not have adequate management controls in place to ensure that the
business needs for each project were accurately developed in the
Exhibit 300. With the ITIM process, the board reviews of the concept
papers and Exhibit 300s provided assurance that these business needs
were clearly identified and defined. In instances where the business
needs were vague, the boards, especially the Technical Review Board,
returned the concept papers and Exhibit 300s to the project sponsor
for re-work. This re-work demonstrates that board review of these IT
proposals was an effective control over the business needs
identification process. Our review of Exhibit 300s that were ultimately
recommended to the Executive Review Board for inclusion in the
FY 2004 budget cycle confirmed that business needs were clearly
identified and defined.
Regarding Activity 2, the FBI identified specific users for each IT
project. Based on our reviews of several Exhibit 300s both before and
after the initiation of the ITIM process in March 2002, we determined
that the users for the IT project were identified and documented.
b. The FBI Must Execute Five of the Eight Key Practices
Required to Identify its business Needs and Associated
Users
Although progress has been made in identifying its business
needs and associated users, the FBI has yet to execute five of the
eight key practices associated with this critical process. Specifically,
the FBI must ensure that:
- 49 -
pg_0067
it has formalized written policies and procedures for identifying
the business needs (and the associated users) of each IT project
(Organizational Commitment 1);
adequate resources are provided for identifying business needs
and associated users (Prerequisite 1);
IT staff are trained in business needs identification
(Prerequisite 3);
all IT projects are identified in the IT asset inventory
(Prerequisite 4); and
identified users participate in project management throughout
the project life-cycle (Activity 3).
Regarding Organizational Commitment 1, we determined that
the FBI does not have written policies and procedures for identifying
the business needs (and the associated users) of each IT project. The
FBI has been defining business needs for IT projects in the
Exhibits 300 and related concept papers. The Post-Implementation
Review acknowledges that the FBI needs more formally developed
policies and procedures to support the ITIM process. By formalizing
these procedures in writing, the FBI reduces the risk that it will neglect
to perform this practice in the future.
Regarding Prerequisites 1 and 3, FBI officials told us that
adequate resources were not allocated to identifying business needs
and associated users. Specifically, FBI officials from the Information
Resources Management Section told us that there has not been
sufficient resources dedicated to the ITIM process, including the
training of ITIM users. The importance of training ITIM users in the
many facets of the ITIM process cannot be underestimated. Part of
the required ITIM training must include the business needs
identification process. Examples of training in this critical process
include organizational requirements for ongoing education, rotation of
ITIM users through supported business units, and relevant conference
attendance. As previously mentioned, many ITIM users have only
received one training session on the FBI’s ITIM process. Additionally,
the FBI has not provided us with specific plans for future training
sessions that include business needs identification. As a result, these
key practices have not been executed.
- 50 -
pg_0068
The ITIM training that occurred in February 2002 provided only
an overview of the ITIM process, rather than role-specific training that
addressed the business needs identification. The Post-Implementation
Review stated that re-work of Exhibit 300s and concept papers were
required after these products were submitted to the ITIM program
office. This re-work was necessary because there was not a clear
alignment between the IT proposal and the FBI’s strategic goals.
Better training that included business needs identification may have
reduced some of the re-work. Further, a more clearly defined
enterprise architecture framework would have increased the IT staff’s
knowledge in business needs identification.
Regarding Prerequisite 4, as previously mentioned, the FBI has
not completed its IT asset inventory. Identifying all projects in an IT
asset inventory is a fundamental step in having a fully developed
business needs identification process. The availability of this inventory
assists board members in recommending IT projects that support one
or more business needs or mission goals.
Regarding Activity 3, FBI officials have acknowledged that
identified users do not consistently participate throughout the project’s
life-cycle. FBI officials informed us that not keeping IT system users
actively involved in the creation and implementation of IT projects is a
major factor in the development of multiple IT systems (including
ACS) that do not effectively meet user needs. When we asked the
former Chief Information Officer for other examples of systems that do
not effectively meet user needs, his response was “pick one.” Clearly,
this is a significant need that must be addressed by the ITIM process.
The DOJ’s System Development Life-Cycle requires user participation
throughout the life-cycle, but as we previously noted in this finding,
the System Development Life-Cycle is not used by the FBI on a
consistent basis. Board oversight of project teams should be required
to ensure that users are engaged throughout the project’s life-cycle.
FBI officials told us that there has not been ample time since the
implementation of the Plan to adequately train its IT staff and board
members in business needs identification. A complete explanation as
to why the FBI did not have ample time for training was previously
discussed in section A.3 of this finding.
Although FBI officials have told us that additional training for IT
staff and board members is expected to occur sometime in the future,
we were not provided evidence that shows there will be any training
specifically related to business needs identification. Further, we have
- 51 -
pg_0069
not been provided with a timetable as to when this training will take
place. In addition, an effective business needs identification process
requires an organization to have a comprehensive IT portfolio and
enterprise architecture, neither of which the FBI currently has. Our
assessment of the FBI’s efforts to implement a basic enterprise
architecture is discussed later in this report.
Without a comprehensive business needs identification process,
FBI management and board members do not have adequate assurance
that they are selecting IT projects that align with mission needs and
priorities. Additionally, projects under development are at risk of not
meeting the needs of users, as has been the case with ACS and other
FBI systems.
c. Recommendations
We recommend that the Director of the FBI ensures:
11. Written policies and procedures are developed for identifying the
business needs (and the associated users) of each IT project
(Organizational Commitment 1).
12.
13.
Adequate resources are allocated to train ITIM users in identifying
business needs and associated users (Prerequisites 1 and 3).
Identified users participate in project management throughout a
project's life-cycle (Activity 3).
(7) Critical Process 5: IT Proposal Selection
The proposal selection critical process establishes a structured
methodology for selecting new IT proposals. The FBI should have this
critical process fully implemented to ensure that it selects the most
meritorious IT proposals to meet its mission critical needs. According
to the Framework, this critical process requires:
designating an official to manage the proposal selection process;
using a structured process to develop new proposals;
making funding decisions based on an established process; and
analyzing and ranking new IT proposals against criteria that
includes cost and schedule data.
- 52 -
pg_0070
The following table summarizes the key practice ratings for the
proposal selection critical process.
FBI Progress Toward Establishing an IT Proposal Selection
Process (Critical Process 5)
Key Practice
Key Practice
Execution
Status Prior to
March 2002
Key Practice
Execution
Status as of
June 2002
Organizational Commitment 1.
Executives and managers are committed
to follow an established selection
process.
Not Executed Executed
Organizational Commitment 2. An
official is designated to manage the
proposal selection process.
Not Executed Executed
Prerequisite 1. Adequate resources
are provided for proposal selection
activities.
Not Executed Not Executed
Activity 1. The organization uses a
structured process to develop new IT
proposals.
Not Executed Executed
Activity 2. Executives analyze and
prioritize new IT proposals according to
established selection criteria.
Not Executed Executed
Activity 3. Executives make funding
decisions for new IT proposals according
to an established process.
Not Executed Executed
Source: OIG analyses
a. The FBI Has Executed Five of the Six Key Practices
Associated With Establishing an IT Proposal Selection
Process
As previously discussed, the FBI pilot tested its ITIM proposal
process in March 2002. The Plan outlined a conceptual framework for
selecting projects, while subsequent documents further defined the
process. We determined that the FBI has executed five of the six key
practices associated with this critical process. The five key practice
are:
FBI managers are committed to follow an established selection
process (Organizational Commitment 1);
- 53 -
pg_0071
an official is designated to manage the proposal selection
process (Organizational Commitment 2);
the FBI uses a structured process to develop new IT proposals
(Activity 1);
FBI managers analyze and prioritize new IT proposals according
to established selection criteria (Activity 2); and
executives make funding decisions for new IT proposals
according to an established process (Activity 3).
Regarding Organizational Commitment 1 and Activity 1, we
concluded that in pilot testing its proposal selection process in
March 2002, FBI managers were committed to and followed an
established selection process for the FY 2004 budget cycle.
Prior to the initiation of the ITIM process in March 2002, the FBI
did not have an established process for selecting IT proposals. Several
FBI officials told us that individual divisions determined their IT needs
in a “stovepipe,” without knowledge of the business needs and
priorities of the Bureau as a whole. Once each division decided on its
IT request, the request was forwarded to the Information Resources
Management Section for a “technical” review. This review, performed
by the Information Resources Management Section Chief, was
designed to ensure that the request was consistent with the FBI’s
existing IT infrastructure. However, without an established enterprise
architecture, the review could not adequately provide assurance that
the proposal aligned with the FBI’s business needs and priorities.
Once approved by the Information Resources Management
Section Chief, the request was then forwarded to the Finance Division
to determine if similar requests for budget enhancements were
previously denied by Congress. Requests approved by the Finance
Division were forwarded to a committee comprised of executive
managers for final evaluation and selection. However, personnel from
the Finance Division told us that it was not uncommon for the IRD,
Laboratory, and CJIS Divisions to submit requests for IT projects that
were duplicative but were approved anyway. This indicates that the
Information Resources Management Section did not adequately
perform its role in overseeing IT proposals. Additionally, according to
FBI officials, the committee of executive managers did not have a
formalized charter, follow approved polices or procedures, or maintain
- 54 -
pg_0072
documentation detailing committee activities. Therefore, the process
was not standardized or repeatable.
With the initiation of the ITIM process in March 2002, the FBI
established a proposal selection process for the FY 2004 budget cycle.
IT proposals were developed by the project sponsor with a preliminary
feasibility analysis, referred to as a concept paper. The concept paper
was submitted to the Enterprise Architecture Technical Committee for
a preliminary technical review, and then forwarded to the
Technical Review Board with a recommendation as to whether the
project should be approved. Upon the Technical Review Board’s
approval, the project sponsor was asked to prepare a more
comprehensive business case analysis, which was documented in the
Exhibit 300. The project proposal package, which includes the concept
paper and Exhibit 300, was then submitted to the Project Oversight
Committee for a business review. The Project Oversight
Committee assembled the multiple requests and recommended a list
of projects for the Executive Review Board’s review. The
Executive Review Board selected projects for the FY 2004 budget
cycle. Because this process was documented in the Plan, and
enhanced with training materials, we concluded that the FBI effectively
established a selection process. The following flowchart outlines the
FBI's proposal selection process.
- 55 -
pg_0073
FLOWCHART OF FBI’S ITIM SELECT PHASE
Approve?
Wor th
next level of
investment?
Approve?
IDEA for
IT Initiative
Stop
Stop
No
No
No
Yes
Project ori ginator
develops a Concept
Paper
Div is ion
management r eviews
Concept Paper
Review for project,
technical , financi al ,
and securi ty ri sk
rati ng with the TRB
Review for busi ness
al ignment & vision
with the POC
Project ori ginator
devel ops Busi ness
Case (Exhibi t 300)
Di vi si on management
revi ews Busi ness
Case (Exhibi t 300)
Project ori ginator
modifi es Business
Case (Exhibi t 300)
Approve?
No
POC revi ew &
priori tizati on of all
Business Cases
Submit pri or itized l ist
to ERB for review &
approval
Submi t budget
enhancement
request
Stop
Yes
Project Sponsor fine
tunes Concept Paper
with suppor t fr om IT
and Fi nance Teams
Pa rt i a l
C o n ce p t
Pa p e r
C o mpl e t e d
Co n c e pt
Pa p er
C omp l et e d
Co n c ep t
P a pe r
C o nc e p t
Pa p e r
D a sh b o ar d
(p a r ti al )
C o mp l e t e d
C o n ce p t
Pa p e r
Co n ce p t
P ap e r
Da s hb o a r d
C om pl e t e d
Co n ce p t
P a pe r
St o p
Yes
TRB r eviews
Busi ness Case and
Updates CP
Dashboard
Re v i se d C P
D a sh b o a r d
Bu s i n e ss
C a se
( Exhi bi t 300)
Pr o j e ct
S u mma r i e s
P ro j e c t
Ra n ki n gs
Source: FBI’s training materials for the ITIM process as of
February 2002.
Regarding Organizational Commitment 2, prior to the initiation of
the select phase of its ITIM process in March 2002, the FBI did not
have a clearly designated official to manage the proposal selection
process. According to Information Resources Management Section
personnel, the Finance Division managed the IT selection process.
However, according to Finance Division personnel, the Information
Resources Management office was responsible for managing the
proposal selection process. With the onset of the ITIM process in
March 2002, the FBI’s Chief Information Officer appointed the ITIM
- 56 -
pg_0074
Program Manager to manage the proposal selection process. This
official reports to the Information Resources Management Section
Chief, who reports to the Chief Information Officer.
Regarding Activity 2, we determined that FBI IT investment
board members analyzed and prioritized new IT proposals according to
established selection criteria for the FY 2004 budget cycle. Projects
were prioritized according to three separate areas: (1) mission fit;
(2) technical criteria (including risk management and architectural
assessments); and (3) financial criteria (including performance
measures, cost/benefit analyses, and acquisition strategy).
Regarding Activity 3, the three IT investment review boards
made funding decisions for new IT proposals according to a process
established for the FY 2004 budget cycle. The Executive Review
Board, chaired by the Director, had the final authority for making IT
funding requests to the DOJ. The Executive Review Board members
based their decisions upon recommendations made by the Technical
Review Board and the Project Oversight Committee. Based on the use
of an established process, this key practice has been executed.
b. The FBI Must Execute One Key Practice Associated With
Establishing an IT Proposal Selection Process
Although the FBI has made substantial progress in establishing
an IT proposal selection process for the FY 2004 budget cycle, in our
judgment it has yet to allocate adequate resources for comprehensive
proposal selection activities. Our conclusion is based upon the
following observations.
The FBI pilot tested the selection process only for proposed
budget enhancements for FY 2004 and not for projects already
included in the base funding for IT.
43
As a result, the selection
process was not comprehensive because it did not include all
FY 2004 funding for IT.
Project sponsors had insufficient time to adequately document
proposals in the concept paper and Exhibit 300. According to
the FBI’s Post-Implementation Review of the pilot test, project
sponsors had as little as three days to develop concept papers
43
Funding for IT projects comes from both base funding and enhancements.
Base funding is usually the prior fiscal year’s budget allocation. Enhancements are
additions to the prior fiscal year’s base that are sought to fulfill certain priorities.
- 57 -
pg_0075
and Exhibit 300s used in the IT proposal selection process. FBI
officials told us that it can take over a month to adequately
prepare a comprehensive business case analysis (Exhibit 300).
As a result of the time constraints, the Post-Implementation
Review stated that concept papers, Exhibit 300s, and IT proposal
summaries were submitted with gaps and omissions in areas
such as: (1) aligning proposed activity with the FBI’s strategic
goals, (2) technical details, (3) acquisition and performance
management approaches, (4) resource requirements and
commitments, (5) expected levels of return-on-investment, and
(6) security.
According to the Post-Implementation Review of the pilot test,
the boards and project sponsors did not maximize the use of
subject matter experts to facilitate the proposal selection
process. Additionally, according to the Post-Implementation
Review, project owners did not adequately consult with internal
staff in various divisions when preparing their IT proposals.
Finally, the ITIM Program Manager, appointed in February 2002,
was not provided any staff to assist her (other than contractor
support). FBI officials stated to us in the self-assessment that
the insufficient staffing is the number one challenge to
implementing the ITIM process. Additionally, according to the
Post-Implementation Review, the ITIM Program Office did not
have sufficient staffing to sustain the ITIM process. Specifically,
the Post-Implementation Review recommends two additional
full-time employees to be added immediately, with an eventual
goal of having at least six full-time employees in the ITIM
Program Office. ITIM staffing is necessary to facilitate
communications between the boards, project owners, and
divisions. Clearly, adequate staffing for the ITIM Program Office
is essential to successfully implement the ITIM process.
Without a comprehensive proposal selection process that
includes adequate resources and training, the FBI cannot ensure that it
is selecting the best IT projects that meet mission-critical needs.
- 58 -
pg_0076
c. Recommendations
We recommend that the Director of the FBI ensures:
14. The ITIM process applies to all IT project proposals, including
proposals that are funded through the FBI’s base funding.
15. Sufficient staffing is provided to the ITIM Program Office, as
recommended in the Post-Implementation Review.
(8) Overriding Cause for the Lack of an FBI IT Investment
Management Foundation
Although the GAO ITIM Framework was originally published in
May 2000, the underlying key practices needed to implement each
critical process are, in essence, tasks that are fundamental to any
project management endeavor. Some of these tasks include the
prerequisite conditions that must be in place in an organization to
successfully implement critical processes. These tasks involve
allocating resources, establishing organizational structures, and
providing training. Another group of tasks include the organizational
commitments that ensure critical processes will endure. These tasks
involve establishing organizational policies and engaging senior
management sponsorship. A third group of tasks include the activities
necessary to implement the critical processes. These tasks involve
establishing procedures, performing and tracking the work, and taking
corrective actions as necessary.
Although these tasks are fundamental to effective project
management, the majority of these tasks had not been executed by
the FBI to select and manage its IT resources. Prior to the
development of its ITIM process in early 2002, the FBI did not give
sufficient attention to IT investment management. Organizational
policies were not clearly established to ensure that critical IT
investment policies endure. Additionally, there were no clearly
defined, uniform procedures for project management, tracking project
performance, and taking corrective actions as necessary.
Because the FBI did not fully implement any of the critical
processes associated with Stage Two, the FBI continues to spend
hundreds of millions of dollars on IT projects without having adequate
selection and project management controls in place to ensure that IT
projects will deliver their intended benefits. However, the FBI has
made progress in improving its IT investment process since it initiated
- 59 -
pg_0077
a new ITIM process in early in 2002. Although further action is
required, the launching of the ITIM process represents improvement in
the FBI’s ability to mitigate the risks that IT projects will not deliver
their intended benefits. Whether the FBI can achieve further
improvement depends on whether the Plan addresses the remaining
key practices not being executed as well as the FBI’s ability to
completely implement the Plan and fully establish its ITIM process.
B. The FBI’s Ability to Improve its IT Investment Practices
As previously noted, the FBI lacks a foundation necessary to
build its IT investment capabilities, and therefore, is in Stage One
maturity. However, in January 2002, the FBI developed an ITIM plan
to build a foundation for selecting, controlling, and evaluating IT
investments. Additionally, during the course of our audit fieldwork
(from January 2002 to June 2002), the FBI initiated its ITIM process,
as defined by the Plan. Consequently, the FBI made progress towards
implementing the Plan, especially in the area of IT proposal selection.
Because the FBI was only in the beginning stages of
implementing the Plan during our audit fieldwork, we assessed the
FBI’s ability to progress through the more advanced stages of the
framework necessary to improve its IT investment maturity. Our
assessment of the FBI’s ability to improve its IT investment
management consisted of the following four areas:
1. the Plan’s coverage of Stage Two key practice activities
that were not being executed during our fieldwork –
necessary to determine adequacy of the Plan;
2. the amount of participation from ITIM users in developing
the ITIM process – necessary to determine buy-in to the
process;
3. the support from the project management function –
necessary to execute the control and evaluate phases of
the ITIM process; and
4. the support from the enterprise architecture function –
necessary to advance through the maturity stages of the
Framework.
Our evaluation of these four areas, documented in the following
sections, includes both the FBI’s strengths and weaknesses in each
- 60 -
pg_0078
area. In our judgment, the FBI’s efforts in these areas are critical to
its ability to maximize the effectiveness of its ITIM process, and
ultimately improve mission performance.
(1) The Plan’s Coverage of Stage Two Key Practice Activities
That Were Not Being Executed During Our Fieldwork
The FBI’s IT Investment Management Model and Transition Plan
addresses the select, control, and evaluate key practice activities
necessary to build an IT investment foundation. However, the Plan
requires further development to ensure effective implementation.
Because the Plan was intended to be a conceptual framework, it was
not written to fully describe the specific policies and procedures of the
select, control, and evaluate phases of the ITIM process. Without
further development of the ITIM process, the FBI will have difficulty
making additional progress in improving its IT investment
management practices, especially in the control and evaluate phases.
a. Importance of the Plan’s Coverage of Stage Two Key
Practice Activities
Because the Plan stated that its purpose is to establish and
define the FBI’s Stage Two methodology necessary to build an IT
investment foundation, we examined the Plan’s coverage of Stage Two
key practice activities. The FBI was pilot testing the select phase of
the ITIM process during our audit fieldwork. As previously noted, we
determined that the FBI executed 14 of 38 Stage Two key practices,
mainly in the area of proposal selection. Of the 24 key practices that
were not executed, 11 specifically related to activities associated with
the control and evaluate phases of the ITIM process. Although the FBI
had made little progress in executing activities from the control and
evaluate phases of the Plan during our fieldwork, we examined the
Plan to determine whether it adequately addressed the 11 Stage Two
key practices activities associated with the control and evaluate phases
that were not being executed. The ability of the FBI to achieve Stage
Two maturity is dependent, in part, on the adequacy of the Plan.
In JMD’s assessment of the Plan, JMD rated the Plan against
elements it considered necessary to comply with GAO, OMB, and DOJ
guidelines. JMD’s assessment indicated that the Plan complied with
the criteria used.
44
Additionally, JMD’s assessment stated that
although the Plan does not fully address a few items, such as the exact
44
JMD’s assessment of the Plan is contained in Appendix 4 of this report.
- 61 -
pg_0079
criteria that will be used to select and evaluate investments, it does
provide a schedule for completing these items.
Our assessment of the Plan focused on whether it addressed the
Stage Two maturity key practices in the GAO ITIM Framework and our
conclusions are consistent with those from JMD.
b. Results of Our Assessment of the Plan’s Coverage of Stage
Two Key Practice Activities Associated with the Control and
Evaluate Phases
In our judgment,
the FBI’s IT Investment Management Model
and Transition Plan addresses the 11 Stage Two key practice activities,
on a conceptual level, that were not being executed during our
fieldwork. Because the key practice activities are addressed
conceptually, further development is needed to clearly define these
activities and to determine how these activities can be implemented.
Our analyses (previously documented in this report) indicated
that the FBI was not executing one or more key practice activities in
each of the following Stage Two critical processes: (1) IT investment
board operation; (2) IT project oversight; (3) IT project and system
identification; and (4) business needs identification. As previously
discussed, 11 of the key practice activities necessary to implement
these four critical processes relate to the control and evaluate phases
of the Plan. The tables below describe how the Plan addresses the key
practice activities that we determined were not being executed during
our audit testing.
- 62 -
pg_0080
IT Investment Board Critical Process
Key Practice Activity Not Executed How the Plan Addresses the Activity
Activity 2: Each IT investment board
operates according to written policies
and procedures in the organization-
specific IT investment process guide.
While the Plan does not provide the
specific written policies and procedures
that the investment boards must
follow, it does indicate that further
development of these policies and
procedures are necessary.
Additionally, the Post-Implementation
Review of the select phase of the ITIM
pilot test recommends that additional
policies and procedures be developed
in a document that is independent of
the Plan. Once the FBI’s ITIM policies
are completely developed, this key
practice can be executed when the FBI
rolls-out the control and evaluate
phases of the ITIM process.
Source: OIG analyses
- 63 -
pg_0081
IT Project Oversight Critical Process
Key Practice Activity Not Executed
How the Plan Addresses the Activity
Activity 1: Each project's up-to-date
cost and schedule data are provided to the
appropriate IT investment board.
The Plan stipulates that the
functioning project management
office will review status reports on
cost, schedule, and performance
measures. The project management
office will then forward selected
reports to the boards for review.
Activity 2: Using established criteria, the
IT investment board oversees each IT
project's performance regularly by
comparing actual cost and schedule data
to expectations.
The Plan states that the Project
Oversight Committee will ensure that
selected projects are meeting
performance measurement objectives,
risks are being appropriately
managed, budgets and schedules are
on track, and resource levels are
adequate.
Activity 3: The IT investment board
performs special reviews of projects that
have not met predetermined performance
standards.
According to the Plan, the Project
Oversight Committee will perform
special reviews of projects whose
status reports are not meeting
predetermined performance standards.
Activity 4: Appropriate corrective actions
for each under-performing project are
defined, documented, and agreed to by
the IT investment board and the project
manager.
The Plan states that the Project
Oversight Committee will review a
portfolio status report to determine if
quick corrective actions can be
executed to get under-performing
projects back on track. When this is
not possible, appropriate
recommendations will be made to the
Executive Review Board.
Activity 5: Corrective actions are
implemented and tracked until the
desired outcome is achieved.
The Plan gives the Project Oversight
Committee the responsibility to
ensure that corrective actions are
implemented.
Source: OIG analyses
- 64 -
pg_0082
IT Project and System Identification Critical Process
Key Practice Activity Not Executed
How the Plan Addresses the Activity
Activity 1: The organization's IT projects
and systems are identified and specific
information about these projects and
systems is collected in an inventory.
The Plan states that an IT investment
portfolio will be built for development
projects as the ITIM process is being
pilot tested. An IT portfolio is expected
to be completed for the full-blown ITIM
roll-out during the FY 2005 budget
cycle.
Activity 2: Changes to IT projects and
systems are identified and change
information is collected in the inventory.
FBI personnel told us that while there
is not a written procedure to
document changes to IT projects and
systems, a policy will be developed
when the IT asset inventory is
complete. The IT asset inventory will
then be updated as changes are
made to IT projects and systems.
Activity 3: Information from the
inventory is available on demand to
decision-makers and other affected
parties.
FBI personnel stated that the IT asset
inventory, when complete, will be
maintained on the FBI’s Intranet, so
that relevant information will be
available on demand to decision-
makers and other affected parties.
Activity 4: The IT project and system
inventory and its information records are
maintained to contribute to future
investment selections and assessments.
FBI personnel stated that the IT asset
inventory and IT portfolio, when
complete, will be updated continually
to become an archive of information
to be used for future investment
selections and evaluations.
Source: OIG analyses
- 65 -
pg_0083
Business Needs Identification Critical Process
Key Practice Activity Not Executed How the Plan Addresses the Activity
Activity 3: Identified users participate in
project management throughout a
project's life-cycle.
The Plan states that it is crucial for
project team members (which must
include identified users of the project)
to work closely together throughout
the project’s life-cycle. These project
teams support the functional project
management office and Project
Oversight Committee.
Source: OIG analyses
With the pilot testing of the select phase, the FBI further
developed and refined the proposal selection process and provided
training on proposal selection to ITIM users. The training materials
supplemented and supported the documentation in the Plan to more
clearly define the roles of ITIM users, such as IT investment review
board members, project sponsors, and ITIM liaison representatives.
Even with these additional materials, the Post-Implementation
Review of the select phase of the Plan (performed by the ITIM
contractor) recommended that the FBI significantly expand its
documentation of polices and procedures relating to the ITIM process
by:
explicitly defining the ITIM Program Office’s roles and
responsibilities so that resources can be concentrated on
enabling and facilitating the process as well as supporting the
development of process input;
developing and documenting detailed policy, processes, and
procedures in a stand-alone document independent of the Plan;
developing a formal ITIM training program that includes focused
training on the roles of various ITIM users, including board
members and ITIM liaison representatives;
developing a formal communications plan to ensure all ITIM
users are provided with visibility and timely feedback from the
ITIM process; and
- 66 -
pg_0084
refining and expanding ITIM tools necessary to sustain the
process, including an “IT investment proposal tracking
management tool.”
45
The FBI recognized that the Plan was never intended to
represent its final policies and procedures for its ITIM process. The
Plan states that it provides a conceptual framework for achieving
Stage Two maturity, and will evolve as the FBI’s ITIM process
advances to higher levels of maturity.
Without further development and refinement of the ITIM
process, the FBI will have difficulty making additional progress in
improving its IT investment management practices. Because the goal
of Stage Two maturity is to build standardized methodologies for
selecting and controlling IT investments, the FBI must have adequate
documentation of these methodologies to make them repeatable and
institutionalized. The Post-Implementation Review, prepared by the
ITIM contractor, acknowledged the necessity for further developing
and refining the Plan. In our judgment, the FBI must implement the
recommendations set forth in the Post-Implementation Review prior to
taking further action in pilot testing the control and evaluate phases of
the ITIM process.
c. Recommendation
We recommend that the Director of the FBI ensure:
16. The recommendations set forth in the Post-Implementation
Review relating to expanding the policies and procedures of the
ITIM process are implemented.
(2) The Amount of Participation from ITIM Users in Developing
the ITIM Process
In our judgment, the Plan was written with minimal input
and coordination from relevant ITIM users. The main reason cited by
45
According to the Post-Implementation Review, this tool would formally
track and document the entire life-cycle of an IT investment proposal from the time
the ITIM Program Office receives a concept paper to the time the final disposition is
made.
- 67 -
pg_0085
IRD officials
46
for the limited participation from ITIM users was
insufficient time allotted to develop the Plan. As a result, the
institutionalization and buy-in
47
of the ITIM process may have been
hampered.
a. Importance of ITIM User Participation in Developing the
ITIM Process
Good management practices dictate that organizations involve
relevant stakeholders when attempting to implement a new
management process. This involvement aids in the institutionalization
of the process. Institutionalization of the ITIM process is a key goal of
the Plan, which states: “[The ITIM] process applies to ALL information
technology projects, from ALL business units, from ALL funding
sources, whether they be new, in development or operational.”
Because of the broad applicability of the ITIM process, in our
judgment the FBI should have involved representatives from
throughout the Bureau when developing the Plan. In particular,
individuals from the three divisions that manage major IT projects
(the IRD, CJIS, and Laboratory Divisions) should have had substantial
input into the creation of the Plan. Further, the Inspection Division’s
Major Project Management Oversight Unit (MPMOU) has a
responsibility to oversee major projects in the Bureau, including IT
projects, and thus should also have been involved in creating the Plan.
b. Results of Our Assessment of ITIM User Participation in
Developing the ITIM Process
We found that relevant ITIM users from the IRD, CJIS Division,
Laboratory Division, and Inspection Division were not given significant
input into how the Plan was developed. Our interviews with IRD
personnel indicated that the FBI gave the ITIM contractor the primary
responsibility to write the Plan, without requiring significant
participation from ITIM users in developing the initial draft of the Plan.
46
The Information Resources Management Section, maintained within the IRD
until February 2002, was directed to oversee the development of the FBI’s ITIM
process. In February 2002, the Information Resources Management Section was
moved from the IRD to the Office of the Director. The ITIM Program Office was then
formed within the Information Resources Management Section to oversee the ITIM
process.
47
According to the Framework, institutionalization and buy-in of the ITIM
process is signified by ITIM users supporting and executing ITIM process activities.
- 68 -
pg_0086
Additionally, we determined that while the contractor interviewed
numerous individuals from the IRD, it only interviewed two people
from the Inspection Division, one person from the CJIS Division, and
none from the Laboratory Division.
48
Further, as we discuss below,
the enterprise architecture office (part of the IRD until February 2002)
was not given adequate input into the development of the ITIM
process. Also, the interviews that did occur outside of IRD mainly
focused on the individuals’ current responsibilities for managing IT
investments, rather than their insights into how the new ITIM process
could be shaped to best meet the needs of the Bureau. The following
paragraphs provide the perspectives of ITIM users from the IRD,
CJIS Division, Laboratory Division, and the Inspection Division.
Personnel from the enterprise architecture office told us that
because the FBI’s ITIM process had been developing concurrently with
the enterprise architecture function, there should have been more
coordination between the ITIM contractor and enterprise architecture
office to increase effectiveness and reduce duplication of effort. For
example, the enterprise architecture office drafted charters for a
three-tiered IT investment review board structure, similar to what was
ultimately written by the ITIM contractor. Additionally, the enterprise
architecture office was preparing initiatives to improve the FBI’s IT
investment management practices. While the enterprise architecture
office was drafting board charters and other processes designed to
improve the FBI’s IT investment management practices, the ITIM
contractor, supervised by the ITIM Program Office, wrote the Plan
without incorporating the work already accomplished by the enterprise
architecture office.
Additionally, an individual from the enterprise architecture office
told us that although he believed the ITIM process represents a
positive step for the FBI, it must incorporate more involvement from
the enterprise architecture function to ensure success of the process.
He further stated that the IT investment review boards must rely more
on the vast knowledge, expertise, and talents of FBI IT personnel prior
to making decisions.
Further, according to a manager in the Information Resource
Management Section, the Enterprise Architecture Technical
Committee, which supports the Technical Review Board, has not been
given the responsibility to ensure that IT proposals align with the
48
The ITIM Program Office has the ultimate responsibility for directing the
actions of the ITIM contractor.
- 69 -
pg_0087
mission of the FBI. The responsibilities of the Technical Review Board,
as defined in the Plan, are focused on reviewing the technical risks of
IT projects. These technical risks include compliance with the
“technical architecture” or configuration management of the FBI,
rather than the business architecture which shows how the business
processes work together to satisfy the mission. The Plan and board
charters assigned this responsibility to the Project Oversight
Committee. In our judgment, because the responsibilities of the
enterprise architecture office comprise both the technical and business
architecture, the Enterprise Architecture Technical Committee should
not only be responsible for assessing compliance with the technical
architecture, but should also be responsible for assessing compliance
with the business architecture. This added responsibility would
provide greater assurance to FBI executives that IT proposals selected
will enhance the Bureau’s capability in achieving its mission.
An official from the CJIS Division told us that he was interviewed
by representatives from the ITIM contractor on one occasion to
determine what role the CJIS Division had in managing IT projects.
However, he was not consulted on how the FBI’s ITIM process should
be created. He stated the only opportunity he had to comment on the
Plan was after it was written in January 2002. His belief was that the
ITIM Program Office was relying solely on the contractor to write the
Plan, rather than building a Plan that has the input and buy-in from all
FBI divisions.
While this official from the CJIS Division said to us that the Plan
was an improvement over the FBI’s current process for managing IT
investments, he was not convinced that the process could be
effectively implemented without addressing other pressing issues, such
as the need for: (1) standardized methodologies in configuration
management, quality assurance, and IT security; (2) improved support
of contractors that work on IT systems; and (3) more representation
of individuals with IT technical expertise on the IT investment review
boards.
An official from the Laboratory Division’s project management
office told us that he first became aware of the Plan when training was
announced for the new ITIM process in February 2002. Another
official from the Laboratory Division told us that to his knowledge, no
one from the Laboratory Division was consulted by the ITIM contractor
prior to the preparation of the Plan. He told us that the Laboratory
Division’s current process was working fine and not in need of change.
- 70 -
pg_0088
Additionally, Inspection Division personnel, including individuals
from the MPMOU, told us (as of June 2002) they were only consulted
by the ITIM contractor as to how they acquired IT, not for their project
oversight role.
An official from the Information Resources Management Section
cited the insufficient amount of time allotted to prepare the Plan as the
main cause for the limited involvement from ITIM users. As we
previously mentioned, the FBI waited until December 2001 to engage
the ITIM contractor to develop the Plan, despite learning of the DOJ’s
requirements to prepare a plan in January 2001. The ITIM Program
Office Manager stated that the former Chief Financial Officer did not
initially approve the use of an outside contractor to develop the Plan,
causing a delay in hiring the contractor. The former Chief Financial
Officer confirmed to us that there were initial concerns in using an
outside contractor to develop a management process that affects how
the IT budget is allocated and spent. Because the DOJ required
initiation of the ITIM process during the FY 2004 budget cycle (which
for the FBI begins in March), there was limited time between the
development of the Plan (December 2001) and the initiation of the ITIM
process (March 2002). In fact, the FBI only gave the contractor
approximately two weeks to write the Plan because of the impending
deadline to submit the Plan to JMD. As a result, FBI personnel told us
that the ITIM contractor did not have ample time to include more ITIM
users in the Plan’s development.
While FBI officials from the Information Resources Management
Section acknowledged the ITIM contractor’s time constraints in
developing the Plan, they also stated that the Plan is only a draft, and
will be modified as the ITIM process is pilot tested. Additionally,
because the three IT investment review boards established by the ITIM
process include representatives from the major divisions that manage
IT projects, officials from the Information Resources Management
Section told us that there is significant opportunity for input into
refining the ITIM process as it is being pilot tested.
Despite the Information Resource Management Section’s position
that the pilot test provides ample opportunity for input into refining the
ITIM process, in our judgment, the ITIM Program Office, along with the
ITIM contractor, continues to develop the ITIM process without
incorporating sufficient input from relevant stakeholders. For example,
a manager from the enterprise architecture office stated to us in
July 2002 that the ITIM Program Office had not requested his
participation during development of the control phase of the ITIM
- 71 -
pg_0089
process. This individual told us the enterprise architecture function
should have a role in enhancing the control and evaluate phases of the
ITIM process, but has not had the opportunity to demonstrate this role.
Additionally, the process for the development of the control phase has
not substantially changed from the select phase: the ITIM contractor,
supervised by the ITIM Program Office, writes the policies and
procedures which are then pilot tested by the ITIM users. In our
judgment, this approach is not conducive to a process whose success
depends on institutionalization and buy-in from ITIM users.
c. Summary
In our judgment, the lack of involvement by relevant ITIM users
inhibits management buy-in to the ITIM process. If there had been
more participation in the development of the Plan, some of the
concerns stated above by key ITIM users might have been mitigated.
The FBI must address these concerns to facilitate the
institutionalization and buy-in the of the ITIM process, and ultimately
improve its effectiveness.
d. Recommendations
We recommend that the Director of the FBI ensure:
17. The ITIM Program Office and the ITIM contractor incorporate the
input from various ITIM users, including those from the
enterprise architecture office, the CJIS Division, the Laboratory
Division, and the Inspection Division as the control and evaluate
phases of the ITIM process are being developed and refined. This
input should be solicited through working group sessions
scheduled on a periodic basis.
18. The ITIM process is modified so that the Technical Review Board
and Enterprise Architecture Technical Committee perform a
business architecture compliance review of IT project proposals to
ensure these proposals support the mission of the FBI.
(3) The Project Management Function’s Support of the ITIM
Process
The FBI’s project management function needs improvement to
adequately support the ITIM process, especially in the control and
evaluate phases of the process. The FBI recognizes the importance of
upgrading the project management function. In particular, the Plan
- 72 -
pg_0090
states that the project management office must fulfill a critical role in
supporting the Project Oversight Committee. In addition to the Plan,
the FBI has taken other steps towards improving its project
management function. Specifically, in June 2002, the FBI announced
plans to create an Office of Programs Management. The Office of
Programs Management will serve as a centralized project management
office
49
that FBI officials from this office and the Information Resources
Management section expect to play a key role in implementing the
ITIM process. Despite the progress being made, the FBI still has
critical areas to address, such as integrating a project management
methodology with its ITIM process.
a. Relationship Between Project Management and ITIM
Numerous legislative mandates, including the Results Act and
the Clinger-Cohen Act, require federal agencies to establish and
maintain processes for managing systems throughout their life-cycle.
These legislative mandates indicate that basic project management
practices are essential if an organization is to ensure that its IT
projects have established cost, schedule, and technical performance
baselines that are monitored throughout the project’s life-cycle.
Additionally, project management is fundamental to supporting an
ITIM process. In particular, the control phase of an ITIM process
requires an organization to have a project management function. For
example, IT project oversight, which encompasses basic project
management practices, must be implemented for an organization to
achieve Stage Two maturity. However, the Framework does not by
itself provide a comprehensive model for how an organization should
develop its project management function.
According to the Framework, an ITIM process is not a substitute
for good project management. While an ITIM process takes an
enterprise-wide focus, good project-level management forms the
foundation for successful IT investments.
In our judgment, for the FBI’s project management function to
effectively support its ITIM process, the Bureau must have: (1) a fully
operational centralized project management office whose
responsibilities are directly integrated with the ITIM process, and
(2) a standardized project management methodology that is
49
In this context, a centralized project management office is independent of
any division. As a result, the Project Management Executive, who heads the Office
of Programs Management, reports to the Director.
- 73 -
pg_0091
integrated with the ITIM process. Because of the importance of these
efforts, we assessed the FBI’s progress in integrating these areas with
its ITIM process.
b. Importance of a Centralized Project Management Office
The Plan recommends that project teams be staffed from a
“pool” of managers and developers maintained in the project
management office. These project teams would not be dedicated to
solely one division, function, or application; instead, these teams
would work on all types of IT projects across the Bureau. According to
the Plan, this approach has many benefits, including:
critical IT skills are available across all projects;
personnel have more opportunities to work in multiple
environments, which creates a richer, more interesting job
environment;
expertise across projects enhances and encourages the use of
best practices; and
managers are better able to assess IT personnel as they perform
in multiple project environments.
We concur with the Plan’s recommendations. Although the Plan
does not specifically state that the project management office should
be centralized (independent of any division), in our judgment, such a
structure is most conducive to attaining the benefits listed above.
In addition to the above benefits, a centralized project
management office can ensure that IT project teams are following a
standardized project management methodology that is integrated with
the ITIM process. In our judgment, this added control is especially
important to the FBI since we previously concluded that the FBI’s three
main divisions that manage IT projects (the IRD, CJIS, and Laboratory
Divisions) have not been consistently using a standardized project
management methodology.
c. Importance of a Standardized Project Management
Methodology
The DOJ recognized the importance of integrating project
management with the ITIM process. In January 2001, it issued DOJ
- 74 -
pg_0092
Order 2880.3 to require components to manage IT investments in a
way that demonstrates good stewardship, complies with applicable
laws, and accomplishes the agency’s diverse mission. Among its
policies, the Order required each DOJ component to establish an ITIM
process that is integrated with a structured system development life-
cycle methodology. While the FBI is mandated to use the DOJ’s
System Development Life-Cycle methodology, we previously stated in
this report that it has not been used consistently.
d. Results of Our Assessment of the FBI’s Progress in
Integrating its ITIM Process with the Responsibilities of a
Centralized Project Management Office
As discussed below, we concluded that the FBI has recently
made progress in integrating its ITIM process with the responsibilities
of a centralized project management office. Not only does the FBI
recognize the importance of this integration, but it has taken major
steps towards incorporating the ITIM process with the responsibilities
of a centralized project management office. This progress was
evidenced by: (1) how the Plan defined the role of the project
management function, and (2) the FBI’s recent efforts to establish a
centralized project management office.
The Plan recommends centralization of IT investment
management through the use of IT investment review boards that
have Bureau-wide oversight. Of the FBI’s three IT investment review
boards, the Project Oversight Committee has the primary responsibility
for controlling IT projects. Additionally, the Plan calls for a project
management office, a subcommittee of the Project Oversight
Committee, to have discretion in managing IT projects Bureau-wide.
Specifically, the Plan defines how the primary responsibilities of
the project management office must be integrated with the activities
of the ITIM process, particularly during the control and evaluate
phases. These responsibilities include:
ensuring that resources, funding, and schedule timeframes are
reasonable for each individual project;
determining what staff and funding are needed for a project, and
assigning staff and funding accordingly;
- 75 -
pg_0093
providing advice and counsel to internal project teams in the
execution of ITIM activities;
providing a consistent set of project management tools and
processes for ITIM projects;
providing tools to project team members, such as Gantt charts,
Pertt charts, and Microsoft Project;
providing governing responsibility and oversight to day-to-day
project managers; and
determining whether project goals are achieved on time, on
budget, and as designed.
We were told in June 2002 that the Director of the FBI approved
the creation of a centralized project management office, whose chief
executive would report to the Director.
50
This project management
office, which would be independent of all other FBI divisions, would
have the primary responsibility of managing projects in the Bureau.
These projects would include, but not be limited to, information
technology. The proposed mission for this new office is: “To assist the
FBI in effectively managing, implementing, and deploying high-
priority, complex and high risk development projects of high dollar
value to successfully support the FBI’s operational mission.” To
achieve this mission, this office will be:
developing a repeatable process for the efforts described in the
mission statement (defined above) and for training a skilled
corps of FBI project management subject matter experts;
advising on program management and acquisition-planning
related organizational issues, proposals, and strategies;
providing direct project management support in developing the
crucial technology infrastructure for FBI investigation operations;
and
coordinating organizational resource allocation and management
services and supporting the FBI’s mission and priorities.
50
The FBI is calling this office the “Office of Programs Management.” As
planned by the FBI, this office will be under the Director’s office and independent of
any division.
- 76 -
pg_0094
In addition, the Office of Programs Management has the
following core functions for which it will ultimately be responsible:
(1) system engineering, (2) schedule, (3) budget, (4) risks,
(5) contract management, (6) certification and accreditation of IT
systems, (7) configuration management, and (8) quality assurance.
In our judgment, the creation of the Office of Programs
Management represents a critical first step towards centralizing the
project management function and improving its effectiveness.
Additionally, officials from the Information Resources Management
Section and the Office of Programs Management have told us that they
are working together to facilitate the integration of the responsibilities
of the eight core functions listed above. The ITIM process needs the
full support of the Office of Programs Management to implement the
control and evaluate phases of the Plan. Therefore, in our judgment,
the FBI should continue its efforts to integrate the responsibilities of
the Office of Programs Management with the ITIM process.
Specifically, a plan should be developed that outlines activities that
must be performed to complete the integration, along with reasonable
suspense dates. Additionally, this plan should provide the criteria and
thresholds that the Office of Programs Management will use to select
IT projects for review.
e. Results of Our Assessment of the FBI’s Progress in
Integrating its ITIM Process with a Standardized Project
Management Methodology
We concluded that the FBI has not taken the necessary actions
to integrate the ITIM process with a standardized project management
methodology. While officials from the Information Resources
Management Section have acknowledged to us that the ITIM process
needs to be integrated with a standardized project management
methodology, they have not taken sufficient action to ensure that
these processes are integrated in a timely manner. This conclusion is
evidenced by the Information Resources Management Section’s lack of
coordination with the Inspection Division’s Major Project Management
Oversight Unit (MPMOU), as previously reported in this section.
Additionally, as discussed in the following paragraphs, the FBI risks
duplicating efforts in managing IT projects if it implements the control
and evaluate phases of the ITIM process without integrating these
phases first with a standardized project management methodology.
- 77 -
pg_0095
To improve the FBI’s ability to manage projects, including IT
projects, the prior FBI Director requested that the MPMOU establish a
standardized project management methodology for Bureau-wide use.
In October 2001, the MPMOU completed the Project Management
Process and submitted it to executive management for approval. The
Project Management Process, which incorporates the DOJ’s System
Development Life-Cycle methodology, provides a framework that
encompasses all phases of a project’s life-cycle, including planning,
developing, support, and disposal.
Personnel from the MPMOU stated to us that the Project
Management Process provides a mechanism to fulfill certain
requirements of the ITIM process. Specifically, personnel from the
MPMOU told us that the project management process facilitates the
ITIM process by:
providing documentation to support investment decisions that
span the life-cycle of the IT investment;
providing a select, control, evaluate approach to managing
validated IT needs;
providing quantifiable measurements for monitoring cost,
schedule, and performance baselines and processes for
identifying baseline breaches;
providing an executive oversight forum for monitoring the
management of IT investments; and
acknowledging the interdependencies between cross-cutting
processes.
According to MPMOU personnel, given their knowledge of the
FBI’s requirement to develop an ITIM process, they made repeated
attempts beginning in 2001 to work with individuals from the
Information Resources Management Section to develop these
processes concurrently.
In November 2001, personnel from the MPMOU prepared a
presentation entitled “Project Management Process Compatibility with
the ITIM Process” to show appropriate individuals from the IRD the
similarities between the two processes. However, according to MPMOU
personnel, individuals from the IRD who were managing the
development of the ITIM process never gave MPMOU the opportunity
- 78 -
pg_0096
to make their presentation. In April 2002, after the development and
initiation of the ITIM process, the MPMOU sent an electronic
communication to the Director’s office explaining the need to integrate
these processes. The electronic communication stated that integration
of these processes would improve efficiencies, streamline reporting
and paperwork requirements, and improve the FBI’s compliance with
applicable regulations, including DOJ Order 2880.3. As of
June 2002, no additional action had been taken by the Information
Resources Management Section to integrate these processes.
Despite the efforts by the MPMOU to integrate the two
processes, the Information Resources Management Section (with the
support of the ITIM contractor) developed and began implementation
of the FBI’s IT Investment Model and Transition Plan without
attempting to integrate it with the Project Management Process. Until
the FBI integrates these two processes, the FBI will not be in
compliance with DOJ Order 2880.3. Additionally, the FBI will be
unable to effectively implement the control phase and evaluate phases
of the ITIM process. Further, the FBI risks inefficient use of resources
as a result of the duplication of efforts that could occur if the FBI fails
to integrate these processes. FBI officials from the Information
Resources Management Section have acknowledged to us that they
must integrate the control and evaluate phases of the ITIM process
with a standardized project management methodology. Despite their
recognition of this need, as of June 2002 they did not have the details
of how or when this will occur.
f. Summary
Although the FBI has taken a critical first step in (1) centralizing
its project management structure, and (2) incorporating the
responsibilities of the Office of Programs Management with the ITIM
process, the FBI must take further action in integrating its ITIM
process with a standardized project management methodology.
Without this further action, the FBI’s project management function will
not adequately support the ITIM process. Consequently, the FBI risks
ineffective execution of its control and evaluate phases as well as
inefficient use of resources in managing its IT investments.
- 79 -
pg_0097
g. Recommendations
We recommend that the Director of the FBI ensure:
19. The FBI prepares a plan that specifically details how the project
management office will support the ITIM process. This plan
should include the project management office’s criteria and
thresholds for: (a) selecting IT projects to manage, and
(b) identifying projects that the Project Oversight Committee will
review.
20. The FBI develops and implements a specific plan detailing how
and when it will integrate the ITIM process with a system
development life-cycle methodology such as the Project
Management Process.
(4) The Enterprise Architecture Function’s Support of the ITIM
Process
The FBI’s enterprise architecture function needs improvement to
adequately support the ITIM process. The FBI has taken a critical first
step in establishing an enterprise architecture framework with a
limited amount of time and resources dedicated to this effort. Despite
the progress being made, the lack of a fully developed enterprise
architecture framework will hamper the FBI’s ability to advance
through the ITIM maturity framework.
a. Importance of Having Support from the Enterprise
Architecture Function
Enterprise architecture is the organization-wide blueprint that
defines an entity’s functions and systems, including IT systems. It
provides a comprehensive view (through models, narratives, and
diagrams) of the interrelationships of an organization’s operations and
structures and how these structures align with the organization’s
mission. The Clinger-Cohen Act of 1996 recognizes the
interrelationship between enterprise architecture and IT investment
management by requiring federal agencies to develop an enterprise
architecture.
- 80 -
pg_0098