CSA
Summary
Read the report at California State Auditor ↗
REPORT BY THE STATE AUDITOR
OF CALIFORNIA
THE DEPARTMENT OF MOTOR VEHICLES AND THE
OFFICE OF INFORMATION TECHNOLOGY DID NOT
MINIMIZE THE STATE’S FINANCIAL RISK
IN THE DATABASE REDEVELOPMENT PROJECT
94107 AUGUST 1994
The Department of Motor Vehicles and the
Office of Information Technology Did Not
Minimize the State’s Financial Risk In
the Database Redevelopment Project
94107, August 1994
California State Auditor
Bureau of State Audits
Table of Contents
Page
Summary S-1
Introduction 1
Chapter
1 The Department of Motor Vehicles Should Have
Stopped or Redirected the Implementation
of the Database Redevelopment Project
When Problems Arose During Early Testing 11
Recommendations 25
2 The Office of Information Technology
Should Not Have Recommended Continued
Funding for the Database Redevelopment
Project When the Department of Motor
Vehicles Did Not Follow the Approved
Policies To Minimize Financial Risk 27
Recommendations 34
3 The DMV’s True Costs of the DBR Project
Are Higher Than Originally Reported 35
Recommendations 39
4 The Department of Motor Vehicles
Violated Numerous Contracting
Laws and Regulations 41
Recommendations 48
Appendices
A Chronology of Key Events Related to
the Database Redevelopment Project 49
B Description of Technological Elements of the
Proposed Database Redevelopment System 55
C Responsible Officials 57
D Unreported Expenditures for the DBR Project 61
Responses to
the Audit Department of Motor Vehicles 63
California State Auditor’s Comments
on the Response From the
Department of Motor Vehicles 83
Department of Finance 85
California State Auditor’s Comments
on the Response From the
Department of Finance 91
Summary
Background Over the years, the Department of Motor Vehicles (DMV) has
developed very large and complex databases to support its activities.
More than a million transactions are processed by the DMV database
system every day. The current databases and their application
software programs were developed in the 1960s. Thus, the basic
technology of the databases is 30 years old, and it has become
increasingly difficult to maintain these old systems. As a result, the
DMV determined that it was necessary to modernize their databases.
The Database Redevelopment (DBR) project was initiated in 1987 to
redesign the DMV’s systems and databases to meet all existing
requirements and functions, structure the system to be more responsive
to future changes, and improve the efficiency of electronic data
processing services. The DBR project was intended to provide a
variety of improvements to the system that would allow the DMV to
improve service levels and move into the future. In addition to
addressing technical concerns, the DMV intended the DBR to address
certain strategic objectives. These objectives were prompted by
legislative mandates and proposals that would require the DMV to
cross-match information in their various databases, a capability it did
not have at that time.
Results in Brief During our comprehensive audit of the DMV’s DBR project, we found
that the DMV continued its effort to fully implement the project despite
significant unresolved problems and deficiencies, which led to the
ultimate failure of the project in 1994. Further, our audit revealed that
the Office of Information Technology (OIT), the State’s information
technology oversight body, continued to recommend additional funding
for the project despite the fact that the DMV had not followed approved
policies to minimize financial risk to the State. Additionally, we found
that the DMV’s actual and obligated costs were $5.1 million higher for
the project than originally reported to the Legislature and the
Department of Finance, and that the DMV violated numerous
contracting laws and regulations, including falsifying a purchase order
for approximately $46,000. Specifically, we noted the following:
S-1
The DMV progressed beyond the developmental stages of the
DBR—the operational assessment and the working model—even
though it had failed to accomplish the objectives of each stage and
had not resolved significant technical problems encountered during
the development process. In its unsuccessful attempt to implement
the DBR, the DMV spent an additional $34.6 million. Rather than
complete each stage as planned, the DMV substantially modified
the stages or failed to complete them altogether.
Initially, the DMV established a meaningful set of development
objectives for an operational assessment and working model of the
project. Each of these objectives was supposed to be accomplished
before proceeding with the project and incurring additional
expenses. However, the DMV continued to develop the system
and incur expenses without first solving technical and performance
problems that arose.
The OIT continued to recommend additional funding for the DBR
project, thus allowing the DMV to attempt to put the DBR project
into operation, even though it knew the DMV had not successfully
developed a working model of the project as the OIT had
previously required before the DMV could proceed with the
project.
The DMV did not use a formal cost-reporting system to monitor
expenditures related to the DBR project. The DMV’s actual and
obligated costs for the project were at least $49.4 million, which is
$5.1 million more than the $44.3 million in project costs that the
DMV originally reported to the Legislature and the Department of
Finance.
The DMV falsified a software purchase order for approximately
$46,000 to pay for consulting services that Tandem Computers, Inc.
had provided through the use of subcontractors. Approximately
$28,000 was for services provided before the DMV awarded
Tandem a contract, and approximately $18,000 was for services
that Tandem provided during the contract period, but for which
sufficient funds did not exist under the contract to pay for the
services. Although the DMV paid an invoice which indicated the
software was received, no software products were ever delivered.
The DMV did not always adequately justify its use of sole-source
consulting contracts for the DBR project. Specifically, of the four
sole-source consulting contracts into which the DMV entered for
the DBR project, the DMV did not adequately justify three, entered
into from November 1987 to July 1990 and totaling $2.8 million.
S-2
Additionally, the Department of General Services (DGS) approved
the contracts even though the DMV had not provided adequate
justification.
From November 1987 to November 1991, the DMV allowed
contractors to begin work before the DGS approved the contracts or
contract amendments. In addition, the DMV significantly
modified one contract without obtaining DGS’ approval.
These problems occurred because the project’s management desired to
keep the project on schedule, incorrectly believed that the technical
problems could be resolved, and incorrectly asserted that critical
developmental objectives had been achieved. In addition, the situation
was allowed to continue because the DMV management failed to
establish an effective quality assurance process that would provide an
independent assessment of progress and because management did not
satisfactorily react when they were ultimately informed by the project’s
management team that problems existed.
Furthermore, the OIT did not exercise its authority to insist that the
DMV accomplish the working model before recommending continued
funding.
Recommendation Department of Motor Vehicles
s
To ensure that the DMV does not unnecessarily incur costs of this
magnitude in the future, the DMV should complete the milestones it
establishes for the development of new electronic data processing
(EDP) systems; conduct sufficient testing and analysis to determine that
proposed EDP projects will achieve their projected benefits, both
monetary and programmatic; and ensure that technical problems
identified during quality assurance reviews of EDP projects are
resolved before continuing to devote resources to those projects.
To ensure that the DMV maintains and reports accurate cost records of
EDP projects, it should implement a standard cost reporting system for
all EDP projects. In addition, the DMV should obtain approval from
the Department of Finance if it expects to deviate by more than
10 percent from its approved level of funding for EDP projects.
S-3
To ensure that the DMV adequately protects the State’s financial and
legal interests and obtains the highest quality of services at the lowest
possible price, the DMV should follow all contracting laws and
regulations when awarding contracts. Further, it should consider
taking disciplinary action against the employees involved in falsifying
the purchase order.
Office of Information Technology
To ensure that departments are following approved policies to
minimize financial risk, the OIT should exercise its authority to
approve proposed expenditures for electronic data processing only if
established policies and procedures, which have been published and
maintained in the State Administrative Manual, have been met and
followed. Also, the OIT should ensure that departments accomplish
the objectives and requirements included in approved feasibility study
reports and special project reports before allowing projects to move
forward.
Agency Department of Motor Vehicles
Comments
The DMV agrees that there were mistakes in judgment and form during
the DBR project. Specifically, the DMV agrees with our conclusion
that the project should have been stopped and reassessed in 1990. The
DMV stated that, in general, our report provides a fair assessment of
the project history. However, the DMV does not believe that we
adequately recognized the revitalization efforts undertaken after 1991.
The DMV also agrees that a purchase order was falsified to make
payment for services rendered. However, the DMV does not agree
that there were numerous incidents of contracting law and regulation
violations. Finally, the DMV stated that it will immediately work to
implement each of our recommendations.
Department of Finance
The Department of Finance (department) agrees that our report presents
fairly the facts surrounding the DBR project. However, the
department disagrees with many of the conclusions drawn from these
facts. In particular, the department does not believe it violated any of
its policies and procedures in recommending approval of the DMV
project. Finally, the department indicated that it has implemented
steps in recent years to address some of the issues we identified.
S-4
Introduction
The Department of Motor Vehicles (DMV) fulfills a variety of
responsibilities, including the following:
Protecting the public interest in vehicle and vessel ownership;
Providing various revenue collection services for state and local
agencies;
Regulating the issuance and retention of drivers’ licenses;
Providing personal identification services to drivers and nondrivers;
Licensing and regulating occupations and businesses related to the
manufacture, transport, sale, and disposal of vehicles; and
Licensing and regulating occupations and businesses related to
driving instruction.
Over the years, the DMV has developed very large and complex
databases to support its activities in fulfilling these responsibilities.
For example, the driver’s license database consists of more than 30
million records, and the vehicle registration database consists of more
than 40 million records. More than a million on-line and batch
transactions are processed by the DMV’s database system every day.
The transactions are distributed nearly equally among three types:
Transactions that support DMV programs, such as vehicle
registration;
Transactions that support law enforcement activities; and
Transactions that support DMV headquarters activities and other
organizations, such as insurance companies.
The current databases and their application software programs were
developed in the 1960s and were originally designed to operate on
RCA computers. Since that time, the databases and their application
software have been moved twice, once in 1981 to Unisys computers
and once in 1989, when the DMV moved their databases and their
application software to the Stephen P. Teale Data Center (data center)
1
and changed to IBM computers. During these moves, the databases
and the software programs remained essentially the same. Thus, the
basic technology of the databases is 30 years old.
It has become increasingly difficult to maintain these old systems for
several reasons. First, many of the system’s software programs are
written in a computer language, “assembler,” that is no longer widely
used. Many of the system’s original programmers no longer work at
the DMV, and because this language is no longer widely used, it is
difficult to recruit programmers familiar with it. Second, the systems
have been changed so many times that making further changes requires
a great deal of time and effort because the programs have unusual,
redundant, and obsolete code. This problem is exacerbated by the fact
that the flow of the assembler language application programs is
extremely complex and convoluted; therefore, the effect of software
changes on other software programs is hard to determine. Third, all of
the software programs were written exclusively for the DMV;
therefore, the programs are not compatible with industry-standard
software programs that have been developed since the system was
designed. This means that new technological advances cannot be
readily implemented to improve DMV operations.
For these reasons, the DMV determined that it was necessary to
modernize its databases. The Database Redevelopment (DBR) project
is intended to achieve the following:
Redesign the DMV’s systems and databases to meet all existing
requirements and functions;
Structure the system to be more responsive to future changes; and
Improve the efficiency, productivity, quality, and timeliness of
electronic data processing services through the use of standard
software and new system development technologies.
The DBR was intended to provide a variety of improvements to the
system that would allow the DMV to improve service levels. These
improvements included the following:
Increased accuracy of records;
Improved collection of parking fines;
Greater capability to respond to nonroutine information requests;
2
Enhanced ability to adjust to program changes;
Improved ability to expand the databases; and
Reduced costs to maintain the system.
Although the DBR project was initiated to address the DMV’s
technical concerns about the system, the DMV decided to use the
database renovation as an opportunity to achieve certain strategic
objectives. In some instances, these objectives exceeded the scope of
the DMV’s responsibilities for licensing drivers and registering
vehicles. With the vision of a broadening mission in the future, the
DMV identified the need for a flexible information system to adapt
quickly to a changing set of mandates. This vision of a broadening
mission was prompted by certain legislative mandates and proposals
related to the DMV performing functions beyond licensing drivers and
registering vehicles.
Legislative Before and during the DBR project, the State Legislature has mandated
Mandates or proposed various laws, the implementation of which would require
changes to the DMV’s computer system. Many of the laws were
intended to achieve certain enforcement goals. For example, the
Legislature has proposed laws that would have required the DMV to
refuse to register vehicles whose owners had failed to pay traffic
violation fines, been convicted of driving under the influence, or failed
to satisfy a small claims judgment. The DMV opposed all the
proposals because the DMV’s databases did not have the capabilities
required to take these actions and because changing the databases to
achieve these capabilities would be expensive.
For example, the Legislature passed a law in 1984 requiring the DMV
to study the feasibility of cross-matching the names of vehicle owners
with the names of licensed drivers. The purpose of this
cross-matching was to enable the DMV to refuse to register vehicles to
uninsured drivers and uninsured vehicle owners. The conclusion of
this study was that the DMV could achieve the ability to cross-match
drivers to vehicle owners, with one-time costs of $8.3 million, by
collecting additional information from drivers and vehicle owners and
using the DMV’s existing automated name index. The successful
completion of the DBR project would have allowed the DMV to
quickly and easily cross-match much more information than drivers’
and owners’ names.
3
In 1987, the Legislature passed a bill requiring the DMV to establish a
program to cross-match the names of licensed drivers with those of
registered vehicle owners. The proposed law would have authorized
the DMV to recover the costs of the program through the collection of
fees from the users of the resultant data. However, the governor
vetoed this bill because of the significant costs and concern over the
intrusion upon citizens’ personal privacy.
Currently, the DMV maintains information about each of its three
major activities, licensing drivers, registering vehicles, and regulating
vehicle-related occupations, in three separate databases. To
accommodate current and future changes proposed by the Legislature,
the DMV would have to relate information contained in one database to
the information contained in another database. The most significant
strategic objective of the DBR was to provide the DMV with the ability
to more easily cross-match data among the three databases.
Cross-matching would allow the DMV to support programs that
depend on relating the information contained in one database to the
information contained in another database. For example, the DMV
would be able to match the following kinds of information:
Drivers to vehicles;
Unpaid parking citations to the drivers responsible (these are
currently matched to the vehicle record);
Vehicle dealerships to the vehicles they sell; and
Financial institutions to the vehicles they finance.
In addition, cross-matching would enable the DMV to support and
enforce a mandatory insurance law.
Project The DMV initiated the DBR project in December 1987 when the Office
Chronology of Information Technology (OIT) approved the first feasibility study
report (study report) that the DMV submitted. At that time, the DMV
had already decided to convert its databases to IBM’s DB2 relational
database management system. The project schedule included in the
study report stated that the DMV planned to develop and test a working
model to determine the feasibility of proceeding with the project and to
refine the project cost and schedule estimates. The DMV’s
preliminary
4
estimate of the project’s cost was approximately $29 million. After
this analysis, the DMV planned to request OIT’s approval to proceed
with the project.
In July 1988, after expanding its research of relational database
management systems, the DMV selected IBM’s DB2 and Tandem
Computers’ NonStop SQL for a final comparison to
determine the relational database product that best met its needs.
From September 1988 to December 1988, the DMV conducted an
operational assessment of the two relational database management
systems. As a result of this assessment, the DMV selected the Tandem
database management system.
In September 1988, the DMV requested OIT’s approval to proceed
with the project. At this time, the DMV stated that it planned to
conduct a working model of the project. In December 1988, the OIT
authorized the DMV to proceed with the working model, on the
condition that the DMV not proceed with full implementation activities
until the OIT approved the DMV’s report on the working model’s
results.
Beginning in September 1988, the DMV conducted a request for
proposal process that eventually procured the services of a consultant to
act as co-project manager for the implementation of the driver’s
license, vehicle registration, and occupational licensing databases on
the new database system. In addition, the request for proposal
required the consultant to evaluate the working model and design and
develop the new database systems using IBM’s DB2 system or another
system to be named later. After the bidding consultants’ draft
proposals were received, the DMV amended the request for proposal to
substitute IBM’s DB2 with Tandem Computers’ NonStop SQL system.
The contract was let to Arthur Young (later Ernst & Young) in June
1989. In June 1990, one year after the contract’s commencement, the
DMV and Ernst & Young mutually agreed to terminate this contract.
Even though the DMV had contracted for co-project management in
recognition of its need for project management assistance, the DMV
assumed total project management responsibility for development of
the entire database system after Ernst & Young left.
By June 1990, the DMV began full implementation of the system. At
that time, the DMV began to hire contractors to assist with specific
elements of the DBR project. Seven of the next eight consulting
contracts were for work on the DBR project where tasks were assigned
as needs arose. As the consultants raised additional technical
5
problems, the DMV as project manager had the responsibility to
resolve these problems.
Because full project implementation was not progressing to the DMV’s
satisfaction, the DMV accepted Tandem’s assistance in October 1991
to validate and, if necessary, modify the current project plan for the
driver’s license system. Tandem submitted its report on this review in
June 1992. As a result of this review, the DMV increased the project’s
estimated costs from $31.4 million to $57.3 million and extended the
estimated completion date from July 1995 to December 1998.
In April 1993, Tandem, in partnership with Electronic Data Systems
(EDS), began assisting the DMV to develop a new project plan. The
DMV and Tandem/EDS based the review of this plan on Tandem/EDS’
management of the project, Tandem/EDS’ management of the Tandem
computer system, and Tandem/EDS’ guarantee that the project be
completed within a defined implementation period. In November
1993, Tandem presented its proposal for the new project plan to the
DMV. Tandem estimated that the costs for finishing the
implementation of the project would be $185 million—$175 million for
Tandem/EDS’ costs and $10 million for DMV costs. The DMV
rejected this proposal and pursued the DBR project with an alternate
plan.
Appendix A contains a more detailed chronology of key events related
to the DBR project. Appendix B contains information about the
officials responsible for the DBR project.
Scope and We were requested to perform a comprehensive audit of the DMV’s
Methodology DBR project. The audit objectives included the following:
Documenting the chronology of project events;
Determining whether project expenditures were properly authorized
by the DMV and the Department of Finance, and the extent to
which the Legislature was provided appropriate opportunity for
oversight;
Determining whether the DMV complied with laws and regulations
governing contract awards;
Determining whether contractors fulfilled their obligations before
the DMV paid them;
Determining what role the data center played in the project;
6
Determining whether the OIT reviewed and approved the project in
accordance with state law and policy;
Determining the length of time between identification of problems
and actions taken and whether problems could have been
anticipated or prevented; and
Identifying the specific causes for the project’s failure.
In addition, we were requested to determine whether any state
employees or contractors associated with the project had any potential
or actual conflicts of interest and if the State could recover any of the
costs of equipment purchased for the project.
To document the chronology of project events, we reviewed extensive
documentation related to the project contained in the files at the DMV,
the OIT, and the data center. This documentation included study
reports, contracts, contract amendments, personnel records, special
project reports, and correspondence.
To determine whether the DMV and the Department of Finance
properly authorized project expenditures, we reviewed budget
documents, such as budget change proposals and special project
reports. To determine the extent to which the DMV provided the
Legislature with appropriate opportunity for project oversight, we
identified the state laws and the DMV’s internal policies related to
reporting budget transfers and other project changes to the Legislature
and we reviewed the DMV’s compliance with these laws and policies.
To determine whether the DMV reported to the Legislature all project
costs, including funds already spent and future obligations, we
reviewed all supporting documentation for project costs prepared by the
DMV and the data center. We did not audit these reported project
costs; rather, we attempted to identify expenditures that the DMV did
not include in its 1994 special project report.
To determine whether the DMV complied with laws and policies
governing contract awards, we reviewed the Public Contract Code and
applicable sections of the State Administrative Manual. We reviewed
contracts related to the DBR project to determine if the DMV complied
with the applicable laws and policies. In addition, we reviewed the
related maintenance contracts and purchase orders to determine if the
DMV received and paid for the deliverables. During our review of the
purchase orders, we found one instance where the department falsified
7
a purchase order to pay a vendor for services that could not be paid
from an existing contract. We are referring this matter to the Attorney
General’s Office for further review.
To determine if project consultants fulfilled their obligations before the
DMV paid them, we reviewed the consulting contracts related to the
project and the deliverables that the contractors provided to the DMV.
We reviewed the contracts or the related work orders to determine if
each contract specified performance prerequisites for payment. For all
but one of the contracts, we judgmentally selected a sample of up to
five deliverables and determined if the deliverable fulfilled the
obligation specified in the contract or work order. For one contract,
we reviewed all of the contract deliverables because of the unusual
circumstances surrounding the contract.
To determine whether the OIT reviewed the project in accordance with
state law and policy, we identified and reviewed the applicable sections
of the Government Code and the State Administrative Manual. We
then interviewed OIT officials and reviewed the OIT’s documents and
correspondence relating to the project.
To identify the causes of the project’s failure, we interviewed DMV
and OIT officials. We also interviewed the project’s former managers
and directors. However, one former project director refused to allow
us to interview him. In addition, we reviewed the documented events
and decisions related to the project and compared them to the project
management plan the DMV had intended to follow.
Our office is currently conducting an investigation of the state
employees and contractors involved in the project to determine whether
they had any potential or actual conflicts of interest. In addition to our
investigation, the matter has been referred to the Fair Political Practices
Commission. Information related to our investigation of conflicts of
interest is not available for inclusion in this report because the
investigation is not complete. One of the reasons for this delay is the
refusal of one of the former project directors to allow us to interview
him.
Rather than unnecessarily duplicate the work of others, we intend to
rely on the results of a DBR project review performed by an outside
consultant to determine if the State can recover the cost of the
equipment purchased for the project. The DMV has recently issued its
RFP for this contract. We will monitor the consultant’s work, and if
we find that this area is not adequately addressed, we will perform
8
additional work and issue a separate report on the possibility of cost
recoveries through the sale or redirection of computer equipment.
Finally, we engaged the services of two EDP consultants to assist us in
our analysis of the DBR project.
9
Chapter 1
The Department of Motor Vehicles Should Have
Stopped or Redirected the Implementation
of the Database Redevelopment Project
When Problems Arose During Early Testing
Chapter The Department of Motor Vehicles (DMV) continued its effort to fully
Summary implement the Database Redevelopment (DBR) project despite
significant unresolved problems and deficiencies. The DMV
progressed beyond the developmental stages of the DBR project—the
operational assessment and the working model—even though it had
failed to accomplish the objectives of each stage and had not resolved
significant technical problems encountered during the development
process. In its unsuccessful attempt to implement the DBR, the DMV
spent an additional $34.6 million. Rather than complete each stage as
planned, the DMV substantially modified the stages or failed to
complete them altogether.
The DMV continued to develop the system and incur expenses without
first solving technical and performance problems because the project’s
management desired to keep the project on schedule, incorrectly
believed the technical problems could be resolved, and incorrectly
asserted that critical developmental objectives had been achieved.
Furthermore, the situation was allowed to continue because DMV
management failed to establish an effective quality assurance process
that would provide an independent assessment of progress and because
management did not satisfactorily react when informed by the project’s
management team that problems existed. As a result, the DMV did not
stop the project in June 1990 after the working model stage,
when it had spent approximately $8.3 million to develop the
system and incurred obligations of $6.5 million to purchase
computers, for a total cost of $14.8 million. Instead, the DMV
endeavored unsuccessfully to implement the project, ultimately
spending another $34.6 million for a total project cost of approximately
$49.4 million, on a system that has failed because of many of the
unresolved technical and performance problems that were identified
during the developmental stages of the project.
10
Significant A key technical concern arose during the DMV’s testing of two
Concerns relational database management systems. Specifically, the series of
tests the DMV conducted, known as an operational assessment,
Identified During
indicated that the proposed system would require significantly more
Operational
time to process transactions than the current system required.
Assessment
Additionally, the operational assessment did not fully test other critical
technical concerns, such as the difficulty of redeveloping the software
programs or the difficulty of establishing procedures to keep the system
running smoothly. Because these areas were critical to the success of
the DBR project, the DMV should have ensured that it resolved them
during the working model phase of the project, which is discussed in
more detail in the following section.
The operational assessment was designed to evaluate and recommend a
database management system for the DMV to use in redeveloping its
databases. A database management system is the basic software that
provides the capability to store or retrieve data flexibly and accurately.
The operational assessment consisted of a series of standardized tests
applied to two database software products: Tandem Computers’
NonStop SQL and IBM’s DB2. The tests were designed to assess
each product for continuous availability, performance, and ease-of-use.
The DMV had previously narrowed the potential candidates down to
these two products through an evaluation questionnaire sent to four
companies offering products that the DMV identified as potentially
able to satisfy its requirements.
A team composed of staff from the DMV, consultants hired by the
DMV, the Stephen P. Teale Data Center (data center), and the vendors
developed and executed the tests. Some of the operational tests were
conducted at each company’s laboratory, and some were conducted at
the data center. The operational assessment primarily focused on a
number of tests of the system’s continuous availability, performance,
and ease-of-use. The tests were conducted on a simulated database of
vehicle registration records. This simulated database was much
simpler in design than the ultimate design of the actual database
system. For example, the design of relational databases is based on
“tables.” The simulated database contained 12 to 15 tables of data
while the final design of the driver’s license database contained more
than 170 tables. In addition to the operational tests, the DMV
evaluated the products, based on the vendors’ responses to a written
questionnaire, in other areas, such as the availability of training and
computer-aided software engineering tools.
11
The DMV originally planned to conduct 47 operational tests, but
because of time constraints the DMV prioritized the tests to fit within
allowable time frames. The DMV deleted some tests because the
DMV believed they did not relate to the products under evaluation,
they tested low risk areas, or they were redundant with other tests.
The DMV eventually performed 23 operational tests. The DMV
designated 10 of the 47 planned tests to be key pass or fail tests. The
DMV did not assign a pass or fail threshold to the other 37 tests, but
intended to compare the two systems’ performance on the tests. The
DMV conducted all 10 of the key tests. Six of the key operational
tests related to the system’s response time. The other 4 related to the
system’s ability to recover from various equipment failures. The tests
that the DMV dropped from the operational assessment included,
among other things, assessments of the system’s ability to do the
following:
Perform certain types of database modifications;
Update the database statistics;
Upgrade the system’s software;
Recover from an operating system failure; and
Recover from a data center power failure.
The DMV completed the modified operational assessment in
December 1988. Although the assessment resulted in the DMV
selecting Tandem’s NonStop SQL for its new database management
system, it should be noted that neither Tandem nor IBM passed every
key operational test. IBM passed three of the ten key operational tests
and Tandem passed nine. The test that both vendors failed was the test
of the system’s response time while operating in the simulated
“transitional architecture.” The transitional architecture is the system
that would allow the old database and the new database to operate
simultaneously while the new system is gradually phased into
operation. This and other technical features are discussed in more
detail in Appendix C.
Although Tandem passed most of the tests, some tests were
unrealistic given the DMV’s actual requirements. For example, the
DMV established 1.5 seconds as the performance standard for the
operational tests of the system’s response time per transaction. If the
system took longer than this to process the transaction, it was
considered to have failed that test. However, this standard was not
12
established based on the DMV’s actual requirements, but was
established only for comparing the two systems. The DMV’s current
system normally processes transactions in between .50 and .60 seconds
and rarely exceeds 1.0 seconds. The DMV did not evaluate whether it
could tolerate or compensate for an increased response time, nor did it
evaluate how much of an increase in response time it could tolerate
without changing its business processes.
In May 1989, the DMV’s technical staff prepared an issue paper
estimating the possible effects of the increased response time on the
DMV’s computer system. If the new system took 1.5 seconds to
respond to each transaction, the result would be a loss of productivity
for both the DMV’s data entry operators and the DMV’s external users.
If the external users tried to compensate for the loss of productivity by
processing more transactions simultaneously, this could result in the
system’s queue capacity becoming full, causing the system to be
unavailable to other system users. In addition, the system would need
more computer capacity to accommodate the longer queues of
transactions waiting to be processed.
Thus, by early 1989, the DMV was aware that it needed to resolve
significant technical problems related to the system’s response time and
its transitional architecture before it could be assured of the project’s
success. At the end of fiscal year 1988-89, the DMV had spent
approximately $2.8 million on the project.
The Operational Assessment Was Not
Independently Certified
During our review of the DBR consulting contracts, we noted that,
contrary to the DMV’s assertion, the operational assessment was not
“independently” certified as fair to the participating vendors. The
DMV contracted with Codd and Date Consulting Group (Codd and
Date) to audit the operational assessment for fairness to the two
vendors. One of the products that Codd and Date provided to the
DMV as a result of this contract was a letter certifying the equity of the
operational assessment to the vendors involved. The DMV considered
this letter to be an “independent” certification of the operational
assessment. However, since the DMV also contracted with Codd and
Date to participate in the development of the operational assessment,
assist in evaluating the potential vendors, and conduct and evaluate the
results of the operational assessment, it raises questions concerning
Codd and Date’s ability to provide an unbiased certification of the
operational assessment. According to one of Codd and Date’s former
13
contractors, the certification was independent because Codd and Date
subcontracted with another vendor to develop and evaluate the test
results. However, since the DMV’s contract was with Codd and Date,
they would have been ultimately responsible for any work the
subcontractor performed.
The DMV Failed The DMV failed to develop the planned working model of the project
To Develop a that would have tested the new system’s high-risk design features.
Instead, the DMV continued its efforts to implement the project despite
Working Model
the unresolved technical problems noted during the operational
of the Project
assessment and during its attempts to develop the working model. The
DMV spent approximately $8.3 million to develop the system and
incurred obligations of $6.5 million to purchase computers, for a total
cost of $14.8 million, during the time it was attempting to develop the
working model. The DMV had sufficient information in June 1990 to
indicate that the project’s feasibility was in doubt, but rather than
reevaluate the project’s feasibility, it proceeded with activities to fully
implement the project.
In the project’s feasibility study reports and other documents, the DMV
stated that it would design and implement a working model of the DBR
system. At various times, the DMV and the OIT referred to the
working model either as a prototype or a pilot. In the system
development process that the DMV planned to follow, the working
model was considered the key step in determining whether the project,
as designed, would be successful. The stated objectives of the DMV’s
working model included the following:
Implementing an environment at the data center that would allow
the DMV to develop and test DBR software programs;
Implementing the transitional architecture;
Implementing the pilot database;
Implementing a pilot set of new on-line driver’s license
transactions; and
Confirming estimates of subsequent development phases.
In particular, the working model would test the new system’s
high-risk features, including the following: (These technical
features are discussed in detail in Appendix C.)
14
The use of computer-aided software engineering (CASE) tools to
rewrite the existing software programs;
The increased response time noted in the operational assessment;
The gradual movement of data from the old system to the new
system, which necessitated the transitional architecture; and
The DMV’s requirement that the new system not affect the way in
which the system’s users conducted business. This requirement is
known as “transparency.”
In September 1988, the DMV stated in the project’s second feasibility
study report that “portions of the prototype address high risk
components of the system architecture. These components must be
tested early in the prototype phase to ensure success of the Drivers
License/Vehicle Registration Redevelopment project. Working tests
are required for all high risk components to provide proof that a
satisfactory solution exists.” In addition, early prototyping would
reduce the risk of failure, allow adequate time for modification or
termination before a major commitment of resources, and provide the
DMV with actual data to use in estimating the savings to be achieved
after the system was fully implemented.
In September 1990, the DMV reported that it had successfully
completed the working model and that it was ready to begin full
implementation. However, this working model had not successfully
accomplished its objectives and did not test the new system’s high-risk
design features. For example, the DMV had not successfully
determined that the CASE tools could rewrite the existing software
programs nor had it obtained actual data to use for its estimates of
savings. In addition, the DMV was unable to reasonably estimate the
amount of computer equipment needed to fully implement the system.
CASE Tools Failed
The DMV intended to use CASE tools on the DBR project to help
transfer the software programs to the new system. In June 1990, the
DMV terminated its contract with a consulting firm that was
unsuccessful in its attempts to use selected CASE tools to transfer the
existing software programs. The firm was not successful in this
endeavor at least partially because the DMV and Tandem had decided
to upgrade the Tandem NonStop database management operating
systems, and one of the selected tools was not compatible with the
15
upgraded version of the operating system. Further, the company that
supplied that tool was unable to adapt it to the upgraded Tandem
systems. Despite this setback, the DMV reported, in September 1990,
that the new system would achieve cost savings of approximately
$8.9 million per year by fiscal year 1998-99, and it claimed that one
component of the cost savings was to result from the benefits of the
CASE tools, which would provide greater efficiency in maintaining the
new programs compared to maintaining the old programs. This
component of the savings estimate was based on a 1981 textbook on
software engineering and a previous DMV pilot study on the benefit of
using a different CASE tool than the one that the DMV had selected for
the DBR project. The DMV did not base the savings estimate on its
actual experience attempting to use CASE tools on the DBR project,
which had, by September 1990, been unsuccessful. Finally, the DMV
reported in August 1991, that the CASE tools that it had selected for
the DBR project had failed.
Estimates of Personnel Savings Not
Based on Working Model Results
Another component of the DMV’s savings estimate related to savings
in personnel costs in the Headquarters Operations Division resulting
from changes to the driver’s license database. The DMV estimated
that it could eliminate 30 existing positions and avoid the projected
addition of 20 positions by permitting easier modifications to the
driver’s license file and enhanced processing of information requested
from the driver’s license file. The DMV did not base this estimate on
benefits demonstrated by a working model, but based it on the
assumption that these tasks would be made more efficient during the
development of the DBR project.
Computer Equipment Needs
Were Grossly Understated
The DMV’s estimate of the amount of computer equipment needed for
the fully implemented DBR was grossly inaccurate. In October 1990,
the DMV’s technical staff reestimated computer requirements to
support implementation of the driver’s license system only. The
analysis indicated that eight Tandem computers would be needed for
the driver’s license system. The OIT approved the purchase
of six of the computers with the understanding that two more may
be needed later. In December 1990, six Tandem computers, costing
$11.9 million, were acquired to implement the driver’s license system,
16
in addition to the $6.5 million in Tandem computers to develop the
working model of the DBR. Since no detailed design work was
performed on vehicle registration, computer requirements to support
this portion of the project were not reestimated beyond the original
figures included in the operational assessment. In August 1993,
Tandem and EDS prepared an estimate of the computer equipment
required to support the entire DBR project. They determined that 32
equivalent computers would be required to support the driver’s license
system, compared to the original estimate of 6 to 8 computers—an
increase by a factor of at least 4.0. About half of this increase was for
equipment needed to meet the continuous availability requirement.
Other Key Objectives Were Not Met
In addition to the problems with the CASE tools and the
underestimation of the computer equipment, the DMV did not meet
other key objectives of the working model before it progressed to fully
implementing the system. For example, in September 1990, the data
center had raised concerns with the DMV about the Tandem
computer’s ability to operate 24 hours per day, 7 days per week.
Moreover, the DMV had been unsuccessful at designing the transitional
architecture needed to gradually move data from the old system to the
new system. In October 1992, the DMV reported that the original
design of the transitional architecture was not usable because it was too
cumbersome. Furthermore, the DMV did not provide us with any
evidence that it had resolved the problems associated with the increased
response time identified in the operational assessment.
The DMV In June 1990, after the DMV had spent approximately $8.3 million to
Inappropriately develop the system and incurred obligations of $6.5 million to purchase
computers, for a total cost of $14.8 million on the project, the DMV
Proceeded With
proceeded with activities to fully implement the driver’s license
Activities To
system, despite the significant technical problems that remained
Fully Implement
unresolved. In September 1990, the DMV reported to the OIT that it
the Driver’s had completed a pilot of the driver’s license system and requested
License approval to begin implementing the system. Although the reported
Component of the pilot successfully transferred 10 percent of the driver’s license records
to the Tandem system, it was not a working model because it did not
DBR Project
have software programs that could access, manipulate, or change the
data in the Tandem system, nor did it test the feasibility of the
transitional architecture needed to gradually move data from the old
system to the new.
17
Despite the tremendous effort directed at the DBR project, the DMV
was unable to redevelop the software programs that support the DMV’s
primary responsibilities of licensing drivers and registering vehicles.
In addition, it was unsuccessful in designing the transitional
architecture. The DMV spent an additional $34.6 million after it
began to fully implement the driver’s license system, bringing the total
project costs to approximately $49.4 million. Had the DMV prudently
managed the DBR project, it should have stopped to reevaluate the
project’s feasibility in June 1990 in response to significant evidence
that the project, as designed, would be infeasible.
The DMV Also Not only did the DMV fail to achieve the planned developmental
Failed To Heed objectives that would have identified problems, but it also failed to
include independent quality control as part of the DBR project. Also,
Concerns
the DMV allowed the project to continue despite significant problems
Raised During
noted during periodic reviews performed by both independent and
Quality Control
internal parties. In May 1990, the DMV hired a consultant, Computer
Reviews Deductions, Incorporated (CDI) to provide technical assistance and
periodic project reviews. However, CDI could not provide
independent quality assurance because the DMV also contracted with
this contractor to perform project planning and developmental work on
the project. Therefore, the contractor was performing reviews on a
project that consisted, in part, of its own work. Further, according to
the former chief of EDP Services, the DMV established no formal
quality assurance function to be performed from the beginning of the
project at a detailed level by an independent, qualified source who
could raise questions concerning the work performed by either the state
personnel or contractors during the course of the project. Instead of
an independent quality assurance function, the DMV relied on periodic
project reviews and prepared special project reports that were
distributed to the OIT. The DMV conducted internal status meetings
to review project status and to resolve issues. Together, the DMV
expected these elements to provide sufficient quality assurance.
Quality assurance is a function performed on systems development
projects to ensure an independent and impartial assessment of the
project’s methods and techniques and of the work products produced
during the course of systems development. Among its objectives,
quality assurance is expected to assess the project’s adherence to
commitments made by the DMV and to estimate the likely outcome of
the project. Often, suggestions are made to resolve actual or
anticipated problems before they become insurmountable. In essence,
the persons performing quality assurance might be considered
analogous to building inspectors who keep an independent watch over
18
the construction process to ensure it conforms with preestablished
performance standards.
As a result of its periodic project reviews, CDI reported on a number of
occasions that the project was experiencing problems. For example,
from June 1990 to June 1991, CDI reported that the project had the
following problems:
It was on an incorrect course;
It lacked leadership;
It had made no progress on main tasks;
It lacked Tandem-experienced staff because neither the DMV nor
contractor staff had much Tandem experience; and
The use of CASE tools continued to be a major risk factor.
In June 1991, CDI recommended a project reassessment because it was
seriously concerned about the direction of the project. This report
specifically stated that CDI was concerned about the dependence on the
transitional architecture, the lack of knowledge about the current
system and the resulting dependence on CASE tools, the requirement to
jump from 1960s technology to 1990s technology in one step, and the
feasibility of the requirement to maintain user transparency.
In addition to the CDI reviews, the Department of Finance’s Program
Evaluation Unit (PEU) conducted a compliance review of the DBR
project in 1990. The PEU provided its final report on this review to
the DMV in March 1991. During this review, the PEU found that the
DMV intended to place the system into production without completing
a pilot to estimate the costs and benefits of the project. Instead, the
DMV estimated costs and benefits based on industry studies.
Moreover, the PEU found the project was not proceeding on schedule
because the DMV’s schedules were overly optimistic. Specifically,
the PEU’s final report stated, “the DMV had intended, until very
recently, to implement the pilot consisting of a full database conversion
and 70 percent of the driver’s license on-line and batch applications.
Twenty-three to 25 million records were to be loaded in June 1990 for
testing. The entire driver’s license database has about 30 million
records. This means the DMV planned to use most (80 percent) of the
database for the pilot. According to Tandem representatives, the pilot
should be large enough to produce ad hoc reports, production reports
and data, and other database functions to demonstrate and then
19
project, potential benefits and advantages of the system. They also
stated that a much smaller pilot (20 percent of the database) could be
run using proprietary modeling tools developed for Tandem systems to
project systemwide performance. However, running a smaller pilot
did not fit with the project schedule developed by DMV and Tandem
representatives. They expected the Tandem Cyclone (production
machine) to be installed and operational before May 1990. However,
it was not installed because the OIT did not agree that production level
processors (Tandem Cyclone) were needed to run the pilot.
Installation of these processors before receiving and analyzing the pilot
demonstration results could burden the State with a production system
which may not be the best solution.”
The PEU also found that the project was seriously delayed because,
among other things, the DMV’s efforts to use various CASE tools were
unsuccessful, the development of the transitional architecture was
delayed, and the development of the conversion procedures and
programs was not completed on time. Although this report indicated
serious problems with the project, the DMV took no action in response
to the report and continued its implementation efforts.
In addition to the foregoing critical reviews, the project’s management
provided periodic assessment reports to DMV’s management.
Beginning in 1991, these reports indicated that the project was
experiencing significant difficulties. In an August 1991 report, the
project’s management documented many of the problems, including the
following:
A lack of knowledge of Tandem systems among DMV staff;
Difficulties in designing the Tandem system to accommodate batch
processing, one of DMV’s critical functions;
Failure of the selected CASE tool; and
Difficulty recruiting staff experienced in Tandem systems at both
the DMV and the data center.
As a result of these problems, the DMV increased the estimated
development costs of the project from $31.4 million to $38.3 million
and extended the estimated completion date by six months. However,
the DMV continued to project that significant cost savings would be
achieved once it implemented the project. In addition, the DMV’s
management should have resolved problems indicated in its internal
20
project assessment and reevaluated the project’s feasibility before
continuing to devote resources to the project.
The DMV prepared another report in October 1992 that also indicated
the DMV was experiencing major problems with the project. At this
time, the DMV increased the estimated development costs from
$38.3 million to $57.3 million and extended the estimated completion
date from July 1995 to December 1998. Among its reasons for the
increases, the DMV reported it had underestimated the scope of the
project, the system architecture design took much longer than planned
to complete, and the development process proved to be more complex
than expected. Specifically, the DMV stated, “the experience gained
in redeveloping the driver’s license database has given the DMV a
sobering insight into the dimensions of the tasks that lie ahead. The
effort proved to be larger, more complex, and much harder than
originally conceived.” Again, the DMV’s management should have
resolved the problems indicated in this second report and reevaluated
the project’s feasibility before continuing to devote resources to the
project.
According to the DMV’s former chief of EDP Service, the project’s
management felt that it could eventually overcome the technical
problems. Therefore, the DMV’s management saw no reason to halt
the project.
Additional Costs The $34.6 million in additional costs related to the DMV’s decision to
implement the project occurred in four areas. The DMV spent
Incurred in
$5.7 million in contractor fees; $17.4 million for data center costs,
Attempted
which include the purchase of computer equipment; $10 million in
Implementation
DMV staff costs; and $1.5 million for other project expenses.
From July 1990 to December 1993, the DMV spent approximately
$5.7 million on five consulting services contracts with four contractors
and on technical services obtained through master services agreements.
Three of the four contractors provided the same type of services.
These contractor services included the following:
Providing technical project leadership;
Designing software development, including the use of CASE tools;
Assisting in the development of the transitional architecture; and
21
Assisting in designing, testing, and converting the software and
databases to the new system.
The fourth contractor provided training and support on a CASE tool
purchased from the CASE contractor.
Of the $17.4 million in data center costs, $11.9 million was for six
Tandem Cyclones and related equipment, and the remaining
$5.5 million was for other data center costs, such as computer
equipment maintenance, software obtained from companies other than
Tandem, and data center administration costs. The purpose of the six
computers was to provide the computer capacity to accomplish the first
stage of implementation, which was the redevelopment of the driver’s
license database. This purchase brought the total computer investment
to $18.4 million. The DMV purchased these computers at the end of
1990, and the data center installed them in January 1991.
Approximately $3.9 million of the $11.9 million in equipment
purchases represents obligated payments for the Cyclones during fiscal
years 1994-95 and 1995-96.
From July 1990 to June 1994, the DMV spent approximately
$10 million of its own staff time attempting to implement the DBR
project. In this attempt, its staff and contractors engaged in the
following:
Developing a business model to document the DMV’s business
requirements;
Documenting and analyzing the current system’s data and software
programs;
Attempting to design a workable transitional architecture;
Attempting to redevelop the software programs for the new system;
and
Copying the driver’s license database and transferring this copy to
the new computer.
In addition, the DMV’s staff spent time “normalizing” and “scrubbing”
the database. By August 1990, the DMV was aware that the new
system could not tolerate the number of errors and irregularities that the
old database system contained. As a result, the DMV would have to
clean up the driver’s license database in preparation for its transfer to
the new system. This cleanup entailed the following:
22
Identifying obsolete, redundant, and multiple-use data fields;
Verifying that the data in each record was correct;
Verifying that the data items in each record were logical when
compared with the other data items; and
Identifying and correcting millions of errors and inconsistencies in
the database.
In addition, the DMV’s staff developed software programs that would
perform this data scrub automatically. Although the DMV knew of
this requirement by August 1990, it never evaluated the amount of
effort required to accomplish the cleanup. Therefore, these activities
consumed more time and resources than the DMV had estimated they
would.
After the DMV scrubbed the driver’s license database and transferred it
to the new system, it developed a software program to access the
information in the new database. This program performs an on-line
address search that identifies an individual’s name and driver’s license
number from the individual’s address. Law enforcement agencies are
the primary users of this service. This program is not one that supports
any of the DMV’s primary responsibilities, such as licensing drivers
and registering vehicles. In addition, the DMV developed a program
to update the Tandem database periodically to keep the information
current with the old database.
Despite the major investment in the project, the DMV still has not
successfully developed a transitional architecture or any of the software
programs that support the DMV’s basic responsibilities of licensing
drivers and registering automobiles. At the time the DMV decided to
terminate the project, it had spent approximately $49.4 million.
In April 1994, the DMV estimated that approximately $12.7 million of
the DBR project costs resulted in assets with continuing value to the
DMV. However, many of the particular items included in the
$12.7 million estimate should have been accomplished before the DBR
project was attempted or are in actuality part of the sunk costs of the
project. Further, although the DMV claims that $12.7 million of the
total DBR costs of $49.4 million represents assets of continuing value,
it is not currently possible to determine the future value to the DMV for
these costs that were incurred.
23
Conclusion The DMV continued its efforts to more fully implement the project,
despite continued failures in the critical aspects of its development,
and, as a result, spent an additional $34.6 million on a project that
ultimately failed. The DMV should have evaluated the size and
complexity of the DBR project, as well as the difficulty of
implementing it, during the working model stage the DMV established
in the original development plan. Because the DMV failed to
complete the working model, it was unable to determine the project’s
technical feasibility and was unable to reasonably estimate the project’s
costs and benefits. Without a determination of the project’s technical
feasibility, the DMV could not be assured that the system would
perform its required functions. This lack of assurance increased the
risk the project would fail. Without a reasonable estimate of the
project’s costs and benefits, the DMV could not be assured that the
system was economically beneficial and that the project could be
successfully implemented within the approved budget.
Because the DMV failed to complete the working model, ignored the
failures that occurred in its developmental efforts, and failed to heed
the concerns raised in internal and external project reviews, it missed
every opportunity to stop and reassess the project and develop a more
realistic concept of its dimensions.
Recommendation To ensure that the DMV does not unnecessarily incur costs of this
s magnitude in the future, the DMV should complete the milestones it
establishes for the development of new EDP contracts. The DMV
should also conduct sufficient testing and analysis to determine that
proposed EDP projects will achieve their projected benefits, both
monetary and programmatic. Finally, the DMV should ensure that
technical problems identified during quality assurance reviews of EDP
projects are resolved before continuing to devote resources to those
projects.
24
Blank page inserted for reproduction purposes only
25
Chapter 2 The Office of Information Technology
Should Not Have Recommended Continued
Funding for the Database Redevelopment
Project When the Department of Motor
Vehicles Did Not Follow the Approved
Policies To Minimize Financial Risk
Chapter The Office of Information Technology (OIT) continued to recommend
Summary approval of additional funding for the Department of Motor Vehicles’
(DMV) Database Redevelopment (DBR) project despite the fact that
the DMV had not followed approved policies to minimize financial risk
to the State. The OIT has the statutory responsibility to approve
proposed expenditures for electronic data processing projects only if
the policies and procedures it has published in the State Administrative
Manual have been met. The OIT allowed the DMV to attempt to put
the DBR project into operation even though it knew the DMV had not
successfully developed a working model of the project as OIT had
previously required before the DMV could proceed.
Furthermore, the project was allowed to continue because the OIT
continued to recommend approval for additional funding even when it
received a report that the DMV was not developing the system
consistent with requirements from the OIT and consistent with the
policies and procedures published in the State Administrative Manual.
When the OIT became aware the DMV had begun to fully implement
the project in June 1990, the DMV had spent approximately
$14.8 million. This amount consisted of $6.8 million that the OIT had
authorized to develop a working model and $3 million in resources that
the DMV had improperly redirected from other funding sources and
had not reported in its special project reports to the OIT. In addition, it
included $5 million still owed on $6.5 million of computer equipment
obligations that the OIT had authorized but would be included in
subsequent years’ budgets.
In November 1990, the OIT should not have recommended continued
funding when the DMV requested an additional $3.9 million for fiscal
year 1991-92. Ultimately, as of June 30, 1994, the DMV spent an
additional $39.6 million. This amount consisted of $25.7 million of
budgeted funding that had been authorized to complete the working
model and to fully implement the system; $10 million in resources
26
redirected from other DMV funding sources that had been preapproved
by the OIT; and $3.9 million in computer equipment obligations that
the OIT had authorized but would be included in subsequent fiscal
years’ budgets.
The OIT Is The OIT was established in 1983 as a division within the Department
Responsible for of Finance. According to the Government Code, Section 11700, the
purpose of this office is to identify new applications for information
Reviewing the
technology, to improve productivity and service to state agencies, and
State’s Information
to assist state agencies in designing and implementing the use of
Technology
information technology. Also, according to Section 11731 of the code,
Acquisitions the OIT must adopt policies and guidance to carry out the budgeting
and control of expenditures for electronic data processing. This code
section also states that the OIT must approve proposed expenditures for
electronic data processing only if established policies and procedures,
which have been published and maintained in the State Administrative
Manual, have been met and followed. In addition, Section 11711
states that the OIT’s director must develop plans and policies to support
and promote the use of innovative information technologies within state
government as a means of saving money, increasing worker
productivity, improving state services to the public, and demonstrating
effective management tools.
According to the current deputy director, the OIT serves as an
investment committee to determine whether proposed information
technology projects would be a reasonable investment of the State’s
resources. This determination is based on a conceptual and high-level
technical review of how the technology addresses each department’s
business problems and needs. The OIT reviews the technical viability
of a proposed project by determining that the characteristics of the
technology are consistent with the needs of the department. The OIT
performs its oversight role through their review of information included
in submitted feasibility study reports, special project reports, and other
required reports; through on-site discussions with departmental staff to
verify information contained in the reports; on-site discussions with
vendors, consultants, and staff of other agencies that have knowledge
of the technology to be employed; and, on occasion, reviews
by outside consultants hired by the OIT. Also, the OIT’s role as the
investment committee includes making recommendations for budget
augmentations for information technology projects to the Department
of Finance’s Budget Units. These recommendations, once approved
by the Budget Units, the governor, and the Legislature, allow
departments to obtain the funding necessary to complete their
information technology projects.
27
The OIT’s The mechanism for initially approving information technology projects
Review Process is the feasibility study report. The OIT reviews feasibility study
reports to provide an analytical basis for whether the proposed project
should be approved, disapproved, or approved on a modified basis.
During the project life cycle, special project reports are also required to
be submitted to the OIT if total project costs or anticipated project costs
deviate 10 percent or more (higher or lower) from the original
estimates, the project schedule falls behind or is anticipated to fall
behind 10 percent or more, the program benefits deviate or are
anticipated to deviate by 10 percent or more, or a major change occurs
in the project requirements or methodology. The OIT must approve all
special project reports if the OIT has previously approved the
project feasibility study report. Until this kind of report is approved,
departments can only continue work on the project as previously
approved either in a previous feasibility study report or previous special
project report.
The OIT approved the first feasibility study report for the
DBR project in December 1987. The approval recommended the
DMV receive development costs for the project of
$1.5 million for fiscal year 1988-89 and $1.9 million for fiscal year
1989-90. The DMV submitted a second feasibility study report to the
OIT in November 1988. This report was approved by the OIT in
December 1988. Included in the approval were the requirements that
the DMV not proceed with full implementation until after the OIT had
approved the DMV’s completion of the objectives included in the first
feasibility study report and a working model of the project. Also, the
OIT recommended expenditures of an additional $3.4 million for fiscal
year 1989-90 and subsequently recommended $3.7 million for fiscal
year 1990-91. In total, the OIT authorized $10.5 million to develop
the project through the working model stage.
The OIT Should In February 1989, the DMV submitted a letter to the OIT stating that it
Not Have had accomplished the objectives of the first feasibility study report.
These objectives were to confirm database design, to confirm that the
Accepted the
database system would work, to complete necessary design tasks, to
Operational
estimate computer resources, to confirm the feasibility of the planned
Assessment as
transitional architecture, and to simulate the performance of
Satisfying the high-volume driver’s license and vehicle registration programs. The
Requirements of DMV was to accomplish these objectives by defining the technical
DMV’s First requirements of the database, developing a prototype transitional
architecture, and developing a database design. Also, the DMV was to
Feasibility Study
test the prototype to determine if performance issues and technical
Report
requirements were resolved, and it was to finalize the implementation
28
plan. As previously discussed in Chapter 1, we found the DMV had
not designed the operational assessment to meet the objectives of the
first feasibility study report but instead designed the operational
assessment to evaluate and recommend a database management system
for the DMV to use in redeveloping its databases. Nevertheless, the
OIT’s former chief of Statewide Oversight accepted the DMV’s
operational assessment as meeting these objectives.
During our fieldwork, we found that the OIT was unable to provide any
documentation to prove it had reviewed the operational assessment.
We subsequently contacted the former chief of Statewide Oversight
who accepted the operational assessment as meeting the requirements
of the first feasibility study report. He stated that the DMV’s
operational assessment was a prototype that was limited to determining
if relational database technology could be used to meet DMV’s needs.
Further, the next task to be performed, a working model of the project,
would have validated the results obtained from the prototype and would
have provided information for determining if this type of technology
would be cost-effective in modernizing the DMV’s databases.
Moreover, he stated that, at the time, he believed the DMV’s
operational assessment showed that relational database technology
could be used to meet the DMV’s needs.
Thus, because the OIT accepted the DMV’s operational assessment as
satisfying the requirements included in the DMV’s first feasibility
study report, the DMV was allowed to proceed with the project even
though the DMV failed to successfully complete three of the report’s
primary objectives. These objectives included confirming the database
design, completing necessary design tasks, and confirming the
feasibility of the transitional architecture. For example, the database
design and feasibility of the transitional architecture were not
confirmed because they were subsequently found to be unusable. As
previously discussed in Chapter 1, an October 1992 report issued by the
DMV concluded that the original design of the transitional architecture
was unusable because it was too cumbersome, and the design and
development process proved to be more complex than expected.
Therefore, if the three objectives of confirming the database design,
completing necessary design tasks, and confirming the feasibility of the
transitional architecture had been fully tested during the DMV’s
operational assessment, the DMV would have identified these key
system deficiencies and not continued with a full implementation of the
project. As a result of the OIT’s acceptance of the operational
assessment, the DMV’s spending was not limited to the $3.4 million
29
that was approved in its first feasibility study report. Instead, they
were allowed to spend the funds approved in the second feasibility
study report of $8 million.
The OIT Failed The approval of the DMV’s second feasibility study report in
To Require December 1988, also required the DMV to complete a working model
of the driver’s license database. The objectives of the working model
Implementation
included implementing an environment at the Stephen P. Teale Data
of a Working
Center (data center) to allow the DMV to develop and test DBR
Model of the
software programs, implement the transitional architecture, and confirm
Project estimates for subsequent development phases. These objectives were
to be accomplished by executing a working model for the driver’s
license database, including a set of on-line driver’s license transactions
to be processed using the working model. OIT’s approval of the
second feasibility study report specifically required the implementation
and evaluation of the working model using parallel production data.
The working model also would have provided information pertaining to
the objectives of the DMV’s first feasibility study report that were
never addressed in the operational assessment.
During fall 1990, the DMV submitted a special project report. In it,
the DMV requested approval for fully implementing the project, as
opposed to completing a working model of the project. The OIT
approved this special project report, even though the OIT was aware
the DMV had already begun major work on the full production system
and had not completed the working model before the submission of this
special project report. For example, in June 1990, a letter was sent
from the OIT’s director to the DMV’s director that stated that the OIT
had become aware that the actual project was significantly different
from the documented and approved project, and that the DMV had
failed to accomplish the working model and had begun work on the full
production system. It also stated that the working model was approved
to allow the DMV, with minimum risk and expense, to confirm that the
expected system benefits could be captured and to verify whether or not
the project was worth continuing. Further, it stated that the DMV had
spent two years, and more than $5 million on the project; and that OIT
could not continue to support an effort of this magnitude in the absence
of information indicating that the State would receive an adequate
return on its investment. Finally, it required the DMV to prepare a
special project report to accurately update the project status and
propose a cost-effective continuation for OIT to use as its basis for
continued support in the upcoming budget cycle.
30
The OIT subsequently received a special project report in September
1990 that was not approved until November 1990. This approval
allowed the DMV to receive additional funding for fiscal year 1991-92.
We also found that as of June 1990, the DMV had spent approximately
$14.8 million as opposed to the $5 million indicated in the OIT’s letter
to DMV. The $14.8 million consisted of $6.8 million that the OIT had
approved to develop a working model, and $3 million of resources that
the DMV had redirected from other projects and had not reported in its
special project reports to the OIT. In addition, it included $5 million
still owed on $6.5 million of computer equipment obligations that the
OIT had authorized but would be included in subsequent years’
budgets.
In September 1990, the OIT received a draft copy of an Electronic Data
Processing Compliance Review conducted by the Department of
Finance’s Program Evaluation Unit at the OIT’s request. This review
found that the DMV was moving ahead with the project without
completing the required working model, underreporting actual costs,
over budget on the project and not proceeding according to
agreed-upon implementation schedules, and was not preparing project
reports in accordance with the State Administrative Manual or OIT
requirements. The review also found that the activities performed
during DMV’s operational assessment did not demonstrate the
expected performance and benefits of the proposed system based on the
focus of the testing. The OIT could not provide any documentation
that any action was taken in response to this report.
We discussed the compliance review with the former chief of Statewide
Oversight. He stated that he remembered this report and that it
identified quite a few problems with the project. However, OIT staff
took no action as a result of this report.
We also spoke with the former acting director of OIT, who approved
the November 1990 special project report. He stated that he did not
see the draft compliance report at all before the final report was issued.
In addition, he stated he did not follow up on the final audit report
because, at the time it was issued, the prior director of the OIT was in
the process of being reinstated. Also, he was not aware it had been
received, as it was given directly to the line staff working on the DMV
oversight, and was never shown to him. Further, this individual stated
that he allowed the project to continue in November 1990 because of its
great potential value, the lack of objective evidence that it was failing,
and the opinion of technical OIT and Budget staff that it should be
funded for another year.
31
Thus, because the OIT approved the fall 1990 special project report, the
DMV was allowed to proceed with the project without completing a
working model, which again would have identified significant technical
and performance issues, confirmed the costs and benefits associated
with the project, and provided assurance of the project’s success.
Also, for fiscal years 1990-91 through 1993-94, the DMV
ultimately spent approximately $39.6 million attempting to fully
implement the system, including $5 million for computer
equipment purchases obligated before fiscal year 1990-91. This
amount consisted of approved budgeted funding for the project during
fiscal years 1990-91 through 1993-94 of approximately $25.7 million.
The budgeted funding for these fiscal years included $3.7 million to
complete the working model during fiscal year 1990-91; and
$7.6 million during fiscal year 1991-92, $7.2 million during fiscal year
1992-93, and $7.2 million during fiscal year 1993-94 on
implementation. Also, the DMV expended $10 million in resources
redirected from other DMV funding sources, and incurred $3.9 million
in computer equipment obligations that the OIT had approved but that
would be included in subsequent fiscal years’ budgets.
Conclusion The OIT continued to recommend approval of additional funding for
the DMV DBR project despite the fact that the DMV had not followed
approved policies to minimize financial risk to the State. Specifically,
the OIT allowed the DMV to attempt to put the DBR into operation,
even though it knew the DMV had not successfully developed a
working model of the project as the OIT had previously required before
the DMV could proceed. Furthermore, the project was allowed to
continue because the OIT continued to recommend approval for
additional funding even when it received a report that the DMV was
not developing the system consistent with the requirements from the
OIT and consistent with the policies and procedures published in the
State Administrative Manual. When the OIT became aware that the
DMV had begun to fully implement the project in June 1990, the DMV
had spent approximately $14.8 million. This amount consisted of
$6.8 million that the OIT had authorized to develop a working model
and $3 million in resources that the DMV had improperly redirected
from other funding sources and had not reported in its special project
reports to the OIT. In addition, it included $5 million still owed on
$6.5 million of computer equipment obligations that the OIT had
authorized but would be included in subsequent years’ budgets. If the
OIT had exercised its authority and not approved the DMV’s proposed
expenditures subsequent to June 1990, when it found the DMV was not
adhering to the OIT’s requirements and established policies and
32
procedures, the DMV would have been unable to continue its efforts to
fully implement the project and expend the additional $39.6 million
during fiscal years 1990-91 through 1993-94.
Recommendations To ensure that departments are following approved policies to
minimize financial risk, the OIT should exercise its authority to
approve proposed expenditures for electronic data processing only if
established policies and procedures, which have been published and
maintained in the State Administrative Manual, have been met and
followed. Also, the OIT should ensure that departments accomplish
the objectives and requirements included in approved feasibility study
reports and special project reports before allowing projects to move
forward.
33
Chapter 3 The DMV’s True Costs of the DBR Project
Are Higher Than Originally Reported
Chapter The Department of Motor Vehicles (DMV) did not use a cost-reporting
Summary system to monitor expenditures related to the Database Redevelopment
(DBR) project. As a result, it could not identify all of the expenditures
related to the DBR project. Specifically, we identified that the DMV’s
actual and obligated costs for the DBR project were at least
$49.4 million, which is $5.1 million more than the $44.3 million in
project costs that the DMV originally reported to the Legislature and
the Department of Finance. In addition, we determined that the DMV
did not receive preapproval to increase spending above the level
previously authorized by the Department of Finance.
The DMV Did The DMV did not use a formal cost-reporting system to monitor
Not Accurately expenditures related to the DBR project. According to the DMV’s
controller, the DMV implemented a cost-accounting system for the
Monitor Costs for
entire department in 1981 and linked it with its existing accounting
the DBR Project
system. Shortly thereafter, the DMV decided to discontinue the use of
this cost-accounting system. The DMV made this decision primarily
because it determined the system became cumbersome because of the
additional time and resources the divisions spent in preparing and
processing the necessary data required to use the system. In addition,
the DMV also determined that the benefits of the cost-reporting system
were not significant enough to justify its use.
Since the DMV does not currently have a departmental cost-reporting
system, it delegated the responsibility for monitoring project costs to its
divisions. In the case of the DBR project, the Electronic Data
Processing (EDP) division manually kept track of project expenditures
for the purpose of reporting this information to the Department of
Finance. EDP division staff prepared expenditure schedules based on
supporting documentation, such as contracts, purchase orders, and
invoices. However, the division did not establish uniform criteria to
identify those costs to be reported as DBR project
expenditures. As a result, the DMV did not accurately monitor
expenditures related to the DBR project. Specifically, we identified
several errors in the summary of project expenditures supporting the
1994 report to the Office of Information Technology (OIT). We also
identified other project expenditures that the DMV omitted from the
schedule.
34
The DMV presented a summary of project expenditures (cost
summary) in its special project report dated April 1994. The cost
summary presented actual expenditures through December 1993 and
projected expenditures between January and June 1994, all totaling
$39.5 million. In addition, the report also specifies that the DMV has
additional obligations of $4.8 million to pay during fiscal years
1994-95 and 1995-96 to the Stephen P. Teale Data Center (data center)
on the installment purchase of the Tandem Cyclone system.
Therefore, according to the special project report, the total actual and
obligated DBR project expenditures were $44.3 million. Based on our
review of the DMV’s supporting documentation, we identified an
additional $5.1 million that the DMV did not report on the cost
summary. Table 1 summarizes the total expenditures for the DBR
project and those we identified as missing from the DMV’s schedule.
Table 1 Reported, Unreported, and Total
Expenditures for the DBR Project
(In Thousands)
Reported
Expenditures
Per 1994
Special Unreported Total
Project Report Expenditures Expenditures
DMV Staff $10,210 $2,400 $12,610
Consultants 6,690 1,571 8,261
Other DMV Costs 2,972 0 2,972
Data Center Costs
Through June 30, 1994 19,649 1,986 21,635
After June 30, 1994 4,768 (857) 3,911
Total $44,289 $5,100 $49,389
DMV Staff Expenditures
We identified that total personnel expenditures from fiscal year
1987-88 through fiscal year 1993-94 were approximately $12.6 million,
instead of the $10.2 million reported in the cost summary. The
difference of $2.4 million is due to the method the DMV used to
compile these costs. To calculate the total staff expenditures for the
project, the DMV combined the total annual salaries of employees in
the EDP division who worked on the DBR project. However, this
methodology is based on two key assumptions: employees identified
worked 100 percent of their time on the project and EDP staff were the
only employees working on the project. According to the DMV’s
35
controller, the DMV does not have a labor cost system in place to
monitor the number of hours employees spend working on different
projects. Therefore, to assist us in calculating the actual costs of staff
resources used in the DBR project since its inception, the
administrative staff prepared schedules identifying all the employees
who worked on the DBR project as well as the percentage of the year
that they committed to the project. We reviewed these schedules and
determined that there were several employees that were not included in
the cost summary.
Consultant Expenditures
We identified nearly $1.6 million of consultant expenditures that the
DMV did not report in the cost summary. The EDP division staff did
not maintain complete and accurate schedules supporting consultant
expenditures for fiscal years 1991-92 through 1993-94. For example,
approximately $1.1 million of the $1.6 million in unreported costs
relate to consultant expenditures for two contractors during fiscal year
1991-92. The EDP staff had incorrectly reported these expenditures
on the cost summary. We also found that staff were unable to provide
us with schedules that support consulting expenditures incurred before
fiscal year 1991-92.
Data Center Expenditures
The DMV entered into interagency agreements with the data center for
the purchase of hardware and software related to the DBR project on
behalf of the DMV. These purchases included the acquisition of the
CLX/VLX system and the Cyclone system from Tandem Computers,
Inc. As displayed in Table 1, the data center incurred a total of
$21.6 million in expenditures on the DBR project through fiscal year
1993-94.
Based on the data center’s detailed cost reports, we identified additional
expenditures of approximately $2 million, which the DMV did not
include in the cost summary because the data center was not charging
the DMV for all project costs. This amount is comprised of two main
components. According to the manager of the fiscal and business
operations branch at the data center, the data center and the DMV
negotiated a fixed rate that the DMV would pay for the purchase of the
CLX/VLX system. The data center established this rate using an
estimated purchase price for the system of approximately $4.8 million.
The data center’s final negotiated contract with Tandem set the price of
the system at approximately $6.5 million. However, the data center
did not increase the fixed rate that the DMV paid. Therefore, the data
36
center underbilled the DMV by $1.7 million during the period from
May 1989 through April 1994. In addition, the data center did not bill
the DMV for telecommunication costs totaling approximately $247,000
during the period from May 1989 to June 1991. As a result, the DMV
was not aware of its actual costs that the data center incurred.
Accordingly, it appears that the data center recovers the difference in
costs incurred and billed through monthly service rates that it bills to all
client agencies.
In addition, the DMV identified an additional $4.8 million in obligated
expenditures during fiscal years 1994-95 and 1995-96 for the purchase
of the Cyclone system. The DMV is obligated to make monthly
payments through January 1996 for this purchase. However,
according to the Tandem contract, payments for the period from July
1994 through January 1996 add up to only $3.9 million. Therefore,
the DMV overstated its future obligated expenditures by approximately
$900,000.
The DMV Did Not The DMV did not receive preapproval to increase expenditures for the
Receive Preapproval DBR project above the level previously authorized by the Department
of Finance. As described in Chapter 2, a state agency engaging in an
To Increase
information technology project which is subject to approval and
Expenditures Above
oversight by the OIT, is required to submit a special project report
the Level Previously
under certain circumstances. One of these circumstances is when the
Authorized by the total information technology project costs deviate or are anticipated to
Department of deviate by 10 percent (higher or lower) from the estimated information
technology project budget. Section 4819.35 of the State
Finance
Administrative Manual states that if the agency is required to submit a
special project report, the agency shall not expend additional funds to
implement the proposed change until the OIT approves the special
project report.
During fiscal years 1988-89 and 1989-90, the OIT authorized the
amount of expenditures for the DBR project through its review of the
annual budget change proposals and its approval of the feasibility study
reports. According to these documents, the OIT authorized the DMV
to spend up to $1.5 million during fiscal year 1988-89 and up to an
additional $5.3 million during fiscal year 1989-90. However,
according to Appendix D, the DMV reported that it spent
approximately $1.7 million during fiscal year 1988-89 and
approximately $6.3 million during fiscal year 1989-90. In each year,
the amount that the DMV spent exceeded the amount that OIT
authorized by more than
37
10 percent. However, the DMV did not submit a special project report
during either year requesting preapproval to increase expenditures
above the level previously authorized by the OIT.
In 1990, the DMV submitted a special project report that the OIT
approved. The OIT prepared a schedule of actual and projected
expenditures for the DBR project in order to determine the total amount
of expenditures that it would authorize the DMV to spend on the DBR
project. According to this schedule, the OIT authorized the DMV to
spend up to $77.3 million between fiscal years 1990-91 and 1993-94.
However, according to the current deputy director of the OIT, the OIT
cannot identify the amounts in the special project report that are used to
determine the authorized costs. As a result, we are unable to
determine whether the DMV exceeded the project costs that the OIT
authorized for fiscal years 1990-91 through 1993-94.
Conclusion We determined that the DMV’s true costs for the DBR project are
$5.1 million higher than originally reported because the DMV did not
correctly identify all of its project expenditures. Based on our review
of agency documentation, we determined that total actual and obligated
project expenditures are at least $49.4 million. If the DMV had used
its project cost-reporting system, it could have more accurately
identified all project expenditures. In addition, we determined that the
DMV did not receive preapproval to increase spending above the level
previously authorized by the Department of Finance.
Recommendation The DMV should implement a project cost-reporting system for EDP
s projects in order to report accurate expenditures on its special project
reports. In addition, the DMV should obtain approval from the
Department of Finance if it expects to deviate by more than 10 percent
from the level previously approved for an information technology
project.
38
Blank page inserted for reproduction purposes only
39
Chapter 4 The Department of Motor Vehicles
Violated Numerous Contracting
Laws and Regulations
Chapter The Department of Motor Vehicles (DMV) violated numerous state
Summary laws and regulations related to the execution of purchase orders and
contracts when acquiring goods and services for the Database
Redevelopment (DBR) project. Specifically, the DMV falsified a
$46,000 purchase order to pay for services for which it otherwise could
not have paid. In addition, it entered into sole-source contracts
without adequate justification, modified the terms of a contract without
formally amending it or receiving the Department of General Services’
(DGS) approval, and allowed work to be performed before contracts
were approved. It also failed to obtain performance bonds on some
electronic data processing (EDP) contracts.
The DMV The DMV falsified a purchase order for approximately $46,000 to pay
Falsified a for services that Tandem Computers, Inc. (Tandem) had provided
before the DMV awarded the company a contract and for services that
Purchase Order
Tandem provided during the contract period, but for which funds had
To Pay for
lapsed under the contract to pay for the services.
Services Provided
Before a Contract In June 1989, the DMV contracted with Ernst & Young to provide
Was Executed expert assistance to redevelop the DMV’s databases. The DMV and
Ernst & Young mutually agreed to terminate the contract effective
June 22, 1990. Ernst & Young had several subcontractors assisting on
the DBR project. The DMV wanted to retain the services of the
subcontractors after the Ernst & Young contract was terminated.
Under the terms of the termination agreement, the subcontractors were
to be transferred to Tandem. The subcontractors continued their work
after the termination of Ernst & Young while the DMV attempted to
amend a maintenance contract between the Stephen P. Teale Data
Center (data center) and Tandem to include the subcontractors.
However, on July 13, 1990, the DGS disapproved the DMV’s
sole-source request on the basis that it was inappropriate to add
consulting services to a maintenance contract.
Therefore, the DMV entered into a contract, effective July 1, 1990,
with Tandem to obtain the consulting services of the subcontractors.
The contract was approved by the DGS in November 1990. According
40
to the terms of the contract, Tandem was to provide services from
July 1, 1990, to June 30, 1991. The DMV subsequently extended the
contract to September 30, 1991. Over one year later, in
September 1992, Tandem submitted two invoices requesting payment
for consulting services rendered under the contract. However, one of
the invoices, for approximately $28,000, was for services performed
between June 22, 1990, the termination of the Ernst & Young contract,
and July 1, 1990, the start of the Tandem contract. The second
invoice, for approximately $18,000, was for services provided in April
and June 1991. At the time Tandem submitted the invoices, sufficient
funds were not available under the contract to pay for these services.
Since the DMV could not pay for these services, together totaling
approximately $46,000, from funds encumbered under the contract, it
needed to find another source of funds if it was to pay the invoices.
Rather than requesting Tandem to submit a claim to the Board of
Control, or further amending the contract, the DMV prepared a false
purchase order that provided sufficient funds to pay Tandem.
The DMV eventually paid Tandem for these services by falsifying a
purchase order for software totaling approximately $46,000. In
October 1992, the manager of EDP General Administration approved a
request for materials document for the purchase of two packages of
Tandem software. Upon the approval of the request for materials
document, an information systems analyst within the DMV’s EDP
Contracts Administration unit prepared and approved a purchase order.
After the approval of the purchase order, the DMV processed an
invoice requesting payment for two copies of modeling software,
matching the description on the falsified purchase order. According to
the manager, the DMV did not receive two copies of the modeling
software; however, it paid the invoice in December 1992.
According to the EDP Contract Administrator, the DMV paid Tandem
in this manner because it would be highly unlikely that DGS would
approve a contract amendment so long after the contract had
terminated. As a result of this falsified purchase order, the DMV paid
approximately $46,000 that it may not have been legally required to
pay.
The California Penal Code, Section 424, states that each officer of this
State charged with the disbursement of public moneys who, without
authority of law, appropriates those moneys to the use of another is
punishable by imprisonment in the State prison for two to four years.
Additionally, the Government Code, Section 6203, states that officers
authorized by law to make or give any certificate or other writing are
guilty of a misdemeanor if they make and deliver as true any certificate
41
or writing containing statements that the officers know to be false. In
addition, the State Administrative Manual, Section 1247, requires that
amendments or modifications to contracts be approved by the DGS if
the original contract required DGS approval.
The DMV Did The DMV did not always adequately justify its use of sole-source
Not Meet Formal consulting contracts for the DBR project. Specifically, of the four
sole-source consulting contracts into which the DMV entered for the
State
DBR project, the DMV did not adequately justify three, entered into
Requirements for
from November 1987 to July 1990 and totaling $2.8 million.
Sole-Source
Additionally, the DGS approved the contracts even though the DMV
Contracts
had not provided adequate justification. The Public Contract Code,
Section 12102(a), requires that EDP goods and services be acquired
through competitive means, unless the director of the DGS determines
that the goods or services are the only ones that can meet the State’s
need or the immediate acquisition of the goods or services is necessary
for the protection of the public health, welfare, or safety. This code
section also states that the director of the DGS must approve the
acquisition method to be used. The State Administrative Manual,
Section 5209, states that the DGS will approve sole-source EDP
procurements only when the agency can adequately document the fact
that the goods and services are the only ones that can meet the State’s
needs or that the immediate acquisition is necessary for the protection
of the public health, welfare, or safety. This section cites several
examples of situations that meet the sole-source criteria. One example
is if the agency conducts a survey of the marketplace that shows that
only a single vendor can provide the service.
During the course of the DBR project, the DMV entered into four
sole-source consulting contracts. We reviewed the DMV’s
justification for each of these sole-source procurements and found that,
for three of the contracts, the DMV failed to adequately justify the need
for the sole-source acquisition. These three contracts were with
Arthur Andersen, Codd and Date Consulting Group, Inc. (Codd and
Date), and Tandem. For example, the DMV awarded a contract and
subsequent amendments that ultimately reached a total of
approximately $419,000, to Codd and Date. The DMV awarded Codd
and Date the initial contract, totaling $125,000, in June 1988, to
participate in the development of the operational assessment plan, assist
in evaluating the potential vendors, audit the operational assessment for
fairness between the two competitors, and review the DMV’s prototype
plans. The DMV justified the sole-source procurement based on time
constraints. Specifically, the DMV stated that it urgently needed to
make progress on the design of the new database to maintain the
42
project schedule. Although other sections of the State Administrative
Manual allow the DGS to approve sole-source contracts if the director
of DGS determines that it is in the State’s best interest to do so, neither
the Public Contract Code nor the State Administrative Manual allow
this same discretion for sole-source EDP contracts. The State
Administrative Manual exclusively specifies the situations that meet
the criteria for approval of sole-source EDP contracts, and a time
constraint is not one of these situations, outside of a threat to the public
health, welfare, or safety. In November 1988, the DMV increased the
amount of this contract by approximately $228,000 to conduct the
operational assessment and to continue participation in the planning
and auditing for fairness between the two competitors for which it had
already contracted. The DMV again justified this sole-source
procurement based on time constraints. The DMV subsequently
increased this sole-source contract to a total of approximately
$419,000. Codd and Date subcontracted some its work under this
contract to Arthur Andersen, demonstrating that Codd and Date was not
the only contractor that could provide these services.
As a result of inappropriately awarding these contracts on a sole-source
basis, the DMV and the DGS unnecessarily curtailed competition for
state contracts. Such competition is designed to ensure the highest
quality of services at the lowest possible price.
The DMV Allowed The DMV entered into ten contracts associated with the DBR project.
Work To Be In four of these contracts, beginning between November 1987 and
November 1991, the DMV allowed the contractors to begin work
Performed Before
before DGS approved the contract. In fact, the contractors began work
DGS Approved
on these unapproved contracts as much as five months before the DMV
Contracts
obtained the approval, and the DMV paid all the contractors for the
work performed before contract approval once the contracts were
approved.
The State Administrative Manual, Section 1209, requires each agency
to submit each contract to DGS in enough time to permit DGS to
review and comment on it before work commences. Additionally, the
State Administrative Manual, Section 1284, states that work under any
consulting contract may not be commenced, and no payment may be
made, before the contract is approved by DGS, or the DMV, if the
contract is exempt from DGS’ approval. The State Administrative
Manual, Section 1209, also states that, should the contractor begin
work in advance of receiving notice the contract is approved, the work
may be considered as having been done at the contractor’s risk, as if the
contractor were a mere volunteer, and the contractor may go unpaid.
43
By failing to obtain contract approval before contract work began, the
DMV exposed the State to potential monetary liability for work
performed when the contract was not approved.
The DMV Failed The Public Contract Code, Section 12112, identifies the requirements
To Obtain that a department must satisfy to make progress payments on EDP
contracts for goods and services that are manufactured or performed
Performance
especially for the State and not suitable for sale to others. Specifically,
Bonds on Some
the law requires that a department may make progress payments for
EDP Contracts
work performed under this type of contract if two conditions are met.
First, a department must withhold 10 percent of the contract price
until final delivery and acceptance of the goods or services, and second,
the contractor must submit a “faithful performance” bond in a sum not
less than one-half of the total amount payable under the contract. The
performance bond secures the contractor’s faithful performance of the
contract.
During the course of the DBR project, the DGS approved and the DMV
entered into seven contracts with progress payments for EDP goods or
services that were to be manufactured or performed specifically for the
State and would not be suitable for sale to others. The DMV obtained
performance bonds for only one of these seven contracts. For
example, the DMV entered into a contract with Computer Deductions,
Incorporated, in the amount of $1.8 million, for technical support and
assistance from Tandem-knowledgeable experts. The contract was for
EDP services performed specifically for the State and also provided for
progress payments; therefore, the DMV was required by law to obtain a
performance bond from the contractor. However, the DMV did not
obtain a performance bond from this contractor although the contract
allowed for, and the DMV made, progress payments for the contracted
services. According to the DGS’ Office of Legal Services, if an EDP
contract is for goods or services that are to be manufactured or
performed specifically for the State and not suitable for sale to others
and provides for progress payments, then a performance bond is
required. However, the DGS approved these contracts even though the
DMV failed to obtain performance bonds.
As a result of failing to obtain performance bonds on contracts that
allowed for progress payments, the DMV increased the risk that the
State would make payments for services that ultimately were either not
delivered or not satisfactory. In this situation, legal action would be
the only recourse available to the State to attempt recovery of the
payments.
44
The DMV Modified During our review of the contracts connected with the DBR project, we
a Contract Without found that the DMV authorized Ernst & Young to begin work on a
proposed task that was not part of the contractor’s approved contract.
Obtaining DGS’
The approved contract required Ernst & Young to evaluate the DMV’s
Approval
working model prototype and write a report on the validity of the
prototype, and the results of the prototype test, defining
design issues, high-risk areas, low-risk areas, and providing appropriate
recommendations. The estimated cost of this task was approximately
$93,000. This task, which was not done, was to evaluate a product
required by the Office of Information Technology (OIT) as a basis for
OIT’s approval of the first feasibility study report. The purpose of the
substituted task was to develop and execute the pilot project, and
Ernst & Young estimated the cost of completing the task would be
approximately $503,000. This task related to a requirement of the
second feasibility study report. Although the DMV never obtained the
DGS’ approval for this modification, the DMV paid Ernst & Young
approximately $202,000 for the time spent on this task when the
contract was terminated.
The State Administrative Manual, Section 1215, states that all contracts
of more than $15,000 are subject to approval by DGS, including
contract amendments, except a first amendment that only extends the
time for completion of the contract for a period of one year or less.
The Department We reviewed all the deliverables related to the DMV’s contract with
Had the Ernst & Young because of the unusual circumstances surrounding the
contract. The fact that the contract was terminated before it was
Authority To
completed raised concerns about the status of the deliverables at the
Mutually Agree
time of final payment on the contract. In June 1990, the DMV and
To Terminate Its
Ernst & Young mutually agreed to terminate its contract, under which
Contract With the contractor was to provide expert assistance to the DMV on the DBR
Ernst & Young project. According to the Legislative Counsel, the DMV had the
authority to enter into a termination agreement with Ernst & Young.
Additionally, the termination agreement could allow the DMV to pay
for partially completed tasks. Therefore, the DMV did not violate any
state laws or regulations when it mutually agreed with Ernst & Young
to terminate its contract with the contractor and pay for incomplete
tasks.
The DMV contracted with Ernst & Young in June 1989 to provide
expert assistance to redevelop the DMV’s databases. Specifically,
among other tasks, the contractor was to assist in the project planning,
evaluate the DBR prototype, develop the application development
environment, and finalize the database design. The contract, totaling
45
approximately $5.5 million, was scheduled to end on June 30, 1994.
Under the terms of the contract, Ernst & Young was required to provide
a performance bond for 50 percent of the contract amount.
On July 2, 1990, the DGS approved the termination of the contract
effective June 22, 1990. According to DMV’s former chief of EDP,
the DMV and Ernst & Young mutually agreed to terminate the
contract. Terms of the termination agreement released both parties
from any liability and required the DMV to pay the contractor a final
settlement of approximately $822,000. The settlement was based on
an analysis performed by the DMV related to the amount of work the
contractor had performed to date. Including the settlement, the DMV
paid Ernst & Young a total of $1.5 million. However, Ernst & Young
had submitted invoices for services totaling approximately $2 million
to the DMV through May 1990.
Both the DMV and Ernst & Young had concerns about the contract that
ultimately resulted in the mutual agreement to terminate the contract.
A primary concern to both was that the Ernst & Young contract was
predicated on the availability of three integrated computer aided
software engineering (CASE) tools. These tools were never fully
developed by Ernst & Young to operate in an integrated manner for the
Tandem environment. However, certain changes in system software
by the DMV and Tandem affected the development of the CASE tools
by Ernst & Young and another third-party software company included
in Ernst & Young’s proposal, raising the question of who was at fault
for the tools never being fully developed. The DMV settled the
dispute by paying the contractor $370,000 for this task.
Conclusion The DMV falsified a purchase order for approximately $46,000 to pay
Tandem for services that could not be paid from an existing contract.
Also, the DMV did not always adequately document its justification for
the use of sole-source contracts and allowed contractors to begin work
before contracts were approved. Additionally, the DMV and the data
center modified contracts without obtaining the DGS’ approval.
Finally, we found that the DMV properly exercised its authority when
it mutually agreed to terminate its contract with Ernst & Young.
The DGS approved three sole-source contracts even though the DMV
did not provide adequate documentation. The DGS also approved six
contracts that required performance bonds even though the DMV failed
to obtain the bonds.
46
Recommendation The DMV should follow all contracting laws and regulations to ensure
s that it protects the State’s financial and legal interests and that it obtains
the highest quality of services at the lowest possible price.
Additionally, the DMV should consider taking disciplinary action
against the employees involved in falsifying the purchase order.
47
We conducted this review under the authority vested in the state auditor
by Section 8543 et seq. of the California Government Code and
according to generally accepted governmental auditing standards. We
limited our review to those areas specified in the audit scope of this
report.
Respectfully submitted,
KURT R. SJOBERG
State Auditor
Date: August 17, 1994
Staff: Philip Jelicich, Audit Principal, CPA
Steve Cummins, CPA
Star Castro
James Gabler
Alison Hanks, CPA
Jerry Lewis
Linus Li, CPA
Kelley Rogers
48
Appendix A
Chronology of Key Events Related to
the Database Redevelopment Project
November 2, 1987 DMV submitted to the Office of Estimated project costs: $29,209,908
Information Technology (OIT) the
Estimated completion date: July 1, 1993
feasibility study report for Phase I
of the Database Redevelopment
Fiscal year 1987-88 $ 672,289
(DBR) Project.
Fiscal years 1988-89 - 1992-93 28,537,619
$ 29,209,908
December 22, 1987 OIT approved the feasibility study OIT approved DMV to proceed with Phase I of the
report for DBR - Phase I. DBR only. The approved cost for Phase I was
$3.4 million.
November 12, 1987 DMV entered into a sole-source The purpose of the contract was to assist DMV
contract with Arthur Andersen & with the planning phase of the DBR.
Co.
Contract amount: $470,555
Scheduled termination date: June 30, 1989
June 29, 1988 DMV entered into a sole-source The purpose of the contract was to provide
contract with Codd & Date independent, expert assistance during the planning
Consulting Group. phase of the operational assessment.
Contract amount: $125,000
Scheduled termination date: June 30, 1989
June 30, 1988 DMV entered into amendment #1 to The amendment increased the contract amount by
the contract between DMV and $172,564, to a total of $643,119, and added four
Arthur Andersen. tasks to the contract.
November 1, 1988 DMV entered into amendment #1 to The amendment increased the amount of the
the contract between Codd & Date contract by $227,550, to a total of $352,550, and
and DMV. added the task of assisting DMV in the operational
assessment of IBM and Tandem.
November 8, 1988 DMV submitted to the OIT the Estimates costs of $13,240,793 to complete
feasibility study report for Phase II driver’s license pilot by August 1990.
of the DBR.
December 15, 1988 OIT approved the feasibility study OIT approved the continuation of DBR into
report for DBR - Phase II. Phase II, but instructed DMV not to proceed with
Phase II until the OIT approved the Phase I
technological demonstration evaluation.
49
December 23, 1988 DMV released the results of the As a result of the operational assessment, the
operational assessment. DMV selected Tandem’s NonStop SQL as its
database management system.
March 15, 1989 DMV entered into amendment #2 to The amendment increased the amount of the
the contract between Codd & Date contract by $66,000, to a total of $418,550, and
and DMV. added the task of reviewing the proposals received
in response to a second request for information.
April 6, 1989 OIT approved the Phase I OIT accepted the operational assessment as
technological demonstration meeting the OIT’s requirement for a technological
evaluation. demonstration as stated in the December 15, 1988
approval of the Phase II feasibility study report.
May 16, 1989 Teale Data Center (data center) Cost of the system is $6.5 million.
initiated the purchase order for the
Tandem VLX+CLX application
development systems.
June 13, 1989 DMV entered into a contract with The purpose of the contract was to provide
Arthur Young (later Ernst & independent, expert assistance to redevelop the
Young). driver’s license, vehicle registration, and
occupational license databases.
Amount of contract: $5,455,154
Scheduled termination date: June 30, 1994
Procurement process: RFP DMV-8030
October 16, 1989 DMV entered into amendment #1 to The amendment changed the name of the
the contract between Arthur Young contractor from Arthur Young to Ernst and Young.
and DMV.
October 20, 1989 DMV submitted a major project Project schedule was delayed four months because
report to the OIT. of the bid protest.
December 4, 1989 OIT returned the October major The OIT stated that the “project report ... fails to
project report. provide the specific information needed to
evaluate the status of the project.”
April 6, 1990 DMV submitted the special project Project schedule delayed for two reasons:
report for DBR-Phase II to the OIT. an unanticipated request for information process
conducted by DGS at the conclusion of the
operational assessment and the protest of the
contract award to Ernst and Young.
Estimated completion date: January 1995
50
May 7, 1990 DMV entered into a contract with The purpose of the contract was to provide the
Computer Deductions, Inc. independent, expert assistance of a single,
senior-level individual. The contract was exempt
from DGS review under DMV’s delegated
authority.
Contract amount: $296,809
Scheduled termination date: June 30, 1992
Procurement process: RFP DMV-88-165
June 6, 1990 OIT rejected the April special OIT returned the April 1990 special project report
project report. to DMV citing the following problems:
(1) the special project report did not accurately
represent the current project status;
(2) the report did not demonstrate that the project
would produce economic benefits
commensurate with the costs;
(3) the actual project is significantly different
from the documented and approved pilot
project; and
(4) DMV failed to accomplish the pilot, yet
began major work on the full production
system.
June 14, 1990 DMV entered into contract By mutual agreement, effective June 22, 1990, the
amendment #2 with Ernst and amendment terminates the contract, lowers the
Young. contract amount to $1,500,000, and transfers
certain Ernst and Young subcontractors to Tandem
so they can continue to work on the project.
July 1, 1990 DMV entered into a sole-source The purpose of the contract was to provide
contract with Tandem. technical support services including technical
project leadership; design and construction of the
application development environment; application
design and development; and technical support in
designing, testing, and converting the applications
and databases.
Contract amount: $700,000
Scheduled termination date: June 30, 1991
July 18, 1990 DMV submitted a revision of the The OIT did not respond to this special project
April 6, 1990 special project report report in writing.
to the OIT.
51
September 14, 1990 DMV submitted a revision to the This special project report replaces the July 18
July 18, 1990 special project report report. The report states that DMV planned to put
for DBR to the OIT. the relational database technology, running on
Tandem’s NonStop SQL, into production in fiscal
year 1990-91. DMV reported that the driver’s
license pilot was completed in August 1990.
Total project costs to date: $8,433,155
Estimated project costs: $31,418,900
Estimated completion date: January 1995
November 30, 1990 OIT approved the September 1990 OIT’s analysis indicates an estimated net benefit
special project report. of $2,597,000 by 1998-99 and subsequent net
annual savings of $8,888,000. The OIT approved
the purchase of up to six Cyclones.
December 21, 1990 DMV entered into amendment #1 to The amendment increased the contract amount by
the contract with Computer $135,368, to a total of $432,177, and expanded the
Deductions, Inc. role of the consultant in the area of project
planning and development of DBR.
December 27, 1990 The data center submitted a The total cost is $11.9 million.
purchase order for six Cyclones.
The computers were accepted in
February 1991.
January 9, 1991 DMV entered into amendment #1 to The amendment increased the contract amount by
the contract between Tandem and $700,000, to a total of $1,400,000.
DMV.
February 25, 1991 The Department of Finance’s The findings include:
Program Evaluation Unit published
(1) Many of the products developed in Phase I
the final report on the compliance
were not used;
review of the DBR.
(2) The Phase I prototype activities did not
demonstrate the expected performance and
benefits of the proposed new system;
(3) DMV underreported the costs of Phase I;
(4) DMV intends to place the Driver’s License
system into production without completing a
pilot to estimate costs and benefits;
(5) The project is not proceeding on schedule
because of unforeseen project activities;
(6) DMV is spending more than the
OIT-approved spending levels for the project;
and
(7) Project reporting has not been prepared
52
according to State Administrative Manual
standards.
May 15, 1991 DMV entered into amendment #2 to The amendment increased the amount of the
the contract between Tandem and contract by $300,000, to $1,700,000, and extended
DMV. the scheduled termination date to September 30,
1991.
August 9, 1991 DMV submitted a major project DMV reported many of the problems they were
report to the OIT. experiencing with the project.
Estimated completion date: July1995
August 20, 1991 DMV entered into contracts with The purpose of each contract was to provide
the three contractors listed below. technical support services. The procurement
process for all three contracts was RFP
DMV-1003.
Computer Deductions, Inc. Contract amount: $1,802,400
Scheduled termination date: August 20, 1995
Zen Systems, Inc. Contract amount: $754,000
Scheduled termination date: August 20, 1995
Fitech Contract amount: $191,000
Scheduled termination date: August 20, 1995
October 7, 1991 The DMV director sent a letter to This letter states that DMV accepts the TIES
Tandem Integrated Engineering proposal for a review of the DBR project.
Services (TIES).
November 1, 1991, DMV entered into a contract with The purpose of the contract was to provide training
Texas Instruments. for the use of software purchased from the
contractor and use of contractor hardware and
software.
Contract amount: $1,127,079
Scheduled termination date: March 23, 1995
October 14, 1992 DMV submitted a major project Estimated development costs: $57,300,000
report to the OIT.
Estimated completion date: December 31, 1998
January 28, 1993 The DMV director sent a letter to The director accepted the data center’s offer to
the data center. make a proposal regarding the DBR project.
March 17, 1993 Tandem sent a letter to the DMV In this letter, Tandem documents its plan to bring
director regarding Tandem’s in Electronic Data Systems as a project partner and
approach to developing a new DBR to keep the project “within budget limits” and
plan. guarantee deliverables within the defined
implementation period.
March 29, 1993 The DMV director sent a In this memorandum, the director documents the
memorandum to the Secretary of the DMV’s decision to reject the data center’s
53
Business, Transportation and proposal because there was insufficient
Housing Agency. information upon which to base a decision of this
magnitude.
November 1, 1993 Tandem and Electronic Data The estimated cost to implement the new plan is
Systems submitted cost information $185 million.
for the proposed new DBR project
plan.
December 31, 1993 The DMV director informed DMV cited the following reasons for not accepting
Tandem that DMV would not the proposal:
accept its proposal.
Total project costs have increased from
$28.5 million to $185 million ($175 million
proposal ceiling plus $10 million DMV costs).
Project time frames and target dates have
increased from the original five years with
project completion expected in 1993 to ten
years with an estimated completion date in
1997-98.
DMV’s Information Technology planning has
been expanded to a global architectural view
encompassing front-end systems replacement.
DMV’s business requirements have and are
rapidly changing which necessitates initiation
of major business reengineering efforts.
Completion of these prerequisite efforts is not
consistent with the time frames, milestones,
and contractual obligations proposed in the
new DBR project proposal.
April 4, 1994 DMV submitted a special project This report concludes that the DBR failed because
report to the OIT. the decisions that DMV made as a result of the
operational assessment were erroneous. It also
concludes that the revitalization proposal made by
Tandem and Electronic Data Systems did not
support DMV’s total business agenda and that the
proposal’s benefits could not justify its cost.
54
Appendix B
Responsible Officials
Department of Motor Vehicles
Director
Del Pierce February 1986 - January 1991
Frank Zolin March 1991 - present
Chief of Electronic Data Processing Services
Jack Miller December 1985 - February 1988
Don Leachman March 1988 - April 1993
Margie Mullen (Acting) May 1993 - present
Database Redevelopment Project Director
Dennis Walker June 1986 - March 1989
Margie Mullen (Acting) April 1989 - June 1989
Margie Mullen July 1989 - April 1993
Glenn Wilson May 1993 - present
Stephen P. Teale Data Center
Director
David Lema July 1988- June 1989
Jim Wilson (Acting) July 1989 - May 1991
Bob Dell’Agostino May 1991 - September 1991
Chong Ha September 1991 - present
Assistant Director of Information Processing Systems
Don Leachman May 1993 - present
Assistant Director of Enterprise Systems
Tim Wenger February 1993 - present
Department of Finance, Office of Information Technology
Director
Steven Kolodney March 1983 - September 1989
Ronald Kuhnel (Acting) September 1989 - March 1991
Steven Kolodney April 1991 - April 1994
Vacant April 1994 - present
Chief of Statewide Oversight
Tim Wenger January 1985 - February 1993
55
56
Appendix CDescription of Technological Elements of the
Proposed Database Redevelopment System
RelationalA key component of the Database Redevelopment (DBR) project was the use of a
Databaser elational database management system. Based on the Department of Motor Vehicle’s
(DMV) expectation that more and more demands would be placed on the computer
Management
system by outside parties (for example, law enforcement agencies and insurance
System
companies), the DMV chose relational database technology because it allows greater
flexibility in modifying existing computer programs and existing computer files. The DMV
currently uses a nonrelational database to store information. A key difference between
relational and nonrelational databases is the organization of the data. A relational database
management system stores information in tables (rows and columns of data). For example,
information about a specific driver would be stored in several tables with a driver’s name and
address stored in one table, vehicles owned in a separate table, and moving violations in another.
The system matches the driver in the “driver” table to that driver’s traffic violations in the
“traffic violations” table. In a nonrelational database, all related information (for instance, all
information about a particular driver) is stored in one location or “record.” For example, in the
driver’s license database, each record contains specific information about one driver, such as
name, address, driver’s license number, traffic violations, and accidents. This type of database
system is called a “flat file” database.
A relational database allows an item of data, such as a person’s name, to be stored only once,
which reduces inconsistencies in the data. In addition, it allows unlimited, efficient growth in
the database. It also makes it easier and faster to extract specific, limited information from the
database. For example, if a law enforcement agency wanted a report that lists all the drivers
who live at a particular address, the computer would only have to search the address table.
Under the old system, the computer would have to search the entirety of every record to produce
the same report.
However, in a relational database management system, it takes longer to assemble the entire
record for one driver. Those transactions that require the entire record will take much longer to
process under the new system because it must search all the tables in the database to extract the
entire record on one driver. This is significant because under the DMV’s proposed system, the
way in which the end users accessed the data and viewed the information would not have
changed. This concept is known as “transparency.”
TransparencyIn the initial planning phase of the DBR project, the DMV made a decision not to review
or change the end user’s business processes. This decision resulted in the need to
change the system in a way that the system’s users would not be affected by the changes and
would, therefore, not be required to change their business processes. The concept of changing
the system without any noticeable effect on the system’s users is known as “transparency.”
57
The DMV’s databases support the activities of both DMV and external clients, such as law
enforcement agencies and insurance companies. The primary focus of the DBR project was on
improving the storage and retrieval of information necessary to fulfill existing program
requirements, mandated new functions, the DMV’s vision, and the timely delivery of
information to the DMV’s clients. Secondarily, the DBR project was intended to increase the
efficiency of developing and maintaining the computer programs that perform all functions of
the DMV’s computer system. According to the current DMV Manager of Information Systems,
the DMV originally determined that it was not feasible to review or change the business
processes of the system’s internal and external users.
According to the manager, most end users are given all the information about a particular driver
when they make inquiries or update a record. Therefore, under the proposed relational database
system, each time a request for data was made, each table would have to be accessed to compile
all the same information about a driver that an end user was accustomed to seeing. As a result,
the amount of time required to process each transaction would significantly increase.
This transparency concept affected the success of the DBR project because the DMV failed to
analyze the ability of the proposed new technology to process transactions using the existing
business processes developed over the last 25 years. As noted previously, relational databases
are more efficient than the existing flat file databases at relating specific items of information,
but they are less efficient at pulling or searching for an entire record. Because many of the
system users’ business processes called for pulling the entire record and these processes were not
going to be changed, the average processing time per transaction increased in the new system.
CASE ToolsIn addition to improving information storage and retrieval capability, the DMV desired to
update its 25-year-old computer programs and improve the efficiency of developing new
software programs and maintaining existing ones. The DMV planned to accomplish this
objective through the use of computer-aided software engineering (CASE) tools. CASE tools
assist in the development of new software programs. Among other things, CASE tools can be
used to help transition computer programs from one programming language to another.
The DMV anticipated that the use of CASE tools would allow it to more easily and quickly
convert its 25-year-old software programs to the new programming language. Without CASE
tools, the only way to write software programs is by manually rewriting the code in the new
language.
Transitional In the planning phase, the DMV analyzed two alternative plans for implementing the new
Architecturesystem: the phased approach and the slash or cutover approach. In the phased approach,
which was selected by the DMV, the software programs that access, update, and
manipulate the data are translated to the new programming language, transferred to the new
computer system, and implemented in several steps over time. In the slash or cutover approach,
the new system is implemented and run in parallel with the old system until a conversion to the
new system is completed.
The selection of the phased approach created the need to develop additional, temporary computer
systems and programs that would allow both the old and the new computers to communicate
with all computer users. This transitional system, referred to as the “transitional architecture,”
58
between the old IBM system and the new Tandem Computers, Inc. (Tandem) system, two very
different hardware systems, significantly increased the technical problems experienced during
the conversion. This phased implementation approach with transitional computer systems is not
uncommon for system conversions of DMV’s magnitude; however, its use significantly increases
the overall complexity of the conversion.
The DMV determined that the slash or cutover approach represented too great a risk that the new
system would fail and cause the databases to be unavailable. The DMV reasoned that this
approach was unacceptable because of the tremendous detrimental impact this would have on the
DMV’s law enforcement clients and the public. The DMV selected the phased approach
because it believed it to be less risky. The DMV identified two objectives of the phased
approach: to reduce the risk of disrupting services to users and to gradually introduce the new
technology to the DMV’s technical staff.
59
Appendix D
Unreported Expenditures for the DBR Project
(In Thousands)
Fiscal Year
1987-88 1988-89 1989-90 1990-91 1991-92 1992-93 1993-94 1994-95 1995-96 Total
Reported Expenditures Per
1994 Special Project Report
DMV staff $ 770 $ 1,828 $ 1,920 $ 1,984 $ 2,301 $ 1,407 $ 10,210
Consultants $471 557 1,567 1,582 972 827 714 6,690
Other DMV costs 364 1,064 203 756 377 208 2,972
Data center costs
Equipment 970 1,570 3,439 3,439 3,278 $ 2,384 $ 2,384 17,464
Other 876 996 1,684 1,663 1,734 6,953
Total Reported 471 1,691 6,305 6,271 8,835 8,607 7,341 2,384 2,384 44,289
Unreported Expenditures
DMV staff 249 (5) (184) (173) 971 912 630 2,400
Consultants (15) (4) (23) 1,210 62 341 1,571
Other DMV costs
Data center costs
Equipment 217 329 329 330 330 275 86 (943) 953
Other 155 585 374 (285) (340) (313) 176
Total Unreported 249 352 726 507 2,226 964 933 86 (943) 5,100
Actual Expenditures Per
DMV and Data center
Documentation
DMV staff 249 765 1,644 1,747 2,955 3,213 2,037 12,610
Consultants 471 542 1,563 1,559 2,182 889 1,055 8,261
Other DMV costs 364 1,064 203 756 377 208 2,972
Data center costs
Equipment 217 1,299 1,899 3,769 3,769 3,553 2,470 1,441 18,417
Other 155 1,461 1,370 1,399 1,323 1,421 7,129
Total Actual $720 $ 2,043 $ 7,031 $ 6,778 $ 11,061 $ 9,571 $ 8,274 $ 2,470 $1,441 $ 49,389
60
Blank page inserted for reproduction purposes only
61