State of California

The following entry is a record in the “Catalogue of Catastrophe” – a list of failed and troubled projects from around the world.


State of California – California, USA
Project type : Payroll and benefits system
Project name : 21st Century Project (MyCalPAYS) 
Date : Feb 2013  Cost :$254M

Synopsis :
As with many large organizations, the state of California has over the years built up a complex interconnected set of IT systems to support their daily operations. Managing multiple systems can however be costly. There is often duplicated data spread across different databases, complex business processes to help bridge between systems and elevated maintenance and training costs. As has been the trend for the past 15 years, the State of California decided to integrate their systems into a single Enterprise Resource Planning (ERP) solution. Merging 13 separate systems into a single system, the 21st Century Project was to improve efficiency and reduce the cost of handling payroll and benefit payments for the state’s 243,000 employees.

The origins of the 21st Century Project date back to 2004 when $70 million was spent in an initial attempt to integrate payroll functions. The contractor responsible for that initial attempt was terminated in 2009 due to a lack of progress and a fresh attempt was made under the title MyCalPAYS and with a new plan to use an SAP based system. Said to be the largest payroll project in the US in 2012, the MyCalPAYS system was deployed in a phase 1 release in June 2012. Deployed to a single organizational unit with just 1,300 staff (managed under 2 union agreements), the system immediately ran into problems. Staff were being paid incorrectly, simple math functions are reported to have been inoperable and incorrect allowances were being handed out. An announcement from the State Controller’s Office noted that “The errors in the SAP system affect everyday lives. Not only have SCO employees been paid too much, or too little, they and their family members also have been denied medical services despite paying for the insurance coverage. Payments to the State’s dental, vision and deferred compensation partners have been incorrect and delivered late. Improper deductions have been taken, payments have been made to the wrong payee, payroll and pensionable wages have been incorrectly calculated, and union deductions incorrectly determined.”

State officials had to undertake significant investigation and recalculations to correct the multiple problems the system was producing and after 8 monthly cycles, the system had failed to produce a payroll that didn’t contain significant deficiencies. Reports indicate that hundreds of separate problems were present in the payroll and even simple functions could not be relied upon. Requesting that the problems be addressed, the State sent a 37 page “cure” letter to SAP in Oct 2012. The failure to overcome the concerns in the letter resulted in State Controller John Chiang terminating the contract with SAP on 8th Feb 2013 and announcing that MyCalPAYS would be deactivated so that payroll functions could revert to the legacy systems. Since then the State of California is reported to have initiated a review process to determine if parts of the system can be salvaged for a third attempt and whether or not litigation is required.

Contributing factors as reported in the press:
Early reports do not yet indicate the cause of the problems. The project appears to have a meaningful governance structure in place and a supplier with significant prior experience. The fact that the payroll system had so many problems and the fact that such projects do generally have access to prior historical data that can be used to verify test results indicates that quality control / testing issues exist. Additional notes will be added should more information become publicly available.

Reference links :

  1. Project termination announcement from the State Controllers Office – 8 Feb 2013
  2. California ends contract with SAP over troubled IT project