Most state government links are back online Tuesday afternoon after a hardware malfunction at the State Data Center downed them on Friday.
The outage affected approximately 1,100 servers and took down pages like Gov. Ron DeSantis' homepage, FLGov.com, and the state portal, MyFlorida.com. As of 3 p.m. Tuesday, all but 86 of the servers are back online and all site links are expected to return by the end of the day, said DeSantis spokeswoman Taryn Fenske.
"(The Florida Digital Service) along with the manufacturer continue to take all necessary actions to restore the remaining servers and respond to changing circumstances," Fenske told Florida Politics.
In a Tuesday afternoon email, Chief Information Officer Jamie Grant said the Florida Digital Service was entering the home stretch.
"Since discovery of the issue, Florida Digital Service has deployed an all-hands-on-deck approach and worked around the clock alongside significant resources from the machine's manufacturer in response," Grant wrote.
There was a setback at 3 a.m. Monday when some recovery efforts went offline.
The Florida Digital Service, an agency under the Department of Management Services, began responding to the hardware failure Friday, DMS spokeswoman Rose Hebert told Florida Politics on Monday. The Digital Service has been working with Dell EMC to bring state websites back online.
"The backup processor is designed to take over when the main processor fails, but that routine function did not instantly occur as architected," Hebert said. "FLDS continues to work alongside significant resources deployed by the machine's manufacturer as a part of the maintenance contract to restore functionality and mitigate the impact."
Lawmakers created the Florida Digital Service in 2020 to maintain state data, set up testing environments to demo state software before it's rolled out, and facilitate data sharing between government agencies.
Grant, a former Republican state representative who was instrumental in creating the Digital Services, heads the agency. Emails obtained by Florida Politics showed Grant explaining the root cause of the problem could wait till after the team finishes its priority of restoring operations and functionality.
However, according to one source familiar with the issues plaguing the state system, that "sh*t happens in computing but what CIO Jamie Grant has done since is a head scratcher."
Because of delays in supply chains, getting new hardware quickly is extremely difficult. In lieu of using the cloud to host and run the applications a decision was made to try and fix the same Dell hardware that initially failed the state early Friday which resulted in another crash Saturday morning.
Grant took to Twitter Monday evening after Florida Politics reported that the state websites were down.
"Your source is more than welcome to quit scratching their head long enough to explain to the dedicated technical resources who have been & still are working around the clock that there's an instantaneous, automagical solution to solve for the hardware failures. We're all ears …" he tweeted.
___
Florida Politics reporter Christine Sexton contributed to this story.
No comments:
Post a Comment