Snapshot

When a major West Coast utility set out to enhance the resiliency of critical public safety applications and the underlying infrastructure supporting their Emergency Operations Center (EOC), they turned to their long-standing partnership with Logic20/20. As a high-value business and technology consulting firm—and a validated AWS Advanced Consulting Partner—we are helping our client achieve their public safety goals through multiple means:

  • Refactoring applications using a modern coding language to enable containerization
  • Running applications in clusters using AWS Elastic Kubernetes Service (EKS)
  • Creating an API data service layer with rapid caching capabilities
  • Implementing a CI/CD pipeline using Infrastructure as Code to increase automation and reduce development cycle time
  • Enabling true Active-Active cloud capabilities with multi-region and on-prem fail-over

 

While the road to highly resilient systems is never ending, our utility client is closer to achieving their goals and actively investing in their commitment to protecting the communities they serve.

Our team of certified AWS developers and architects leveraged the AWS Well Architected Framework and a suite of AWS tools, including:

  • Application load balancers
  • CloudFront
  • CloudWatch
  • Cognito
  • DynamoDB
  • Elastic Kubernetes Service
  • Elasticache
  • Fargate
  • GuardDuty
  • API Gateway
  • Lambda
  • Resiliency Hub
  • Route-53
  • S3
  • SecurityHUB
  • Shield
  • Terraform
  • Web Application Firewall

Navigating the complexities of Public Safety Power Shutoffs

Wildfires and extreme weather are a year-round risk in California, especially during fire season, generally September through December. Our client regularly monitors local fire danger and extreme weather conditions across the state. To help protect customers and communities during extreme weather events, the EOC may decide to shut off electric power to prevent electric systems or downed power lines from becoming the source of an ignition. This is called a Public Safety Power Shutoff (PSPS) activation. Not only do these events have a serious public safety impact, but they are also highly scrutinized by multiple regulatory bodies.

The EOC uses a suite of applications to make the decision to turn power off and back on, and they cannot afford any downtime during a PSPS activation. The availability of these applications is critical for public safety. Below are some examples of the applications’ functionality:

  • Monitor weather and electrical circuit information to help make decisions on PSPS de-energizations during an EOC activation.
  • Display current distribution electrical outages in the service territory.
  • Monitor real-time air quality index to notify field staff when to use the appropriate ventilation mask if the safety threshold is exceeded.
  • Communicate PSPS outage details to utility partners during fire season.
The EOC uses a suite of applications to make the decision to turn power off and back on, and they cannot afford any downtime during a PSPS activation. The availability of these applications is critical for public safety.

Addressing cloud dependency and network resiliency challenges

At the outset of our engagement, the EOC systems had a strict dependency on the uptime of a single region within a single cloud provider. Furthermore, our client relies on exchanges to connect to the cloud. Recently, their data center connection to the cloud failed when an exchange’s own path to the cloud provider failed. A natural disaster such as an earthquake could cause loss of connection as well.

The Active-Active multi-region / on-prem failover approach was born out of these challenges.

Developing the solution

We worked with the utility to deliver significant improvements to their EOC systems reliability and ultimately removed five single points of failure that the EOC systems rely on.

Leveraging the AWS Well Architected Framework, we centered our approach around Kubernetes, the cloud-native de facto standard to deploy and operate containerized applications. This approach, along with the myriad other AWS tools listed above, enables the deployment of software images across cloud providers while remaining serverless, thus avoiding traditional sys-ops patching and upgrading of operating systems.

Developers can now check in code to a single repository, and the pipeline builds, security-scans and deploys the updates to multiple cloud providers.

Using the AWS Resilience Hub, we assess, improve, test, and track the resiliency posture of each application to protect it from disruptions and improve overall business continuity.

Within this ongoing engagement, Logic20/20 has introduced the most up-to-date tools and approaches for software development and architecture. Our team keeps abreast of the rapidly changing ecosystem of cloud solutions and has the expertise and flexibility to change tools mid-project when warranted. We’re also working closely with our client’s in-house developers and architects, helping them upgrade their skillsets to encompass the latest cloud technologies.

We worked with the utility to deliver significant improvements to their EOC systems reliability and ultimately removed 5 single points of failure that the EOC systems rely on.

Partnering for enhanced community safety

Over the years, our client has built a strong reputation on its commitment to preserving the safety of the communities it serves. In working with their emergency operations center to enhance the resiliency of its applications, Logic20/20 is helping them maintain that reputation. Together, we are taking the first steps towards the utility’s goal of maximizing availability for continuous, reliable emergency operations that will serve Californians today and for years to come.

 

Person reading papers in front of laptop screen

Claim your competitive edge

We create powerful custom tools, optimize packaged software, and provide trusted guidance to enable your teams and deliver business value that lasts.

  • RPA
  • Virtual assitants
  • Solution engineering
  • Cloud solutions
  • Enterprise architecure
  • Digital strategy

Like what you see?

Sean Quealy

Sean D. Quealy is a Manager in Logic20/20’s Digital Transformation practice. 

Author