VictorOps Portal Unavailable on Web and Mobile - Still Ingesting Alerts & Notifications
Incident Report for VictorOps
Postmortem

We detected the issue due to the Cloudflare outage at 7:49am MDT (UTC -6). The VictorOps Technical Support Team was notified at 7:53am MDT. After verifying the issue and acknowledging Cloudflare’s status we created our StatusPage incident at 8am MDT.

Cloudflare offered resolution at 8:11am MDT and VictorOps became fully operational. We remained in a monitoring state until 9:12am MDT, upon that time we resolved the incident.

Please see the following link to the Cloudflare StatusPage for the 7/2/19 incident: https://www.cloudflarestatus.com/incidents/9wjyx63y2xsy

And please find the most recent explanation from Cloudflare here: https://blog.cloudflare.com/cloudflare-outage/

As always, we welcome and encourage you to reach out to the VictorOps Support team with any questions: victorops-support@splunk.com

Posted about 2 months ago. Jul 02, 2019 - 14:26 MDT

Resolved
We've been actively monitoring the implemented fix and the overall situation at the moment. Based on positive technical observations, we're moving this incident from Monitoring to Resolved.

Thank you for your patience during this time.

Again, we sincerely apologize for any inconvenience this issue may have caused. As a follow-up to this issue, our teams will be conducting a Post Incident Review and will follow-up with our findings on this StatusPage. If you have any immediate questions in reference to this incident, please don't hesitate to contact our Support Team with an email to victorops-support@splunk.com or submit the form on our Contact Support page (https://victorops.com/contact-support/).
Posted about 2 months ago. Jul 02, 2019 - 09:12 MDT
Monitoring
Portal access has been restored.
Posted about 2 months ago. Jul 02, 2019 - 08:11 MDT
Update
We are continuing to work on a fix for this issue.
Posted about 2 months ago. Jul 02, 2019 - 08:02 MDT
Identified
The VictorOps portal is currently unavailable via web and mobile. We are ingesting alerts, processing incidents, and delivering notifications to users as normal. You can ack or resolve incidents by responding to phone call and SMS notifications.

Cloudflare is facing network issues (which is in turn having downstream effects on those who utilize its services, including VictorOps) and is working towards a resolution. Please see https://www.cloudflarestatus.com/

We are actively working to resolve this issue as quickly as possible.

Use the Subscribe to Updates option or follow @VOSupport on Twitter for updates. If you have any immediate questions or concerns, our support team is standing by to respond via email victorops-support@splunk.com or submit the form on our Contact Support page (https://victorops.com/contact-support/).
Posted about 2 months ago. Jul 02, 2019 - 08:00 MDT
This incident affected: Clients (Web client (portal), Android client, iOS client).