All Systems Operational

Updated a few seconds ago

**AS45177 (Devoli) Core Network Status** For unplanned Chorus outages/notifications, please go to http://outages.chorus.co.nz/ Chorus unplanned outages recently resolved please go to http://outages.chorus.co.nz/ and click on "Recently Resolved". For UFF planned and unplanned outages, please review: https://ultrafastfibre.statuspage.io/

Locations

xDSL / UFB / RBI / NBN Handovers

Operational

DNS

Operational

Devoli Ethernet

Operational

Voice

Operational

IP Transit

Operational

IntelliPath Services

Operational

Web Services

Operational

External Services

AWS EC2

AWS S3

Cloudflare

GitHub

New Relic

Salesforce

Slack

Zendesk

0

Active Incidents

3

Days Since Last Incident

0

Upcoming Maintenances

History (Last 7 days)

Southern Cross Planned MaintenancePlanned Maintenance

Description

Event Type: Planned UTC Start Event Date / Time: 2019-07-25 17:00:00 UTC End Event Date / Time: 2019-07-25 19:00:00 Impact Duration: 2 Hours To Network Operations Centre: Work Reference SR#236827_7 (Day 4) Planned Maintenance Software Update on Ciena Platform UTC 25th July 2019 There is an ongoing fault on the node that need to be fixed by a software update on our Ciena platform by our equipment vendor tomorrow. A 2-hour Maintenance Window has been established to enable a number of nodes (SR#236827_6) to be updated as well in the same Maintenance Window. Impact: The nodes will be out of service for a period of up to 25 minutes, and as 4 nodes are being updated, and depending on the routing of individual circuits, there could be up to 4 outages during the Maintenance Window. This activity is programmed in the following Maintenance Window: UTC 1700 hrs to 1900 hrs on Thursday, 25th July 2019 Manager Southern Cross NOC


Components

IP Transit


Locations

Los Angeles


Schedule

August 26, 2019 05:00 - August 26, 2019 07:00 NZST
August 25, 2019 17:00 - August 25, 2019 19:00 UTC


August 26, 2019 07:00 NZST
August 25, 2019 19:00 UTC
[Update] Planned maintenance is completed

August 26, 2019 05:00 NZST
August 25, 2019 17:00 UTC
[Update] Planned maintenance has begun
Scheduled maintenance rollbackService Disruption

Incident Status

Service Disruption


Components

xDSL / UFB / RBI / NBN Handovers, Voice, IP Transit, IntelliPath Services


Locations

Vodafone RBI, UFF New Plymouth NGA, UFF Hamilton NGA, Enable Christchurch NGA, Citylink Wellington, Chorus Christchurch NGA, Chorus Dunedin NGA, Chorus Wellington NGA, Chorus Palmerston North NGA, Chorus Wellington EUBA/HSNS, Chorus Auckland NGA, Auckland, Wellington


Post-Mortem

https://www.dropbox.com/s/rttm154mtm3dbix/%20Devoli%20RFO%20200819%20-%20final%20.pdf?dl=0


August 21, 2019 18:45 NZST
August 21, 2019 06:45 UTC
[Resolved] RFO is avialible in the above dropbox link.

August 20, 2019 10:35 NZST
August 19, 2019 22:35 UTC
[Monitoring] Issue has been resolved and most services restored. Some rollback is occurring with a small number of individual customers may experience a small interruption. We are continuing to closely monitor the situation and will provide a full RFO as soon as possible.

August 20, 2019 10:06 NZST
August 19, 2019 22:06 UTC
[Monitoring] Services are starting to return, we are closely monitoring things.

August 20, 2019 09:11 NZST
August 19, 2019 21:11 UTC
[Identified] Rollback to previous stable software release underway, services will start to recover shortly.

August 20, 2019 08:46 NZST
August 19, 2019 20:46 UTC
[Identified] Network team investigating issue, with engineer onsite at MDR exchange. Further update shortly.

August 20, 2019 08:20 NZST
August 19, 2019 20:20 UTC
[Identified] We are investigating an outage, an engineer is on route to MDR exchange.

August 20, 2019 08:05 NZST
August 19, 2019 20:05 UTC
[Identified] We are aware of the current outage, which is linked to the scheduled maintenance conducted overnight. We are currently rolling back changes. Further updates to follow.
Emergency maintenance work at MDRPlanned Maintenance

Description

To further correct issues from this mornings outage we are doing an emergency restart on a line card at MDR Datacentre - this should have minimum (if any) customer impact as were have moved any direct traffic off the router. Maintenance scheduled for 2.15pm


Components

xDSL / UFB / RBI / NBN Handovers, DNS, Devoli Ethernet, Voice, IP Transit, IntelliPath Services, Web Services


Locations

Vodafone RBI, UFF Wanganui NGA, UFF Tauranga NGA, UFF New Plymouth NGA, UFF Hamilton NGA, Enable Christchurch NGA, Citylink Auckland, Citylink Wellington, Northpower Whangarei NGA, Chorus Dunedin NGA, Chorus Hamilton NGA, Chorus Christchurch NGA, Chorus Greymouth NGA, Chorus Queenstown NGA, Chorus Auckland EUBA/HSNS, Chorus Oamaru NGA, Chorus Nelson NGA, Chorus Paraparaumu NGA, Chorus Invercargill NGA, Chorus Gisborne NGA, Chorus Whakatane NGA, Chorus Rotorua NGA, Chorus Timaru NGA, Chorus Dunedin NGA, Chorus Napier NGA, Chorus New Plymouth NGA, Chorus Wellington NGA, Chorus Taupo NGA, Chorus Masterton NGA, Chorus Palmerston North NGA, Chorus Christchurch EUBA/HSNS, Chorus Wellington EUBA/HSNS, Chorus Auckland NGA, Melbourne, Sydney, Auckland, Broadsoft Cloud PBX, London, San Jose, Los Angeles, Wellington, Intellipath Web Services, Mynxnet Web Services, Wholesale Ordering Portal, Auckland Hosting


Schedule

August 20, 2019 14:15 - August 20, 2019 14:30 NZST
August 20, 2019 02:15 - August 20, 2019 02:30 UTC


August 20, 2019 14:31 NZST
August 20, 2019 02:31 UTC
[Update] Completed.

August 20, 2019 14:31 NZST
August 20, 2019 02:31 UTC
[Update] Maintenance work complete.

August 20, 2019 14:18 NZST
August 20, 2019 02:18 UTC
[Update] Maintenance in progress.