INFRA unscheduled hypervisor reboot

[floeff]

Hello,

earlier today, one of our hypervisors rebooted and brought down a couple of services.

Infra is currently working on bringing the services back online again.

Amongst services currently AFFECTED are:

- BugZilla: available, but throws error messages for certain write operations

- the TDF and LibreOffice website

- the help system

- the pad

- the download archive

NOT affected are e.g. Gerrit, Wiki, AskBot, mailing lists, Jitsi, SSO, Redmine and the main download website.

The infra team is available at the #tdf-infra IRC channel on Freenode for questions. The status website at https://status.documentfoundation.org will be updated as soon as there are news.

We apologize for the inconveniences.

Florian

[floeff]

Hello,

a quick update on the situation. Still AFFECTED are:

- the main websites documentfoundation.org and libreoffice.org
- the pad
- OpenGrok
- the downloadarchive
- some Bugzilla tables are affected too, we're trying to reindex these

Infra is working on bringing the services up bit by bit, with the priority on the website, and then will analyse the situation to find out why the hypervisor rebooted itself.

The infra team is available at the #tdf-infra IRC channel on Freenode for questions. The status website at https://status.documentfoundation.org will be updated as soon as there are news.

We apologize for the inconveniences.

Florian

[floeff]

Hello,

another update:

- the main websites documentfoundation.org and libreoffice.org are back ONLINE

- OpenGrok is back ONLINE

- the download archive is back ONLINE

- the pad is still OFFLINE

- Bugzilla is ONLINE; if you spot errors, please let us know

The infra team is available at the #tdf-infra IRC channel on Freenode for questions.

Thanks to Cloph for looking into the issues immediately after they were spottted!

Florian