Author Archives: Eddie Chapman

Loss of network at our Data Centre

12:30PM Tues 5th April – the Data Centre where our equipment is located are currently experienceing a total loss of network across a big part of their facility. If you VPS is currently unreachable, please stand by, we are monitoring updates form the DC, who are working to restore service, and will post an update here as soon as we have an update from them.

Update 12:35PM – we’ve just seen that network service has returned to all of our VPS. You should find your VPS is reachable again now, if it is not, please raise a support ticket so we can look into it.  But please note the DC has yet to confirm that the outage is over, we will post another update once this is confirmed.

Update 12:44PM – unfortunately it appears their problems are not over, as network connectivity has just been lost again at the DC.  Customers will be seeing their VPS’ are unreachable again at this time.

Update 12:53PM – network service has again returned to all of our VPS, you should find your VPS is reachable again now. But please note the DC has yet to confirm that the outage is over, we will post another update once this is confirmed.

Update 13:08PM – We have not seen any further interruptions to service since the last update, and the DC have confirmed that service should now be normal. You should find your VPS is reachable, if it is not, please raise a support ticket so we can look into it.  We’d like to apologise to our customers for the inconvenience caused by this outage. It is likely the DC will communicate to us a reason for this outage once they have completed their investigations. If and when they do, we will communicate that here.

14:20PM – The DC have provided the following update: “We feel we have resolved the immediate issues with the network here we are still considering this investigation as ongoing.  At this time we know an issue with a piece of power infrastructure appears to have led to a routing issue on our network. In turn, this caused network service to be lost across our two data centres.  A root cause analysis will be made available within 7 days.”

We at Manchester VPS will pass on here any further information we get.  Having been with the DC for a long time now we are very confident in their infrastructure, expertise, and excellent service they always provide to us. It is extremely rare for them to suffer a major outage, so we would like to reassure our customers that you are in good hands.

Centos & Ubuntu new releases and ISO images updated

We’re pleased to announce that we’ve updated our Centos and Ubuntu OS ISO collection today, following new releases from both these extremely popular Linux projects.

The Centos 6.6 collection has been updated to the newly released Centos 6.7. More information can be found in the release announcement from 3 days ago.

The Centos 7.0-1406 collection has been updated to the latest rolling ISO media release for May 2015 (1505).  To quote from the Centos project:

“The rolling builds are a point in time snapshot of a given CentOS version including all updates on mirror.centos.org.  This includes all all security, bugfix, enhancement and general updates for CentOS Linux, in this case they include updates up to and including May 28th, 2015.”

We have also expanded our Centos 7 collection to include the Minimal and Net install ISOs. More info about this latest release can be found in the Centos 7 Release 1505 announcement.

The Ubuntu 14.04.2 LTS (Long-Term Support) collection has been updated to the newly released version 14.04.3. More info can be found in the release announcement from 4 days ago.

If you already have an older version of any of these OS installed, you can easily update them, and we urge you to do so as soon as possible.

For Centos, just run the command yum update

For Ubuntu, see Ubuntu’s guide to Trusty updates.

Additionally, as announced recently here, Ubuntu 14.10 (Utopic Unicorn) reached End of Life on July 23, 2015. As such we have deleted it from our ISO collection. If you are still running this version of Ubuntu you should upgrade to 15.04 (Vivid Vervet) as soon as possible so you can continue to release security updates. You can find guidance about doing that here.

If you need help upgrading your OS, we can often perform updates for you at very reasonable hourly rates, just open a support ticket to discuss further.

As always, if there is a particular OS you’d like us to update if you spot there is a newer version available, or if there’s an OS you’d like us to add to the collection that we don’t already have, just let us know, we’ll almost certainly say yes. The full list of OS in the collection can be found here.

What we’ve done to protect against VENOM and Logjam

In the last 7 days the internet community has seen two quite significant (but unrelated) security vulnerabilities made public; affectionately named “VENOM”  (CVE-2015-3456) and “Logjam”.

We’ve taken steps to protect our customers from both.

“VENOM”  (CVE-2015-3456) is a vulnerability affecting the vast majority of Cloud and VPS providers such as ourselves. It is a flaw in the virtualisation software used by most providers, called “qemu” which could potentially allow an administrative user on one VPS to access the memory area of other VPS running on the same physical host. Due to the critical nature of this vulnerability, we applied the fix made available for qemu across all of our physical VPS hosts and, very soon after the vulnerability was made public, all customer VPS were either restarted with a new patched qemu instance or live migrated to a new, patched qemu instance, and thus no longer vulnerable.

“Logjam” is a set of weaknesses identified by security researchers in Diffie-Hellman for TLS, a cryptographic algorithm fundamental to many internet protocols including HTTPS. This research was made public today. The weaknesses allows attackers to downgrade vulnerable TLS connections (including HTTPS) to use 512-bit export-grade cryptography. This allows the attacker to read and modify any data passed over the connection.  We immediately assessed our web servers which rely on HTTPS to encrypt the Client Area and VPS Control Panel, and found that they were not vulnerable to the worst of the weaknesses as we already configured them using a strong set of cipher suites when they were originally deployed, and have hardened the configurations further over the last couple of years as new vulnerabilities have surfaced (such as Heartbleed and Poodle).  As a result we had already disabled export cipher suites, which are vulnerable to Logjam attacks, a long time ago.  However, we found our web servers were vulnerable to one of the weaknesses the Logjam researchers found, which is that most web servers use the same prime numbers for Diffie-Hellman key exchange, making them potentially vulnerable to eavesdropping by a nation state. As a result, we followed their advice and generated a strong, unique Diffie Hellman Group on all our web servers and restarted them, closing the vulnerability.  So our customers can be reassured that their communications with us are extremely secure, as we have always taken the security of our HTTPS protected website areas extremely seriously.

More information about VENOM can be found here, and for information about Logjam see here.

Maintenance Sunday 2nd Nov, 3:00 AM

We will be performing essential maintenance work on our VPS host servers between 3AM – 5AM GMT on Sunday 2nd November. During this window  your VPS will be unavailable for up to 20 mins while the host server is rebooted.

Before rebooting we will be sending each VPS an ACPI Shutdown signal. If you have configured your VPS to respond to this, it will start it’s shutdown procedure, and shutdown cleanly. If you would like help configuring your VPS to respond to this, please raise a support ticket and we will be happy to help.

We will proactively checking that your VPS boots up successfully after reboot. You can help us in this by ensuring that your VPS IP address responds to ICMP ping. If it currently doesn’t, and you would like any assistance, please  raise a support ticket and we will be happy to help.

Updates will be posted here on this blog post during the work.

We’d like to apologies in advance for any inconvenience caused by this essential work.

Update Sun 2nd Nov, 04:55AM – Unfortunately we are running behind schedule, some VPS will have their downtime between 5AM-6AM. Apologies to those affected for the inconvenience this may cause.

Update Sun 2nd Nov, 06:35AM – We have decided to finish the work at 06:20AM and restore normal service, although we still have more VPS hosts to do that we were unable to finish. All VPS which we had to stop are back up again, there should not be any customer without service, so if you find you don’t have service at this point please raise a support ticket. We will re-schedule another maintenance window for the remaining VPS hosts.

Update Sun 2nd Nov, 14:55PM – We have rescheduled the remaining work for tonight, 2AM – 3AM GMT and have informed affected customers by email. This remaining work only affects some customers, who we have emailed today.

Update Mon 3rd Nov, 01:57AM -The remaining work is now complete and any VPS affected are now fully operational again. If you find you don’t have service at this point please raise a support ticket. We’d like to thank all customers for their patience while we completed this essential work.

Some ISO image updates

We’ve updated some ISOs today, the following are the new versions that have replaced the old ones:

 

The Centos 5.10 collection has been updated with the recently released Centos 5.11 equivalents.

Clonezilla Live 2.2.4-12 (amd64)

Clonezilla Live 2.2.4-12 (i686-pae)

KNOPPIX DVD V7.4.2 2014-09-28 EN

System Rescue CD 4.3.1

 

If there is a particular OS you’d like us to update if you spot there is a newer version available, or if there’s an OS you’d like us to add to the collection that we don’t already have, just let us know, we’ll almost certainly say yes. The full list of OS in the collection can be found here.

Network outage Sept 28th

Sun Sept 28th 2014 – 16:50: Unfortunately the data centre we are in has just suffered a brief loss of network connectivity lasting about 5-10 mins. It was very brief so may not have been noticed but our monitoring detected it. We’d like to apologise to any customers affected. At the moment we don’t have any information from the DC as to what the cause was, but we are very pleased to see that they were able to resolve whatever it was so quickly. If we do get any information from them we’ll pass it on here on this post.

Lots of OS ISO images updated

We’re pleased to let everyone know we’ve been going through updating our OS ISO image collection the last couple of days to refresh some of them where newer versions are available. These are the ones we’ve updated:

Ubuntu 12.04.5 Precise Pangolin LTS Server (amd64 & i386)
Ubuntu 14.04.1 Trusty Tahr LTS Server (amd64 & i386)
Debian 6.0.10 Net Install CD (amd64 & i386)
Debian 7.6 DVD Set (amd64 & i386)
System Rescue CD 4.3.0
KNOPPIX V7.4

We’ve also added Desktop versions of Ubuntu 12 and 14, having previously only carried the Server versions, which is a great choice for those wanting to run a Desktop VPS, especially if you are unfamiliar with Linux:

Ubuntu 12.04.5 Precise Pangolin LTS Desktop (amd64 & i386)
Ubuntu 14.04.1 Trusty Tahr LTS Desktop (amd64 & i386)

We’ll be doing some more updating in the very near future, if there is a particular OS you’d like us to update sooner rather than later, or if there’s an OS you’d like us to add to the collection that we don’t already have, just let us know, we’ll almost certainly say yes. The full list of OS in the collection can be found here.

Network Outage

Sat 23rd Aug 2014 11:16AM – Some VPS customers will currently be experiencing a lack of network connectivity to their VPS. We are investigating urgently and will update here as soon as we have further news.

Update 11:28AM – We have identified the cause of the issue and have raised a request with on site technicians at the Data Centre to help resolve the problem.

Update 11:34AM – We’d like to apologise to any customers affected by this, we expect to have this resolved very shortly.

Update 11:36AM – This issue has now been resolved and network connectivity has now resumed. Once again, we apologise for the inconvenience caused, a post mortem will follow shortly.

Network packet loss

The Data Centre we are in are this evening experiencing intermittent network issues. There have been two occasions this evening, once at around 18:15, and again at around  20:55, where VPS users have seen varying levels of packet loss lasting between 2 – 4 minutes.  Rest assured the Data Centre have a team of network engineers aware of the issues and currently working to resolve them. We’d like to apologise to customers for the inconvenience caused. We will provide a further update in due course.

Update 22:45 – The network has been stable since the last incident at about 20:55. If we see any further incidents we’ll add a further update but, for now, all is looking fine.

Update Mon 11th Aug, 18:41 – The Data Centre have informed us that the packet loss yesterday was the result of a persistent DDoS attack against one of their other customers. The DC have managed to mitigate against the attack, and we have not seen any further packet loss since yesterday as a result.

Centos 7 is here, ISOs added

The CentOS team have just announced a few hours ago the very first generally available release of the next major version of CentOS Linux, version 7.

The initial release is for for 64 bit x86 compatible machines only. The CentOS project say on their site homepage “There are many fundamental changes in this release, compared to previous releases of CentOS. Notably the inclusion of SystemD, Gnome3, and a default filesystem of XFS.”  For more information see the release announcement and release notes.

Existing Centos 6 users will be interested to know that, for the first time, there is a supported upgrade path from CentOS 6 to CentOS 7. See the release notes for further info.

We have added the main DVD install ISO to our collection, for customers who would like to install the OS into their VPS already.  We’ve also added two Live CD versions for Gnome and KDE desktops, for those who’d like to explore the CentOS 7 environment without installing yet. The Live CDs will not modify the content of your VPS hard disk(s), unless you choose to install CentOS 7 from within the Live environment.

As always, if there is a particular operating system you’d like to see in our collection, whether you are a prospective customer thinking about joining us, or an existing customer, do not hesitate to suggest it, we will almost certainly say Yes!