Plusnet Service

THE PROBLEM
========================
* Describe the problem.
Over recent days we have seen a spate of complaints in the forums from customers receiving well below expected speeds during peak hours. The Faults Team have also reported an increase in the volume of Fibre provisioned speed faults which support the theory that there is a problem somewhere.

* Describe the correct functionality (if applicable)
Customers should be receiving close to expected speeds.

* How can the problem be replicated? Please add step by step details and URLs or location of the service.
Connect an external WBMC/IPSC circuit ensuring you're routed over our Faraday host links (this can be confirmed at our side) and run a number of speed tests between 7pm and Midnight.

* Any other detail you think may be useful.
AAISP have an open incident regarding packet loss on their Faraday hostlinks -
http://status.aaisp.net.uk/apost.cgi?incident=1455
This is consistent with our investigation that shows customers reporting problems to be connected via Faraday, where as those unaffected seem to be connected via Collindale.

EXTERNAL IMPACT
========================
* How may customers are affected?
- confirmed: 10+
- potential: All customers connected via Faraday hostlinks during peak times (more than a third of our subscriber base?)

* Do you have any examples of affected users? (usernames/SIDs etc)
This is being discussed in our forums here: http://community.plus.net/forum/index.php/topic,103498.0/topicseen.html

* Is there a workaround for this problem?
Yes. Customers can disconnect/reconnect until they hit a Collindale host-link. The problem with this approach is that customers cannot influence which host-link they connect to and neither is there any way for them to tell which host-link they're being routed via after establishing a connection.

If you were logged in to the portal, you could associate your username with this problem.

Our suppliers have acknowledged the existence of a problem and assigned us a case reference. We are presently in the process of escalating the problem given the significant volume of customers it has the potential to affect.

Service Status thread opened - http://usertools.plus.net/status/archive/1332846515.htm

The case has been escalated with our supplier and we're awaiting a response.

Our suppliers have identified the problem.It's being caused by the traffic going over a single route on the core network instead of being distributed. This is causing degradation of service during peak times.

Engineers are continuing to work towards a fix however we're already starting to see some customers speeds deteriorting significantly.

Service Status updated - http://usertools.plus.net/status/archive/1332865052.htm

Bandwidth has been increased in the core network. The incident will be left open through the peak period tonight and the traffic levels will be checked before it is closed. We are now monitoring for stability.

Service Status updated - http://usertools.plus.net/status/archive/1332868440.htm

Our suppliers have now reported the problem as fully resolved:

"As a part of the process for providing early warning of possible Broadband network incidents, BT would like to advise that the Broadband issue in the following geographical area(s) has been resolved.

Geographical area : Faraday"

Service Status thread closed - http://usertools.plus.net/status/archive/1332926157.htm