[c-nsp] Internet speed

[r***@mail.com]

Hi everyone,

I wanted to ask how do you guys handle the customer complains about slow Internet speed? Today almost everyone takes the measurement from speedtest.net and reports that as the speed their getting.

As far as how speedtest works is that is uses multiple TCP connections which is not real measurement as opposed to Iperf for example.

It also selects a public server which is outside of your AS thus taking into consideration the busy international links which are outside of your administration andas a result for a 30Mbps package the measure shows 15 for example.

Do you ask customers to select the local server when doing speedtests? Would like to know how do you treat those cases, any special tool or measurement?

Thanks,
Ton
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[Jared Mauch]

Host your own. Here’s a good one:

https://github.com/adolfintel/speedtest

Jared Mauch

Post by r***@mail.com
Hi everyone,
I wanted to ask how do you guys handle the customer complains about slow Internet speed? Today almost everyone takes the measurement from speedtest.net and reports that as the speed their getting.
As far as how speedtest works is that is uses multiple TCP connections which is not real measurement as opposed to Iperf for example.
It also selects a public server which is outside of your AS thus taking into consideration the busy international links which are outside of your administration andas a result for a 30Mbps package the measure shows 15 for example.
Do you ask customers to select the local server when doing speedtests? Would like to know how do you treat those cases, any special tool or measurement?
Thanks,
Ton
_______________________________________________
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.

[Jason Lixfeld]

In addition to an on-net server, we ask our customers to test from a hardwired connection directly into our CPE instead of Wi-Fi. Wi-Fi can negatively affect performance due to variables that are impossible to predict or compensate for.

Sent from my iPhone

Post by Jared Mauch
https://github.com/adolfintel/speedtest
Jared Mauch

Post by r***@mail.com
Hi everyone,
I wanted to ask how do you guys handle the customer complains about slow Internet speed? Today almost everyone takes the measurement from speedtest.net and reports that as the speed their getting.
As far as how speedtest works is that is uses multiple TCP connections which is not real measurement as opposed to Iperf for example.
It also selects a public server which is outside of your AS thus taking into consideration the busy international links which are outside of your administration andas a result for a 30Mbps package the measure shows 15 for example.
Do you ask customers to select the local server when doing speedtests? Would like to know how do you treat those cases, any special tool or measurement?
Thanks,
Ton
_______________________________________________
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://pu

[Scott Whyte]

As a customer, I demand to see that the bandwidth I am paying for is
available off-net, to at least SOME endpoints on the Internet, via both
UDP and TCP.

Post by Jason Lixfeld
In addition to an on-net server, we ask our customers to test from a hardwired connection directly into our CPE instead of Wi-Fi. Wi-Fi can negatively affect performance due to variables that are impossible to predict or compensate for.
Sent from my iPhone

Post by Jared Mauch
https://github.com/adolfintel/speedtest
Jared Mauch

Post by r***@mail.com
Hi everyone,
I wanted to ask how do you guys handle the customer complains about slow Internet speed? Today almost everyone takes the measurement from speedtest.net and reports that as the speed their getting.
As far as how speedtest works is that is uses multiple TCP connections which is not real measurement as opposed to Iperf for example.
It also selects a public server which is outside of your AS thus taking into consideration the busy international links which are outside of your administration andas a result for a 30Mbps package the measure shows 15 for example.
Do you ask customers to select the local server when doing speedtests? Would like to know how do you treat those cases, any special tool or measurement?
Thanks,
Ton
_______________________________________________
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermai

[Mark Tinka]

Post by Scott Whyte
As a customer, I demand to see that the bandwidth I am paying for is
available off-net, to at least SOME endpoints on the Internet, via
both UDP and TCP.

It is unrealistic to expect this of any network that does not own and
operate the entire end-to-end session.

That's like saying to Etihad, "Make sure that when I land there is no
traffic on the roads between the airport and my hotel".

Mark.
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[Scott Whyte]

Post by Mark Tinka

Post by Scott Whyte
As a customer, I demand to see that the bandwidth I am paying for is
available off-net, to at least SOME endpoints on the Internet, via
both UDP and TCP.

It is unrealistic to expect this of any network that does not own and
operate the entire end-to-end session.
That's like saying to Etihad, "Make sure that when I land there is no
traffic on the roads between the airport and my hotel".

Its more like "you don't get my money."

Post by Mark Tinka
Mark.

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[Mark Tinka]

Post by Scott Whyte
Its more like "you don't get my money."

Which is fine, because in most cases, ISP's spend the most money
troubleshooting the least revenue.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[Mikael Abrahamsson]

Post by r***@mail.com
It also selects a public server which is outside of your AS thus taking
into consideration the busy international links which are outside of
your administration andas a result for a 30Mbps package the measure
shows 15 for example.

My experience is that speedtest.net works well up to around 500 megabit/s,
after that it starts to get unreliable. This of course means their test
server needs to be not on the other side of the world, but for North
America and Europe this shouldn't be the case.

Without knowing exactly your conditions, I'd say your customers getting 15
megabit/s in Speedtest.net on a 30 megabit/s package actually indicates
that there is a real problem.

1. Require that your customers do the measurement wired (not wifi),
directly connected to your equipment (if you provide one).

2. If speedtest.net isn't nearby you or you have a weird network path to
them that doesn't work well, look into how you can improve it, plus host
your own speedtest server. If speedtest.net testing servers aren't able to
provide 30 megabit/s to your customers, you most likely actually have a
connectivity issue negatively affecting your customers, not only for their
testing.

I frequently test 500-1000 megabit/s subscriptions. If customer gets 200
megabit/s in a wired test, it's typically indicative of a problem. If they
get 500-800, that's usually fine and it's other issues outside of your
control that is affecting this (different operating systems have different
TCP window scaling settings etc).

But getting 15 meg out of 30, I'd say you have a problem you should look
into.
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[Jason Lixfeld]

This thread has me thinking about cases where a MetroE customer might call and complain about throughput issues, and troubleshooting would normally require a truck roll to hook up an Ethernet test set.

Does anyone know of a Y.1564 client application, or know of any past work done around creating one?

I’m wondering if a client-side application could be a practical alternative to said truck roll.

Sent from my iPhone

Post by r***@mail.com
It also selects a public server which is outside of your AS thus taking into consideration the busy international links which are outside of your administration andas a result for a 30Mbps package the measure shows 15 for example.

My experience is that speedtest.net works well up to around 500 megabit/s, after that it starts to get unreliable. This of course means their test server needs to be not on the other side of the world, but for North America and Europe this shouldn't be the case.
Without knowing exactly your conditions, I'd say your customers getting 15 megabit/s in Speedtest.net on a 30 megabit/s package actually indicates that there is a real problem.
1. Require that your customers do the measurement wired (not wifi), directly connected to your equipment (if you provide one).
2. If speedtest.net isn't nearby you or you have a weird network path to them that doesn't work well, look into how you can improve it, plus host your own speedtest server. If speedtest.net testing servers aren't able to provide 30 megabit/s to your customers, you most likely actually have a connectivity issue negatively affecting your customers, not only for their testing.
I frequently test 500-1000 megabit/s subscriptions. If customer gets 200 megabit/s in a wired test, it's typically indicative of a problem. If they get 500-800, that's usually fine and it's other issues outside of your control that is affecting this (different operating systems have different TCP window scaling settings etc).
But getting 15 meg out of 30, I'd say you have a problem you should look into.
_______________________________________________
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.

[Brian Turnbow]

Hi,

Here in Italy the ministry of telecommunications has built a system for
consumers to "certify" the speed of their connection.
Most of the information is in Italian on the site
https://www.misurainternet.it/
Basically they have placed servers is the major IXs and end users can
download the client runs tests and obtain a "certification" that can be used
against the isp for breach of contract if the speeds are lower than
contracted .


The software is based upon ETSI EG 202 765-4, not y.1564, and at least the
client side is open source with some English notes etc.
https://github.com/fondazionebordoni/nemesys




Brian

-----Original Message-----
Jason Lixfeld
Sent: domenica 12 agosto 2018 21:39
To: Mikael Abrahamsson
Subject: Re: [c-nsp] Internet speed
This thread has me thinking about cases where a MetroE customer might call
and complain about throughput issues, and troubleshooting would normally
require a truck roll to hook up an Ethernet test set.
Does anyone know of a Y.1564 client application, or know of any past work
done around creating one?
I’m wondering if a client-side application could be a practical
alternative to
said truck roll.
Sent from my iPhone

Post by Mikael Abrahamsson

Post by r***@mail.com
It also selects a public server which is outside of your AS thus taking
into

consideration the busy international links which are outside of your
administration andas a result for a 30Mbps package the measure shows 15
for example.

Post by Mikael Abrahamsson
My experience is that speedtest.net works well up to around 500

megabit/s, after that it starts to get unreliable. This of course means
their
test server needs to be not on the other side of the world, but for North
America and Europe this shouldn't be the case.

Post by Mikael Abrahamsson
Without knowing exactly your conditions, I'd say your customers getting
15

megabit/s in Speedtest.net on a 30 megabit/s package actually indicates
that
there is a real problem.

Post by Mikael Abrahamsson
1. Require that your customers do the measurement wired (not wifi),

directly connected to your equipment (if you provide one).

Post by Mikael Abrahamsson
2. If speedtest.net isn't nearby you or you have a weird network path to

them that doesn't work well, look into how you can improve it, plus host
your own speedtest server. If speedtest.net testing servers aren't able to
provide 30 megabit/s to your customers, you most likely actually have a
connectivity issue negatively affecting your customers, not only for their
testing.

Post by Mikael Abrahamsson
I frequently test 500-1000 megabit/s subscriptions. If customer gets 200

megabit/s in a wired test, it's typically indicative of a problem. If they
get
500-800, that's usually fine and it's other issues outside of your control
that
is affecting this (different operating systems have different TCP window
scaling settings etc).

Post by Mikael Abrahamsson
But getting 15 meg out of 30, I'd say you have a problem you should look

into.

Post by Mikael Abrahamsson
_______________________________________________
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
arch

[Jon Wolberg]

There was a very similar discussion on NANOG last month about the same
topic. You can read the thread here:

https://mailman.nanog.org/pipermail/nanog/2018-July/096205.html

You may find it useful.

Jon Wolberg

Post by Brian Turnbow
Hi,
Here in Italy the ministry of telecommunications has built a system for
consumers to "certify" the speed of their connection.
Most of the information is in Italian on the site
https://www.misurainternet.it/
Basically they have placed servers is the major IXs and end users can
download the client runs tests and obtain a "certification" that can be used
against the isp for breach of contract if the speeds are lower than
contracted .
The software is based upon ETSI EG 202 765-4, not y.1564, and at least the
client side is open source with some English notes etc.
https://github.com/fondazionebordoni/nemesys
Brian

-----Original Message-----
Jason Lixfeld
Sent: domenica 12 agosto 2018 21:39
To: Mikael Abrahamsson
Subject: Re: [c-nsp] Internet speed
This thread has me thinking about cases where a MetroE customer might

call

and complain about throughput issues, and troubleshooting would normally
require a truck roll to hook up an Ethernet test set.
Does anyone know of a Y.1564 client application, or know of any past work
done around creating one?
I’m wondering if a client-side application could be a practical
alternative to
said truck roll.
Sent from my iPhone

Post by Mikael Abrahamsson

Post by r***@mail.com
It also selects a public server which is outside of your AS thus

taking

Post by Mikael Abrahamsson

Post by r***@mail.com
into

consideration the busy international links which are outside of your
administration andas a result for a 30Mbps package the measure shows 15
for example.

Post by Mikael Abrahamsson
My experience is that speedtest.net works well up to around 500

megabit/s, after that it starts to get unreliable. This of course means
their
test server needs to be not on the other side of the world, but for North
America and Europe this shouldn't be the case.

Post by Mikael Abrahamsson
Without knowing exactly your conditions, I'd say your customers

getting

Post by Mikael Abrahamsson
15

megabit/s in Speedtest.net on a 30 megabit/s package actually indicates
that
there is a real problem.

Post by Mikael Abrahamsson
1. Require that your customers do the measurement wired (not wifi),

directly connected to your equipment (if you provide one).

Post by Mikael Abrahamsson
2. If speedtest.net isn't nearby you or you have a weird network path

to

them that doesn't work well, look into how you can improve it, plus host
your own speedtest server. If speedtest.net testing servers aren't able

to

provide 30 megabit/s to your customers, you most likely actually have a
connectivity issue negatively affecting your customers, not only for

their

testing.

Post by Mikael Abrahamsson
I frequently test 500-1000 megabit/s subscriptions. If customer gets

200

megabit/s in a wired test, it's typically indicative of a problem. If

they

get
500-800, that's usually fine and it's other issues outside of your

control

that
is affecting this (different operating systems have different TCP window
scaling settings etc).

Post by Mikael Abrahamsson
But getting 15 meg out of 30, I'd say you have a problem you should

look

into.

Post by Mikael Abrahamsson
_______________________________________________
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp

[Mark Tinka]

Post by Jon Wolberg
There was a very similar discussion on NANOG last month about the same
https://mailman.nanog.org/pipermail/nanog/2018-July/096205.html

That's the one.

Bottom line, I don't think speedtest.net is actually adding value to the
Internet. In fact, I think it's making things worse, centered on passing
blame rather than fixing the actual user experience. When every Internet
problem is because of throughput (like a WhatsApp message that won't
deliver), the focus is misplaced, time is wasted, energy is lost.

Connectivity is getting faster and faster, and speedtest.net comes from
a world where ADSL and low-speed services were the norm. The testing
platforms and criteria need to adapt to the increase in last mile speed,
and I believe a lot of this works lies with the content sources and
CDN's, and less so with the 3rd party services like speedtest.net who
bear very little responsibility to actually achieving the desired effect
- a good user experience.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[Charles Sprickman via cisco-nsp]

Post by Mark Tinka

Post by Jon Wolberg
There was a very similar discussion on NANOG last month about the same
https://mailman.nanog.org/pipermail/nanog/2018-July/096205.html

That's the one.
Bottom line, I don't think speedtest.net is actually adding value to the
Internet. In fact, I think it's making things worse, centered on passing
blame rather than fixing the actual user experience. When every Internet
problem is because of throughput (like a WhatsApp message that won't
deliver), the focus is misplaced, time is wasted, energy is lost.

The number of hours we spend chasing bad speedtest results is just
infuriating. For some issues, it’s great (like customer should have multi-
megabit service and multiple speedtest.net <http://speedtest.net/> servers report sub-1Mb/s).

But with 100Mb/s and 1Gb/s connections becoming the norm in the more
densely populated suburbs, just how meaningful are these tests? I am
lucky enough to have 1Gb/s FTTH at home. It’s wonderful. I have one
desktop where the link between this switch (consumer $30 switches)
will sometimes drop to 100Mb/s. I can go a week or more without even
noticing that. Home or business - do people really feel the difference
between 400Mb/s and 900Mb/s on a 1Gb/s metro-e circuit? How many
endpoints they’re talking to can support that consistently?

Post by Mark Tinka
Connectivity is getting faster and faster, and speedtest.net comes from
a world where ADSL and low-speed services were the norm. The testing
platforms and criteria need to adapt to the increase in last mile speed,
and I believe a lot of this works lies with the content sources and
CDN's, and less so with the 3rd party services like speedtest.net who
bear very little responsibility to actually achieving the desired effect
- a good user experience.

At least in the NYC area speedtest.net <http://speedtest.net/> is getting really negligent in verifying
that their partners that host the test servers actually have the capacity to act
as test servers. It’a a complete mess and it’s as if the folks at speedtest.net <http://speedtest.net/>
don’t grasp that the northeast is chock-full of FiOS and other high-bandwidth
home and SoHo connections
 It’s a real PITA for anyone supporting
customers of said connections, either as an ISP or an MSP.

Charles

Post by Mark Tinka
Mark.
_______________________________________________
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[Mark Tinka]

The number of hours we spend chasing bad speedtest results is just 
infuriating.  For some issues, it’s great (like customer should have
multi-
megabit service and multiple speedtest.net
<http://speedtest.net> servers report sub-1Mb/s).

You need to start asking yourself whether all the costs you incur to
troubleshoot a customer that does not fully understand how TCP works,
LFN's, the limitations of speedtest.net and friends, e.t.c., is worth
the revenue you are making from them, if all that cost eats up the
revenue for that user 10 times over?

But with 100Mb/s and 1Gb/s connections becoming the norm in the more
densely populated suburbs, just how meaningful are these tests?

It's the same question Mac asks Gin, "What can you do with 7 billion
that you can't do with 4?"

When you have a customer complaining that he won't sign-off the service
because he is achieving 920Mbps on his speed test instead of of 1Gbps,
you can see how dark and deep that rabbit hole goes. Okay, on the
off-chance that my network was badly built, what's happening to the
920Mbps while you're asleep, having dinner, on the throne, out with the
family, entertaining guests...

 I am
lucky enough to have 1Gb/s FTTH at home.  It’s wonderful.  I have one
desktop where the link between this switch (consumer $30 switches)
will sometimes drop to 100Mb/s.  I can go a week or more without even
noticing that.  Home or business - do people really feel the difference 
between 400Mb/s and 900Mb/s on a 1Gb/s metro-e circuit?  How many
endpoints they’re talking to can support that consistently?

Exactly.

I have a 100Mbps at my house. I know that I can't expect to push files
to Brazil from South Africa at 100Mbps, mostly because I know nothing
about the state of the network in Brazil (or anything else in between
beyond my ISP). Am I going to bang my head against the wall trying to
figure out why? Probably not.

If I can download my 2GB file in 2 minutes, do I really care whether I
hit 100Mbps or 50Mbps, or 30Mbps, or that that 2GB file didn't come down
in 30 seconds?

The Internet is such a complex web (no pun intended), and with
everything becoming part of the cloud more and more, this issue is going
to get worse.

At least in the NYC area speedtest.net <http://speedtest.net> is
getting really negligent in verifying
that their partners that host the test servers actually have the
capacity to act
as test servers.  It’a a complete mess and it’s as if the folks at
speedtest.net <http://speedtest.net>
don’t grasp that the northeast is chock-full of FiOS and other
high-bandwidth
home and SoHo connections…  It’s a real PITA for anyone supporting
customers of said connections, either as an ISP or an MSP.

You and I get that. The frustration is that customers think
speedtest.net is infallible... that they've built this very robust
global network that can test anything from 64Kbps to 1Tbps, and if that
falls short, it is the ISP's fault, and not speedtest.net. And customers
are unwilling to understand the mechanics that go into what
speedtest.net do.

It is such a rife problem, and I think it's time the folk at Ookla did
something about it.

We have stopped supporting Ookla's on our network, simply because we
don't believe they are part of the solution. We shut those boxes down
months ago, and down, they will remain.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net

[Giles Coochey]

Post by Mark Tinka
Exactly.
I have a 100Mbps at my house. I know that I can't expect to push files
to Brazil from South Africa at 100Mbps, mostly because I know nothing
about the state of the network in Brazil (or anything else in between
beyond my ISP). Am I going to bang my head against the wall trying to
figure out why? Probably not.
If I can download my 2GB file in 2 minutes, do I really care whether I
hit 100Mbps or 50Mbps, or 30Mbps, or that that 2GB file didn't come down
in 30 seconds?
The Internet is such a complex web (no pun intended), and with
everything becoming part of the cloud more and more, this issue is going
to get worse.

When we get to the point of 1Gbps links I start to think of anything
higher than that is an increase in capacity and not speed, there are not
currently any consumer data transfer applications that are going to
benefit from anything faster, and I would probably propose that for
users who enjoy the convenience of Wifi and laptops over desktops or
workstations that the point where speed becomes capacity is probably
lower than 300Mbps.

Used to work at a small ISP and always avoid the use of the term
'speed', we spoke about bandwidth and capacity. When a corporate
customer phoned one day to check whether we were experiencing problems
(they were also a friend of mine by the way) and complained of slow
speed Internet, I said "Gosh, let me check.... do you know you're right,
the electrons are moving slower, and the speed of light seems a lot
lower than it was yesterdday!!".
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[Mark Tinka]

Post by Giles Coochey
When we get to the point of 1Gbps links I start to think of anything
higher than that is an increase in capacity and not speed, there are
not currently any consumer data transfer applications that are going
to benefit from anything faster, and I would probably propose that for
users who enjoy the convenience of Wifi and laptops over desktops or
workstations that the point where speed becomes capacity is probably
lower than 300Mbps.
Used to work at a small ISP and always avoid the use of the term
'speed', we spoke about bandwidth and capacity. When a corporate
customer phoned one day to check whether we were experiencing problems
(they were also a friend of mine by the way) and complained of slow
speed Internet, I said "Gosh, let me check.... do you know you're
right, the electrons are moving slower, and the speed of light seems a
lot lower than it was yesterdday!!".

I struggle to explain this - most customers equate bandwidth with speed.

The simplest analogy I've always offered is "with bandwidth, 2 lanes @
60km/hr only moves far fewer cars than 8 lanes @ 60km/hr". Oh, look at
that, the speed didn't change...

As you say, at a certain point (and I think waaaaaaay below 1Gbps), an
expectation of a physical increase in the speed of data transfer becomes
stable, and at that point, the extra bandwidth is allowing you to
accommodate more users, with each one being happy at the same time. When
customers expect that that taking their Enterprise service from 10Gbps
to 20Gbps will dramatically improve how quickly Youtube loads, you can
understand the nightmare ISP's have to deal with. And Heaven forbid we
only extract 19.5Gbps out of that, instead of the full 20Gbps :-\...

Mark.
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[Rasto Rickardt]

Well, it might depend of what customer experience is for their use-case.

If you have residential user with 1Gbit with 4 users behind wifi, i can
imagine more than 100Mbit/s will not alter their experience. Not way
less, 4 times 4K youtube video will eat around 80Mbit/s.

But if you have power soho/enterprise customer which is using the line
for offsite backups or file synchronization and is well aware of
limitation of underlying protocols and is able to use multiple
sessions(lanes) it might be a bit different here.

From this point of view is speedtest.net providing best available
service. It runs on all of end-devices, can mimic end-user experience,
and network engineers did not come with anything remotely comparable :).

r.

Post by Mark Tinka
I struggle to explain this - most customers equate bandwidth with speed.
that, the speed didn't change...
As you say, at a certain point (and I think waaaaaaay below 1Gbps), an
expectation of a physical increase in the speed of data transfer becomes
stable, and at that point, the extra bandwidth is allowing you to
accommodate more users, with each one being happy at the same time. When
customers expect that that taking their Enterprise service from 10Gbps
to 20Gbps will dramatically improve how quickly Youtube loads, you can
understand the nightmare ISP's have to deal with. And Heaven forbid we
only extract 19.5Gbps out of that, instead of the full 20Gbps :-\...

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[Mark Tinka]

Post by Rasto Rickardt
Well, it might depend of what customer experience is for their use-case.
If you have residential user with 1Gbit with 4 users behind wifi, i can
imagine more than 100Mbit/s will not alter their experience. Not way
less, 4 times 4K youtube video will eat around 80Mbit/s.

Correct - but 100Mbps does not make a Skype call clearer either.

My contention is falling back on "speed and throughput" as the reason
that things don't work, all the time.

It is more important that both customers and their providers work on
what the actual issue is, as opposed to lumping every problem into, "I
didn't receive my e-mail yet, so throughput is bad".

Post by Rasto Rickardt
But if you have power soho/enterprise customer which is using the line
for offsite backups or file synchronization and is well aware of
limitation of underlying protocols and is able to use multiple
sessions(lanes) it might be a bit different here.

And this is fine because power user knows what he needs to do to fully
use up his entire link. Frees the NOC long enough to attend to a real
problem.

Post by Rasto Rickardt

Post by Rasto Rickardt
From this point of view is speedtest.net providing best available

service. It runs on all of end-devices, can mimic end-user experience,
and network engineers did not come with anything remotely comparable :).

Which is fine if your average user is on ADSL. With 1Gbps quickly
becoming the norm in much of the world (or with a good portion of the
world still running their primary connectivity on 3G/4G), I think it's
starting to show its age.

I do agree that there probably isn't a more "user-friendly" alternative
right now, but considering the physical limitations of handheld devices,
the quality of web browsers, how often we have less RAM in our computers
than we actually need, how bloated apps have become, e.t.c., are we in
any real danger of blowing up a 10Gbps link on any device that it is
necessary to test it?

Mark.
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[Giles Coochey]

Post by Rasto Rickardt
Well, it might depend of what customer experience is for their use-case.
If you have residential user with 1Gbit with 4 users behind wifi, i can
imagine more than 100Mbit/s will not alter their experience. Not way
less, 4 times 4K youtube video will eat around 80Mbit/s.
But if you have power soho/enterprise customer which is using the line
for offsite backups or file synchronization and is well aware of
limitation of underlying protocols and is able to use multiple
sessions(lanes) it might be a bit different here.

You get what you pay for really, if you buy a residential connection,
which has contention in the small print, no guarantees of service in the
contract and then start to use it for business use then it's a case of
caveat emptor.

Enterprise customers (with a business contract and strict SLA) can be
asked to produce traceroutes (to prove everything is on-net), run iperf
on clean systems directly connected to CPEs, etc... Start doing that
with a residential customer and you'll quickly reach the point where
they haven't got a clue what you're saying.

Post by Rasto Rickardt
From this point of view is speedtest.net providing best available
service. It runs on all of end-devices, can mimic end-user experience,
and network engineers did not come with anything remotely comparable :).
r.

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[Mark Tinka]

Post by Giles Coochey
 
You get what you pay for really, if you buy a residential connection,
which has contention in the small print, no guarantees of service in
the contract and then start to use it for business use then it's a
case of caveat emptor.

Agreed.

There appears to be an unrealistic expectation that you can obtain all
manner of service (technical, support, training, hardware, e.t.c.) for
an Internet service regardless of which product you buy.

"It's just BGP, so I should be able to sell it to any of our customers,
residential, enterprise or service provider", the sales folk will say.
And I say, "Well, if they want BGP, they should upgrade... we have a
product for that."

Like the little kid that doesn't understand when they board the plane
walking past the big comfy chairs, as they walk toward the back, "Daddy,
why aren't we sitting in those ones there; they look nice and
comfortable, with a lot of space, just like at home?"

Post by Giles Coochey
Enterprise customers (with a business contract and strict SLA) can be
asked to produce traceroutes (to prove everything is on-net), run
iperf on clean systems directly connected to CPEs, etc... Start doing
that with a residential customer and you'll quickly reach the point
where they haven't got a clue what you're saying.

We've stopped worrying about the lack of clue. What we worry about now
is what is the cost of troubleshooting (sometimes, truck rolls) vs. the
revenue the customer generates us. If the cost to troubleshoot is way
higher than the revenue over a given period for a given troubleshooting
cycle, we'll let the customer know that we will charge them if we find
the fault to be theirs.

This has offered quite a bit of incentive into the customer being
absolutely serious before they call with a complaint.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at h

[Giles Coochey]

Post by Mark Tinka

Post by Giles Coochey
When we get to the point of 1Gbps links I start to think of anything
higher than that is an increase in capacity and not speed, there are
not currently any consumer data transfer applications that are going
to benefit from anything faster, and I would probably propose that
for users who enjoy the convenience of Wifi and laptops over desktops
or workstations that the point where speed becomes capacity is
probably lower than 300Mbps.
Used to work at a small ISP and always avoid the use of the term
'speed', we spoke about bandwidth and capacity. When a corporate
customer phoned one day to check whether we were experiencing
problems (they were also a friend of mine by the way) and complained
of slow speed Internet, I said "Gosh, let me check.... do you know
you're right, the electrons are moving slower, and the speed of light
seems a lot lower than it was yesterdday!!".

I struggle to explain this - most customers equate bandwidth with speed.
that, the speed didn't change...
As you say, at a certain point (and I think waaaaaaay below 1Gbps), an
expectation of a physical increase in the speed of data transfer
becomes stable, and at that point, the extra bandwidth is allowing you
to accommodate more users, with each one being happy at the same time.
When customers expect that that taking their Enterprise service from
10Gbps to 20Gbps will dramatically improve how quickly Youtube loads,
you can understand the nightmare ISP's have to deal with. And Heaven
forbid we only extract 19.5Gbps out of that, instead of the full
20Gbps :-\...
Mark.

I've liked the number of lanes analogy and used it myself quite a few
times. We offered DDoS protection to corporate clients too, so we
analogised that as having better drivers on the road :-)

Locally, here in the UK, the regulator Ofcom, and the Advertising
Standards govt agency have come down on ISPs to clearly explain where
residential connections are contended - sadly they haven't gone as far
as explaining the notion of contention, but have just gone as far to say
that speeds^H^H^H^H^H^H bandwidth can be "up to.. x Mbps" and that
during busy periods achievable performance may be much lower.
I think minimum Service Level Agreements and credits for bad service
have also been applied.
Pricing structure has also been forced to be simpler, as connections
here are DSL, and land line rental was often omitted from advertised prices.

So, regulators and advertising standards bodies can help in ensuring
that the Marketing departments set the right expectation to the user of
what they're buying, probably making it easier on future tech support
departments, whose role previously seems to be a straight revenue
retention exercise rather than technically supporting real issues.
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[James Bensley]

Post by Jason Lixfeld
This thread has me thinking about cases where a MetroE customer might call and complain about throughput issues, and troubleshooting would normally require a truck roll to hook up an Ethernet test set.
Does anyone know of a Y.1564 client application, or know of any past work done around creating one?
I’m wondering if a client-side application could be a practical alternative to said truck roll.

Hi Jason,

I'm VERY interested in this too. I think this is a major gap in the
open source market.

A few years ago I started writing an Ethernet testing tool in C
(mainly to learn C). It doesn't provide any RFC2544 or Y.1564 testing,
it's just like iPerf but for Ethernet. Is it the best tool for the job
- no, did I learn some C - yes:
https://github.com/jwbensley/Etherate

Bandwidth wise it's good for 1Gbps, you can push 10Gbps if you have a
fast enough CPU and use 1500 byte frames. 1Gbps is rapidly become "too
slow" though and we need some software that MUST push 10Gbps with 1500
and ideally but not required 10Gbps with 64 byte frames. If you just
want to saturate a 10Gbps link you can do that easily with the 2nd
program I wrote in C, to teach myself about different network options
within the Linux Kernel:
https://github.com/jwbensley/EtherateMT

^ It's multithreaded so it can send/sink higher traffic rates but has
basically no options to change MAC addresses or Ethertype or insert a
VLAN etc, it's just a dumb load generator/sinker. My test boxes were
both 8 core 2.1Ghz Xeons, 1 worker thread sends about 9.7Gbps so I
needed 2 worker threads to get 10Gbps with 1500 byte frames. I didn’t
have enough cores for 10Gbps @ 64 byte frames, EtherateMT can send
about 1Mpps, so you need approximately 14.4 worker thread and I had 8
core per test box :)

I then started to look at how one might gather statistics and take
measurements at such high traffic rates - it basically can't/couldn’t
be done using native Kernel networking. DPDK allows one to send 10Gbps
@ 64 byte packets using a single core, so you have plenty of spare
cores to gather statistics on. So I started writing RFC2544 tests
using MoonGen (LUA bindings for DPDK):
https://github.com/jwbensley/MoonGen-Scripts

^ This has it’s own set of problems – the Lua bindings need updating
as DPDK evolves. You’re restricted to DPDK support NICs etc. So I’ve
given up on that too for now.

I am interesting in writing an open source RFC2544 and Y.1564
compliant tester, I just recently changed jobs and my new company has
hardware testers for RFC2544, Y.1564, RFC6349 and OAM. I think that
XDP is the way forward in the Linux Kernel and now I have access to
hardware testers I can valid the software results against the hardware
testers.

I’m open to collaboration on this if anyone else is intersted :)

Cheers,
James.
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/c

[Mark Tinka]

Post by James Bensley
I am interesting in writing an open source RFC2544 and Y.1564
compliant tester, I just recently changed jobs and my new company has
hardware testers for RFC2544, Y.1564, RFC6349 and OAM. I think that
XDP is the way forward in the Linux Kernel and now I have access to
hardware testers I can valid the software results against the hardware
testers.

While 2544 tests should do fine to prove a link's worth, I believe the
more "idealic" test should be 6349.

Most complaints from users are due to a lack of understanding about how
TCP works, particularly in LFN scenarios. 6349 focuses on TCP tests,
particularly on the effects of LFN situations, and the realization that
in some cases, window size scaling + parallel TCP sessions is the best
way to prove a link.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[James Bensley]

Post by James Bensley
I am interesting in writing an open source RFC2544 and Y.1564
compliant tester, I just recently changed jobs and my new company has
hardware testers for RFC2544, Y.1564, RFC6349 and OAM. I think that
XDP is the way forward in the Linux Kernel and now I have access to
hardware testers I can valid the software results against the hardware
testers.
While 2544 tests should do fine to prove a link's worth, I believe the more
"idealic" test should be 6349.
Most complaints from users are due to a lack of understanding about how TCP
works, particularly in LFN scenarios. 6349 focuses on TCP tests,
particularly on the effects of LFN situations, and the realization that in
some cases, window size scaling + parallel TCP sessions is the best way to
prove a link.

Depends what are you testing, physical link/node performance or user QoE.

There are cases for both and I agree that both problems require a
different hammer.

Cheers,
James.
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[Mark Tinka]

Post by James Bensley
Depends what are you testing, physical link/node performance or user QoE.
There are cases for both and I agree that both problems require a
different hammer.

QoE is what customers complain about (and what they think speedtest.net,
fast.com, e.t.c. are all about), whether they know it or not.

You and me care more about node and link performance, as QoE is very
subjective, technically and humanly.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[Rasto Rickardt]

Ton,

host your speedtest.net testserver, so customers can use the one on your
network.

https://support.speedtest.net/hc/en-us/articles/203845650-How-can-I-host-a-server-on-Speedtest-net-

Then customers have the ability to compare measurements to your on-net
server and some offnet ones, and you can move to troubleshoot further :).

r.

Post by r***@mail.com
Hi everyone,
I wanted to ask how do you guys handle the customer complains about slow Internet speed? Today almost everyone takes the measurement from speedtest.net and reports that as the speed their getting.
As far as how speedtest works is that is uses multiple TCP connections which is not real measurement as opposed to Iperf for example.
It also selects a public server which is outside of your AS thus taking into consideration the busy international links which are outside of your administration andas a result for a 30Mbps package the measure shows 15 for example.
Do you ask customers to select the local server when doing speedtests? Would like to know how do you treat those cases, any special tool or measurement?
Thanks,
Ton
_______________________________________________
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

[Mark Tinka]

Post by r***@mail.com
Hi everyone,
I wanted to ask how do you guys handle the customer complains about slow Internet speed? Today almost everyone takes the measurement from speedtest.net and reports that as the speed their getting.
As far as how speedtest works is that is uses multiple TCP connections which is not real measurement as opposed to Iperf for example.
It also selects a public server which is outside of your AS thus taking into consideration the busy international links which are outside of your administration andas a result for a 30Mbps package the measure shows 15 for example.
Do you ask customers to select the local server when doing speedtests? Would like to know how do you treat those cases, any special tool or measurement?

I'd suggest you spend some time reading this thread:

    https://mailman.nanog.org/pipermail/nanog/2018-July/096205.html

Mark.
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archiv