[c-nsp] How to troubleshoot input queue drop continuously on Cisco7600?

Discussion:

PlaWanSai RMUTT CPE IX

2014-10-20 10:00:29 UTC

k1#sho int te1/4

Last clearing of "show interface" counters 00:12:35

Input queue: 0/200/69739/69739 (size/max/drops/flushes); Total output
drops: 0

k1#sho ver

Cisco IOS Software, c7600s72033_rp Software
(c7600s72033_rp-ADVIPSERVICESK9-M), Version 12.2(33)SRD1, RELEASE SOFTWARE
(fc4)

k1#sho mod

Mod Ports Card Type Model Serial
No.

--- ----- -------------------------------------- ------------------
-----------

1 4 CEF720 4 port 10-Gigabit Ethernet WS-X6704-10GE
SAL11509UP

k1#sho queueing int te1/4

Packets dropped on Receive:

BPDU packets: 0

queue dropped [cos-map]

---------------------------------------------------

1 0 [0 1 2 3 4 5 6 7 ]

2 0 []

3 0 []

4 0 []

5 0 []

6 0 []

7 0 []

8 0 []

k1# sho processes cpu sor 5se | inc Input

199 1041732368 585523433 1779 1.83% 1.60% 1.65% 0 IP Input

10 205864632 318010747 647 0.79% 0.41% 0.43% 0 ARP Input

63 3419828 173410402 19 0.00% 0.00% 0.00% 0 Net Input

324 8 70 114 0.00% 0.00% 0.00% 0 Probe Input

326 688 2580 266 0.00% 0.00% 0.00% 0 RARP Input

355 895260 5474192 163 0.00% 0.00% 0.00% 0 LFDp Input
Proc

493 3477156 856969731 4 0.00% 0.00% 0.00% 0 IGMP Input

498 698869642660277977 0 0.00% 0.03% 0.02% 0 CLNS Input

Thank you very much.

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Peter Rathlev

2014-10-20 14:50:22 UTC

Permalink

Post by PlaWanSai RMUTT CPE IX
k1#sho int te1/4
Last clearing of "show interface" counters 00:12:35
Input queue: 0/200/69739/69739 (size/max/drops/flushes); Total output
drops: 0

On this platform those drops could be for traffic forwarded to the CPU.
What does the interface configuration look like? Is it a L3 interface
("no switchport"), or is it a L2 interface? If the latter, and you have
SVIs (VLAN interfaces) for some of the VLANs it carries, look for drops
on those SVIs.

If they are "microburst" drops on L3-interfaces then you can possibly
mitigate it using "hold-queue <N> in" on the L3-interfaces (SVIs or the
"no switchport" interface itself). This is a trade-off; don't use a too
high setting for input-queue or you will get other problems.

You could use a SPAN session to look at what traffic might cause this.
The SPAN replication is before traffic is forwarded to the CPU so you
should be able to see floods.

Post by PlaWanSai RMUTT CPE IX
1 4 CEF720 4 port 10-Gigabit Ethernet WS-X6704-10GE

This card has rather small queues so it might simply be microbursts. In
that case you might have to find a port on another card.

For general info about input queue drops this is a good read:

http://www.cisco.com/c/en/us/support/docs/routers/10000-series-routers/6343-queue-drops.html

--
Peter

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Simon Lockhart

2014-10-20 15:45:33 UTC

Permalink

Post by PlaWanSai RMUTT CPE IX
Mod Ports Card Type Model Serial
No.
--- ----- -------------------------------------- ------------------
-----------
1 4 CEF720 4 port 10-Gigabit Ethernet WS-X6704-10GE
SAL11509UP

How much traffic are you trying to put through this card? It runs out of
steam somewhere around 25Gbps for iMix traffic profiles.

Simon
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

PlaWanSai RMUTT CPE IX

2014-10-21 01:42:47 UTC

Permalink

k1#sho int te1/4 | inc dro
Input queue: 1/75/3907395/3653900 (size/max/drops/flushes); Total output
drops: 0

k1#sho int te1/4 | inc rat
Queueing strategy: fifo
30 second input rate 3,570,331,000 bits/sec, 365776 packets/sec
30 second output rate 496789000 bits/sec, 231843 packets/sec
k1#sho int te1/3 | inc rat
Queueing strategy: fifo
30 second input rate 1,752,208,000 bits/sec, 185679 packets/sec
30 second output rate 257731000 bits/sec, 122720 packets/sec
k1#sho int te1/2 | inc rat
Queueing strategy: fifo
30 second input rate 3,602,173,000 bits/sec, 425089 packets/sec
30 second output rate 2326376000 bits/sec, 351745 packets/sec
k1#sho int te1/1 | inc rat
Queueing strategy: fifo
30 second input rate 371,024,000 bits/sec, 175235 packets/sec
30 second output rate 2565494000 bits/sec, 291311 packets/sec

Traffics are we trying to put through this card about 9 Gbps.

k1#sho int te1/4 | inc dro
Input queue: 1/75/3910272/3656732 (size/max/drops/flushes); Total output
drops: 0

Thank you very much.

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Pete Lumbis

2014-10-21 19:24:40 UTC

Permalink

Oversubscribing the card would cause output drops, not input queue drops.
As someone else mentioned, this is punted traffic.

I'd suggest using NetDR to find out.
http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/116475-technote-product-00.html

On Mon, Oct 20, 2014 at 9:42 PM, PlaWanSai RMUTT CPE IX <

Post by PlaWanSai RMUTT CPE IX
k1#sho int te1/4 | inc dro
Input queue: 1/75/3907395/3653900 (size/max/drops/flushes); Total output
drops: 0
k1#sho int te1/4 | inc rat
Queueing strategy: fifo
30 second input rate 3,570,331,000 bits/sec, 365776 packets/sec
30 second output rate 496789000 bits/sec, 231843 packets/sec
k1#sho int te1/3 | inc rat
Queueing strategy: fifo
30 second input rate 1,752,208,000 bits/sec, 185679 packets/sec
30 second output rate 257731000 bits/sec, 122720 packets/sec
k1#sho int te1/2 | inc rat
Queueing strategy: fifo
30 second input rate 3,602,173,000 bits/sec, 425089 packets/sec
30 second output rate 2326376000 bits/sec, 351745 packets/sec
k1#sho int te1/1 | inc rat
Queueing strategy: fifo
30 second input rate 371,024,000 bits/sec, 175235 packets/sec
30 second output rate 2565494000 bits/sec, 291311 packets/sec
Traffics are we trying to put through this card about 9 Gbps.
k1#sho int te1/4 | inc dro
Input queue: 1/75/3910272/3656732 (size/max/drops/flushes); Total output
drops: 0
Thank you very much.
_______________________________________________
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Andrew Miehs

2014-10-21 21:39:32 UTC

Permalink

Just wondering whether you could also see input drops if the fabric/ port you want to egress is full....

Sent from a mobile device