Discussion:
[c-nsp] 3750 high cpu from icmp
Brian Turnbow
2007-05-07 15:58:02 UTC
Permalink
Hello Everyone,
I have been working on a 3750 that has a high cpu usage and wanted to
ask for some help.
My first thought was tcam space , but that was ok and I don't see any
bad adjacencies or routes.

The switch has high interupt cpu levels and checking into it I have
found that it seems to be related to ICMP messages getting kicked to the
cpu.
sh plat port-asic stats drop
Shows this increasing counter (a few seconds apart)
Supervisor TxQueue Drop Statistics
Queue 11: 36618954
Supervisor TxQueue Drop Statistics
Queue 11: 36622889

And I have traced this queue down to icmp. The cpu controller shows high
icmp packets arriving to the cpu.(again a few seconds apart)

3750E-Jenner#sh controllers cpu-interface | i icmp
icmp 1525306547 0 0 0 0
3750E-Jenner#sh controllers cpu-interface | i icmp
icmp 1525456328 0 0 0 0


Tracing on the vlan I found alot of icmp redirects being bounced around
so I tried disabling redirects and the cpu usage went down dramatically
yet it is still high.

I was able to run a debug
debug platform cpu-queues icmp-q
And see alot of these messages.
ICMP-Q:Dropped redirect disabled on L3 IF: Local Port Fwding L3If:Vlan82
L2If:FastEthernet1/0/11

It seems that with no redirects the packets gets sent to the cpu that
proceeds to drop the packet.


I tried to implement copp to see about limiting the messages sent to the
cpu , but it does not seem possible on the 3750.
Control-plane is there yet if I try to apply the service policy I get an
error message

QoS: policymap is supported on physical, VLAN, and ES interfaces only
Service Policy attachment failed
error: failed to install policy map control-plane-in

Besides redesigning to avoid icmp redirects anyone have any ideas?

Thanks in advance

Brian


_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Jared Mauch
2007-05-07 18:03:59 UTC
Permalink
Post by Brian Turnbow
Besides redesigning to avoid icmp redirects anyone have any ideas?
Can you make sure that all your routers have the following
on their "IP" (routed) interfaces:?

no ip redirects
no ip proxy-arp

This should help solve the problem.

These two should really be default these days.

- Jared
--
Jared Mauch | pgp key available via finger from ***@puck.nether.net
clue++; | http://puck.nether.net/~jared/ My statements are only mine.
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Gary Stanley
2007-05-07 18:19:06 UTC
Permalink
Post by Jared Mauch
Post by Brian Turnbow
Besides redesigning to avoid icmp redirects anyone have any ideas?
Can you make sure that all your routers have the following
on their "IP" (routed) interfaces:?
no ip redirects
no ip proxy-arp
This should help solve the problem.
These two should really be default these days.
- Jared
I've seen this same problem on 3550's + Microsoft machines. I've
applied both of these and the high cpu usage went away.

I believe microsoft windows 2k/2k3 default ip redirects to 1, but i'm not sure.


_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Tom Sands
2007-05-07 20:44:22 UTC
Permalink
I would include no ip unreachables, we've seen major impacts from this



------------------------------------------------------
Tom Sands
Chief Network Engineer
Rackspace Managed Hosting
(210)447-4391
------------------------------------------------------
Post by Jared Mauch
Post by Brian Turnbow
Besides redesigning to avoid icmp redirects anyone have any ideas?
Can you make sure that all your routers have the following
on their "IP" (routed) interfaces:?
no ip redirects
no ip proxy-arp
This should help solve the problem.
These two should really be default these days.
- Jared
Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace
Managed Hosting. Any dissemination, distribution or copying of the enclosed
material is prohibited. If you receive this transmission in error, please
notify us immediately by e-mail at ***@rackspace.com, and delete the
original message. Your cooperation is appreciated.

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Adrian Chadd
2007-05-08 03:51:48 UTC
Permalink
Yes, some OSes ignore redirects by default and the Cisco keeps on CPU punting
traffic and sending 'em.

One thing I noticed with the Catalyst 4500 is that the ACLs don't get
"fully loaded" into TCAM if all SVI's (I didn't have l3 interfaces) have
the no ip redir; no ip unreach configured. The rules get loaded, but the
last rule is "punt to CPU" and thus its not "fully loaded" according to
the sh platform commands.

The engineer said (the development engineer said to them that) the last
rule had to be "punt to CPU" for any traffic that didn't match any rules,
so it could be processed by the CPU in case ICMP replies were needed.
I couldn't see what kind of traffic would fall through - except maybe
in the cases where you weren't running a default route? - but
better safe than sorry.




Adrian

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Peter Olsson
2007-05-12 17:01:15 UTC
Permalink
I'm trying to find out what equipment can/can't handle
802.1Q Tunneling (Q-in-Q), but I'm failing. I have googled
and looked in Feature Navigator and other documentation, but
all I find is that high end 6500/7600/10000 can handle it.
I'm at this time mostly interested in low end equipment.

Does anyone have pointers to where/how I should search to
find out which low end switches support Q-in-Q?

And are there any low end routers that can terminate/strip
Q-in-Q tags?

Thanks!
--
Peter Olsson ***@leissner.se
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Tassos Chatzithomaoglou
2007-05-12 18:09:45 UTC
Permalink
Hi Peter,

Searching on cisco.com/go/fn for "802.1Q Tunneling" returned many results.
3400,3550,3750 surely support it.
Usually "802.1Q Tunneling" refers to L2 devices, while "QinQ termination" to L3 ones.


--
Tassos
Post by Peter Olsson
I'm trying to find out what equipment can/can't handle
802.1Q Tunneling (Q-in-Q), but I'm failing. I have googled
and looked in Feature Navigator and other documentation, but
all I find is that high end 6500/7600/10000 can handle it.
I'm at this time mostly interested in low end equipment.
Does anyone have pointers to where/how I should search to
find out which low end switches support Q-in-Q?
And are there any low end routers that can terminate/strip
Q-in-Q tags?
Thanks!
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco
s***@nethelp.no
2007-05-12 18:17:11 UTC
Permalink
Post by Tassos Chatzithomaoglou
Searching on cisco.com/go/fn for "802.1Q Tunneling" returned many results.
3400,3550,3750 surely support it.
And 3560.

Steinar Haug, Nethelp consulting, ***@nethelp.no
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Peter Olsson
2007-05-12 18:32:32 UTC
Permalink
Post by s***@nethelp.no
Post by Tassos Chatzithomaoglou
Searching on cisco.com/go/fn for "802.1Q Tunneling" returned many results.
3400,3550,3750 surely support it.
And 3560.
Great, do you also know about 2960?

Thanks!
--
Peter Olsson ***@leissner.se
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
s***@nethelp.no
2007-05-12 18:38:15 UTC
Permalink
Post by Peter Olsson
Post by s***@nethelp.no
Post by Tassos Chatzithomaoglou
Searching on cisco.com/go/fn for "802.1Q Tunneling" returned many results.
3400,3550,3750 surely support it.
And 3560.
Great, do you also know about 2960?
Sure doesn't look like it from the 2960 12.2(25)SEE documentation.

Steinar Haug, Nethelp consulting, ***@nethelp.no
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Afsheen Bigdeli
2007-05-12 18:51:48 UTC
Permalink
Hello,

I have several 3750G's running the advanced IP services firmware
(c3750-advipservicesk9-mz.122-25.SED.bin). These are all model
WS-C3750G-24TS. I've also got a Metro 3750G with a 10 GE port, model
WS-C3750G-16TD-E, that was formerly in a stack with several 3750G-24TS's.

I've decided to remove the WS-C3750G-16TD-E from the stack and use it as
a standalone. However, as it's now running the advanced IP services
firmware that the 3750G's had, I've found an odd problem: the switch
powers on, passes all self tests, loads IOS; when I log in and configure
it, it functions properly, routes traffic, and for the most part works
without issue, but I'm unable to do a 'write mem', or anything else that
reads or writes to flash:, as the command bails out with an error.
Fsck'ing flash: has had no effect.

After testing in the lab, I've confirmed that a brand new
WS-C3750G-16TD-E will function properly until it's been upgraded with
this firmware, at which point everything I've tested (ACL's, VLAN's,
static routes, BGP, _everythng) will still work properly, except that
I'll no longer be able to write to flash.

Is there a separate advanced IP services firmware for the Metro 3750's I
should be running, or is this a bug? I'd imagine the former, but if that
were the case, I wouldn't expect the switch to get past rommon, much
less function at all.

Thanks for any input,

--afsheenb
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Brad Henshaw
2007-05-13 02:38:25 UTC
Permalink
Post by Afsheen Bigdeli
I've decided to remove the WS-C3750G-16TD-E from the stack and use it as
a standalone. However, as it's now running the advanced IP services
firmware that the 3750G's had, I've found an odd problem: the switch
powers on, passes all self tests, loads IOS; when I log in and configure
it, it functions properly, routes traffic, and for the most part works
without issue, but I'm unable to do a 'write mem', or anything else that
Afsheen,

You've referred to the 3750 switches as Metro 3750's a number of times.
This may be due to your application of the switches, but do not confuse
the Cat3750 and 3750-E switches with the 3750 Metro's which are a
different beast.

As far as I'm aware, the 3750-E needs a different image to the vanilla
3750. For a start you should ensure you have the correct image.

Also 12.2(25)SED has been deferred - you should move to 12.2(25)SEEx/SEG
or 12.2(35)SEx (puns aside).

If you still have problems, please post the exact image you're running
and the actual errors you're receiving.

Regards,
Brad
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Afsheen Bigdeli
2007-05-13 05:20:54 UTC
Permalink
Post by Brad Henshaw
You've referred to the 3750 switches as Metro 3750's a number of times.
This may be due to your application of the switches, but do not confuse
the Cat3750 and 3750-E switches with the 3750 Metro's which are a
different beast.
Ah, you're right. My mistake, I was confused as to which flavor of 3750
fell under which brand designation. However, now that I've looked into
it, I'm even more confused... see below.
Post by Brad Henshaw
As far as I'm aware, the 3750-E needs a different image to the vanilla
3750. For a start you should ensure you have the correct image.
This is true, but apparently the Cisco 3750G-16TD-E is considered to be
part of the 3750 line, and not the 3750-E line? See
http://www.cisco.com/en/US/products/hw/switches/ps5023/index.html - the
16 port model listed there is what we have in the lab. I would think
that the -E at the end of the model number listed in 'show ver' would
classify this as part of the 3750-E line, but the only references I see
on cisco.com for a 16 port 3750G are as part of the 3750 line, not the
3750-E line.

The other thing that stumps me is the odd failure mode. I'd expect the
switch to simply not boot, as opposed to functioning perfectly except
for when access to flash: is required.

Thanks,

--afsheenb
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Brad Henshaw
2007-05-13 06:12:15 UTC
Permalink
Post by Afsheen Bigdeli
apparently the Cisco 3750G-16TD-E is considered to be
part of the 3750 line, and not the 3750-E line?
Ah, you're correct. I noticed the E on the end of the model and failed
to realise that really refers to the 'enterprise' (now IP services...
there's no end of amusement in Cisco's product name changes.)
image rather than meaning it's part of the 3750-E series.

I haven't dealt with this particular model of 3750 but it does look as
though it's supported by the same IOS releases as the other 3750 and
3750G switches.

I notice that given the introduction of the 3750-E series, the 16TD-E
is now EoS from Oct 2007.
Post by Afsheen Bigdeli
The other thing that stumps me is the odd failure mode. I'd expect the
switch to simply not boot, as opposed to functioning perfectly except
for when access to flash: is required.
This does seem unusual - which other flash: related commands have you
tried? (sh flash: / copy flash: tftp:)
Also what are the exact errors you're receiving?

It's probably worth trying an upgrade to a different release and
if the problem remains, refer it to the TAC - unless someone else here
has any useful input.

Bear in mind that the first thing the TAC will probably tell you to do
is to perform the upgrade using the tarball rather than the .bin file.
(I'm truly sick of hearing this one)

Regards,
Brad
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Afsheen Bigdeli
2007-05-14 21:39:43 UTC
Permalink
Post by Brad Henshaw
I haven't dealt with this particular model of 3750 but it does look as
though it's supported by the same IOS releases as the other 3750 and
3750G switches.
Yep. This makes me think I've stumbled upon a bug.
Post by Brad Henshaw
Post by Afsheen Bigdeli
The other thing that stumps me is the odd failure mode. I'd expect the
switch to simply not boot, as opposed to functioning perfectly except
for when access to flash: is required.
This does seem unusual - which other flash: related commands have you
tried? (sh flash: / copy flash: tftp:)
Also what are the exact errors you're receiving?
When I try to write mem initially after an upgrade:

test#wr mem
Building configuration...

nv_done: unable to open "flash:/config.text.new"
nv_done: unable to open "flash:/private-config.text.new"[OK]
NVRAM Verification Failed
cdg-tis-sw1#fsck flash:
Fsck operation may take a while. Continue? [confirm]
flashfs[1]: 358 files, 5 directories
flashfs[1]: 0 orphaned files, 0 orphaned directories
flashfs[1]: Total bytes: 23224320
flashfs[1]: Bytes used: 15832576
flashfs[1]: Bytes available: 7391744
flashfs[1]: flashfs fsck took 13 seconds.
Fsck of flash:: complete

Hmm. Odd. Let's fsck flash and then try it...

test#wr mem
Building configuration...

i28f128j3_16x_write_bytes: command sequence error
flashfs[1]: writing to flash handle 0x261CD80, device 0, offset
0xD40000, length 0x208: Operation Failed
flashfs[1]: sector ptr: {0x6A, 0xFB}
nv_done: unable to open "flash:/config.text.new"
i28f128j3_16x_write_bytes: command sequence error
flashfs[1]: writing to flash handle 0x261CD80, device 0, offset
0x5C0000, length 0x208: Operation Failed
flashfs[1]: sector ptr: {0x2E, 0xC6}
nv_done: unable to open "flash:/private-config.text.new"
i28f128j3_16x_write_bytes: command sequence error
flashfs[1]: writing to flash handle 0x261CD80, device 0, offset
0x5C0000, length 0x208: Operation Failed
flashfs[1]: sector ptr: {0x2E, 0xC4}
flashfs[1]: filesystem marked down. Use "fsck" to recover.[OK]
NVRAM Verification Failed

And that's what I see no matter what I try to do from thatpoint on.
Post by Brad Henshaw
It's probably worth trying an upgrade to a different release and
if the problem remains, refer it to the TAC - unless someone else here
has any useful input.
Agreed. Last try: anyone seen this behavior before or have anything to
contribute?

Thanks,
--afsheenb



_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Afsheen Bigdeli
2007-05-15 18:16:26 UTC
Permalink
Ah, think I've figured this out. This is bug CSCsc41813. Now to figure
out how to upgrade the switch in-situ when I don't have access to flash...

--afsheenb
Post by Afsheen Bigdeli
Post by Brad Henshaw
I haven't dealt with this particular model of 3750 but it does look
as though it's supported by the same IOS releases as the other 3750
and 3750G switches.
Yep. This makes me think I've stumbled upon a bug.
Post by Brad Henshaw
Post by Afsheen Bigdeli
The other thing that stumps me is the odd failure mode. I'd
expect the switch to simply not boot, as opposed to functioning
perfectly except for when access to flash: is required.
This does seem unusual - which other flash: related commands have
you tried? (sh flash: / copy flash: tftp:) Also what are the exact
errors you're receiving?
test#wr mem Building configuration...
nv_done: unable to open "flash:/config.text.new" nv_done: unable to
open "flash:/private-config.text.new"[OK] NVRAM Verification Failed
cdg-tis-sw1#fsck flash: Fsck operation may take a while. Continue?
[confirm] flashfs[1]: 358 files, 5 directories flashfs[1]: 0 orphaned
files, 0 orphaned directories flashfs[1]: Total bytes: 23224320
flashfs[1]: Bytes used: 15832576 flashfs[1]: Bytes available: 7391744
flashfs[1]: flashfs fsck took 13 seconds. Fsck of flash:: complete
Hmm. Odd. Let's fsck flash and then try it...
test#wr mem Building configuration...
i28f128j3_16x_write_bytes: command sequence error flashfs[1]: writing
Operation Failed flashfs[1]: sector ptr: {0x6A, 0xFB} nv_done: unable
to open "flash:/config.text.new" i28f128j3_16x_write_bytes: command
sequence error flashfs[1]: writing to flash handle 0x261CD80, device
0, offset 0x5C0000, length 0x208: Operation Failed flashfs[1]: sector
ptr: {0x2E, 0xC6} nv_done: unable to open
"flash:/private-config.text.new" i28f128j3_16x_write_bytes: command
sequence error flashfs[1]: writing to flash handle 0x261CD80, device
0, offset 0x5C0000, length 0x208: Operation Failed flashfs[1]: sector
ptr: {0x2E, 0xC4} flashfs[1]: filesystem marked down. Use "fsck" to
recover.[OK] NVRAM Verification Failed
And that's what I see no matter what I try to do from thatpoint on.
Post by Brad Henshaw
It's probably worth trying an upgrade to a different release and if
the problem remains, refer it to the TAC - unless someone else here
has any useful input.
Agreed. Last try: anyone seen this behavior before or have anything
to contribute?
Thanks, --afsheenb
_______________________________________________ cisco-nsp mailing
https://puck.nether.net/mailman/listinfo/cisco-nsp archive at
http://puck.nether.net/pipermail/cisco-nsp/
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Peter Olsson
2007-05-16 07:23:29 UTC
Permalink
Post by Tassos Chatzithomaoglou
Hi Peter,
Searching on cisco.com/go/fn for "802.1Q Tunneling" returned many results.
3400,3550,3750 surely support it.
Usually "802.1Q Tunneling" refers to L2 devices, while "QinQ termination" to L3 ones.
I found it very strange that I couldn't see this information,
but now I discovered that not all features are displayed in
feature navigator when you are not logged in. I usually don't
log in just to check features, but now I see the need.

Thanks for opening my eyes!

Peter Olsson
_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Brian Turnbow
2007-05-08 11:25:15 UTC
Permalink
Hello
All routed interfaces have these as well as no unreachables,( all connected routers as well) yet the process cpu is still high.
I still see the cpu controller with high icmp counters , other cpu counters appear normal.
3750E-Jenner#sh controller cpu-interface | i icmp
icmp 1886230815 0 0 0 0
3750E-Jenner#sh controller cpu-interface | i icmp
icmp 1886236301 0 0 0 0
3750E-Jenner#sh controller cpu-interface | i icmp
icmp 1886239093 0 0 0 0
3750E-Jenner#sh controller cpu-interface | i icmp
icmp 1886241081 0 0 0 0

And debugging the queue I see these messages all for vlan 82 (a one second debug has hundreds of these messages)
ICMP-Q:Dropped redirect disabled on L3 IF: Local Port Fwding L3If:Vlan82 L2If:FastEthernet1/0/11
ICMP-Q:Dropped redirect disabled on L3 IF: Local Port Fwding L3If:Vlan82 L2If:FastEthernet1/0/6
ICMP-Q:Dropped redirect disabled on L3 IF: Local Port Fwding L3If:Vlan82 L2If:FastEthernet1/0/1

The addresses listed in the debugs are all correct valid addresses with valid routes.
It seems that the packets are sent to the cpu thinking that there should be a redirect , yet since it is disabled the cpu then drops the packets.


Here is the interface vlan configuration
interface Vlan82
ip address 82.113.194.2 255.255.255.224
no ip redirects
no ip unreachables
no ip proxy-arp
end


Any thoughts?
I am running 12.2.35SE2

Thanks
Brian



-----Original Message-----
From: Jared Mauch [mailto:***@puck.nether.net]
Sent: lunedì 7 maggio 2007 20.04
To: Brian Turnbow
Cc: cisco-***@puck.nether.net
Subject: Re: [c-nsp] 3750 high cpu from icmp
Post by Brian Turnbow
Besides redesigning to avoid icmp redirects anyone have any ideas?
Can you make sure that all your routers have the following
on their "IP" (routed) interfaces:?

no ip redirects
no ip proxy-arp

This should help solve the problem.

These two should really be default these days.

- Jared
--
Jared Mauch | pgp key available via finger from ***@puck.nether.net
clue++; | http://puck.nether.net/~jared/ My statements are only mine.

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Brian Turnbow
2007-05-14 08:58:44 UTC
Permalink
Wanted to post an update on this in case anyone else ever has problems.
The only way I found to resolve this issue was to move traffic onto different interfaces , removing the router on a stick routing.


Regards

Brian

-----Original Message-----
From: Brian Turnbow
Sent: martedì 8 maggio 2007 13.25
To: 'Jared Mauch'; 'Tom Sands'
Cc: cisco-***@puck.nether.net
Subject: RE: [c-nsp] 3750 high cpu from icmp

Hello
All routed interfaces have these as well as no unreachables,( all connected routers as well) yet the process cpu is still high.
I still see the cpu controller with high icmp counters , other cpu counters appear normal.
3750E-Jenner#sh controller cpu-interface | i icmp
icmp 1886230815 0 0 0 0
3750E-Jenner#sh controller cpu-interface | i icmp
icmp 1886236301 0 0 0 0
3750E-Jenner#sh controller cpu-interface | i icmp
icmp 1886239093 0 0 0 0
3750E-Jenner#sh controller cpu-interface | i icmp
icmp 1886241081 0 0 0 0

And debugging the queue I see these messages all for vlan 82 (a one second debug has hundreds of these messages)
ICMP-Q:Dropped redirect disabled on L3 IF: Local Port Fwding L3If:Vlan82 L2If:FastEthernet1/0/11
ICMP-Q:Dropped redirect disabled on L3 IF: Local Port Fwding L3If:Vlan82 L2If:FastEthernet1/0/6
ICMP-Q:Dropped redirect disabled on L3 IF: Local Port Fwding L3If:Vlan82 L2If:FastEthernet1/0/1

The addresses listed in the debugs are all correct valid addresses with valid routes.
It seems that the packets are sent to the cpu thinking that there should be a redirect , yet since it is disabled the cpu then drops the packets.


Here is the interface vlan configuration
interface Vlan82
ip address 82.113.194.2 255.255.255.224
no ip redirects
no ip unreachables
no ip proxy-arp
end


Any thoughts?
I am running 12.2.35SE2

Thanks
Brian



-----Original Message-----
From: Jared Mauch [mailto:***@puck.nether.net]
Sent: lunedì 7 maggio 2007 20.04
To: Brian Turnbow
Cc: cisco-***@puck.nether.net
Subject: Re: [c-nsp] 3750 high cpu from icmp
Post by Brian Turnbow
Besides redesigning to avoid icmp redirects anyone have any ideas?
Can you make sure that all your routers have the following
on their "IP" (routed) interfaces:?

no ip redirects
no ip proxy-arp

This should help solve the problem.

These two should really be default these days.

- Jared
--
Jared Mauch | pgp key available via finger from ***@puck.nether.net
clue++; | http://puck.nether.net/~jared/ My statements are only mine.

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Adrian Chadd
2007-05-14 09:07:04 UTC
Permalink
Post by Brian Turnbow
Wanted to post an update on this in case anyone else ever has problems.
The only way I found to resolve this issue was to move traffic onto different interfaces , removing the router on a stick routing.
Did you stick the port into a SPAN group and get a traffic dump? See if some
other device is actually sending your 3750 ICMP redirects?




Adrian

_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Brian Turnbow
2007-05-14 09:29:53 UTC
Permalink
Yes and there were none.
The icmp queue debugs also list source / destination macs and Ips where you can see that it would be the 3750 that needs to generate a redirect.

Brian

-----Original Message-----
From: Adrian Chadd [mailto:***@creative.net.au]
Sent: lunedì 14 maggio 2007 11.07
To: Brian Turnbow
Cc: cisco-***@puck.nether.net
Subject: Re: [c-nsp] 3750 high cpu from icmp
Post by Brian Turnbow
Wanted to post an update on this in case anyone else ever has problems.
The only way I found to resolve this issue was to move traffic onto different interfaces , removing the router on a stick routing.
Did you stick the port into a SPAN group and get a traffic dump? See if some
other device is actually sending your 3750 ICMP redirects?




Adrian


_______________________________________________
cisco-nsp mailing list cisco-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Continue reading on narkive:
Loading...