UNSOLVED

RRR

updated

15 years ago

R

RRR

6 Operator

•

5739 Posts

0

12594

February 8th, 2012 07:00

Running an ISL at 8Gb produces lots of errors, 4Gb is error free. Don't understand why.

We have 2 locations, separated by 610 meters and 1245 meters dark fiber. The switches are Brocade DS-300B with 8Gb long wave (10 km) SFPs.

When the switch ports run at 8Gb I see a lot of errors and communication between 2 VNX machines is very bad. Mirrorview doesn't show any progress.

When I set the ports down to 4Gb, the ISLs are error free and mirrorview is running smoothly.

The connections between the 2 300's in each fabric is a single ISL. We have 2 fabrics, so 4 switches and 2 VNX's.

We'd like to run the ISL @8Gb, but we keep getting the errors on the link. We tried new patch cables (9 micron of course) and other outlets on the patch panel, but that wasn't it.

Does anyone have an idea of what this might be ?

  • dynamox

    11 Legend

    •

    20419 Posts

    •

    87439 Points

    2221

    0

    Posted February 8th, 2012 07:00

    is there such a thing as buffer credits on Brocade ?

  • dynamox

    11 Legend

    •

    20419 Posts

    •

    87439 Points

    2221

    0

    Posted February 8th, 2012 08:00

    RRR,

    ISL ports set to LD ?

    portCfgLongDistance

  • henrypan2

    13 Posts

    2221

    0

    Posted February 8th, 2012 08:00

    Yes dynamox:>)

    Henry

  • 2221

    0

    Posted February 8th, 2012 12:00

    Still no effect.The errors only dissapear when changing to normal mode (giving me 8 b2b credits) and 4Gb fixed. absolutely no errors then. when changing to 8Gb, change to LD mode (giving me 46 b2b credits) or not and even change portcfgfillword to 3, does not resolve anything. only way of error free ISL is 4Gb Normal mode. I am the colleague of RRR

  • dynamox

    11 Legend

    •

    20419 Posts

    •

    87439 Points

    2221

    0

    Posted February 8th, 2012 13:00

    and it's consistent behavior on both fabrics ?

  • christopher_ime

    6 Operator

    •

    1962 Posts

    2221

    0

    Posted February 8th, 2012 22:00

    Just curious, what are the errors from porterrshow?  Are they by chance class 3 discards (disc c3)?  There is a Primus that may be related:

    emc278061: "Brocade B-Series: 8Gb switches in Fabric OS 6.x ISL connected to Access gateway shows errors in class 3 discards disc c3 counter"

    Also, just want to make sure that you are compliant to the support matrix regarding ISL's and are running at least v6.1.2b.

    ISL.JPG

    Also a good general read may be the "Extended Distance Technologies TechBook",

    https://elabnavigator.emc.com/vault/pdf/Distance.pdf

    This will also talk about the "Extended Fabric" license which is required, and a short chapter on "Data buffering and flow control" which might help troubleshoot.

  • 2221

    0

    Posted February 9th, 2012 00:00

    and the other side when running on 8Gb:

    er_enc_in 36 Encoding errors inside of frames

    er_crc 19 Frames with CRC errors

    er_trunc 0 Frames shorter than minimum

    er_toolong 0 Frames longer than maximum

    er_bad_eof 3 Frames with bad end-of-frame

    er_enc_out 17043 Encoding error outside of frames

    er_bad_os 17045 Invalid ordered set

    er_rx_c3_timeout 0 Class 3 receive frames discardedout

    er_tx_c3_timeout 0 Class 3 transmit frames discardeeout

    er_c3_dest_unreach 0 Class 3 frames discarded due to unreachable

    er_other_discard 0 Other discards

    er_type1_miss 0 frames with FTB type 1 miss

    er_type2_miss 0 frames with FTB type 2 miss

    er_type6_miss 0 frames with FTB type 6 miss

    er_zone_miss 0 frames with hard zoning miss

    er_lun_zone_miss 0 frames with LUN zoning miss

    er_crc_good_eof 16 Crc error with good eof

    er_inv_arb 0 Invalid ARB

  • felipon1

    96 Posts

    2219

    0

    Posted February 9th, 2012 00:00

    Hi,

    According to your last post, we can see er_crc_good_eof in both ends:

    er_crc_good_eof 58 Crc error with good eof

    er_crc_good_eof 16 Crc error with good eof

    So I suppose that there must be some element generating the errors in that link. The more speed the more sensitive and affected the switchs are by dust, a bad connection, sfp, etc.

    Do you get these errors in both fabrics or just in one? If just in one, you can try to locate the faulty element by moving connections from one fabric to the other, with all that that implies...

  • 2221

    0

    Posted February 9th, 2012 00:00

    thanks guys again, here are the errors of one side:

    er_enc_in 67 Encoding errors inside of frames

    er_crc 59 Frames with CRC errors

    er_trunc 0 Frames shorter than minimum

    er_toolong 0 Frames longer than maximum

    er_bad_eof 1 Frames with bad end-of-frame

    er_enc_out 375743 Encoding error outside of frames

    er_bad_os 375747 Invalid ordered set

    er_rx_c3_timeout 0 Class 3 receive frames discarded due to timeout

    er_tx_c3_timeout 0 Class 3 transmit frames discarded due to timeout

    er_c3_dest_unreach 0 Class 3 frames discarded due to destination unreachable

    er_other_discard 0 Other discards

    er_type1_miss 0 frames with FTB type 1 miss

    er_type2_miss 0 frames with FTB type 2 miss

    er_type6_miss 0 frames with FTB type 6 miss

    er_zone_miss 0 frames with hard zoning miss

    er_lun_zone_miss 0 frames with LUN zoning miss

    er_crc_good_eof 58 Crc error with good eof

    er_inv_arb 0 Invalid ARB

  • Richard_Butler

    2 Intern

    •

    84 Posts

    879

    0

    Posted February 9th, 2012 03:00

    Hello RRR,

    Did you look at setting the 8Gb/s fillword with the command: portCfgFillword ?

    Best regards.