all nuances of use

Short description

Working with PCIe involves understanding the possibility of port branching, including bifurcation which refers to the branching of a PCIe slot into multiple channels. The correct implementation of bifurcation is delicately nuanced and requires support from the host. PCIe switches offer an alternative, powerful option. Although they are expensive and can be difficult to debug, documentation is available for Diodes Incorporated and Pericom-acquired PCIe products whilst Broadcom’s products tend to be absent of openness. Accessories utilizing USB3 cables and connectors for PCIe channels, such as slot expanders and NVMe adapters, are ideal for using PCIe and easy to find but require careful attention to pinout and power adapters.

all nuances of use

So, if you have read previous articles on this topic (

part 1

,

part 2

), then the basics of working with PCIe are already familiar to you and perhaps you have a certain idea about using this standard in your project. It is possible to wish to equip a single-payer with several cheap but powerful ones

WiFi cards with PCIe

for wardriving, adding a second NVMe SSD to a laptop instead of an unused Ethernet controller or connection

full-size video card

to Raspberry Pi via

threaded adapter

. Whatever you are going to implement, make sure that you will be familiar with all the possibilities of using PCIe.

▍ PCIe port branching

If you’ve ever worked with PCIe, you might have heard the term “bifurcation,” especially among miners and PC tinkerers. Bifurcation refers to the branching of a PCIe slot into multiple channels and as you can imagine is a very interesting option for electronics. You don’t actually need any additional hardware here – you just need to add a REFCLK buffer. As you can see, this signal is still needed by each port – but you can’t just physically spread one clock diff pair to all slots at the same time, as this results in stubs and therefore signal mapping.

The REFCLK buffer chip receives the clock signal from the host and reproduces a number of identical copies of it, which can then be used individually. You could see x16 branching up to four NVMe card slots in the network – in this case, you can always find a REFCLK buffer chip somewhere in the corner of the card. In an ideal scenario, this is enough to get more PCIe lanes out of one.

In fact, the correct implementation of bifurcation is a very delicate matter. This requires support from the host – and such support turns out to be quite situational in the sense that only some chipsets are able to provide bifurcation, and in most motherboards, developers for some reason prefer to block it in software. Even if you build your own PCIe host with a PVM and all the flexibility, your PVM may simply not support bifurcation for internal reasons. What’s more, you can’t just get sixteen x1 channels from a single x16 slot – branching into a combination of four x4 channels is usually supported, or perhaps two x4 channels in the case of an x16 slot with an actual width of x8.

▍ Alternative ways of branching

However, there is an alternative and that is PCIe switches. These are essentially powerful chips that present a PCIe device interface to your host and multiple PCIe host interfaces to all your devices. In terms of operation, they are comparable to Ethernet switches, except that the PCIe switch lines have the role of either host or device, and the Ethernet ports of the switch are usually all equivalent. These switches range in size from a QFN package for the RP2040 (a 1:2 switch in the case of the ASM1182) to the size of an x86 processor. They always require strong cooling, while they are expensive and consume a lot of energy. If you are looking for bifurcation cards on, say, AliExpress, you will inevitably come across ones equipped with PCIe switches.

The benefit of using them is that they don’t require much support from the host because they look like any regular PCIe device. And the disadvantage, again, is that they are expensive and rarely accompanied by documentation. For example, I was looking for PCIe 1:4 switch chips and came across the ASM1184 1:4 controller, which creates four x1 channels from one. It was exactly what I needed and through a google search I found out that this chip is used in many Chinese motherboards as well as desktop motherboards from manufacturers like Gigabyte. At the same time, such boards are sold even in stores, for example

Walmart

.

The ASM118x chips themselves (1:4 for ASM1184 and 1:2 for ASM1182) can also be easily found on AliExpress – the main problem is that they have no documentation. Technically, the circuit of real devices is the necessary basic minimum, because it is implied that these chips should work autonomously without external control. I found one documentation for ASM1182 and my friend found one for ASM1184. However, components such as external control I2C registers remain a mystery.

So, although chips like the ASM1184 look attractive, are inexpensive, and are available on AliExpress both as individual chips and as part of finished products, debugging them can cause difficulties. Fortunately, there is one western manufacturer that I know of that will save you time if you’re going to buy PCIe switches – Diodes Incorporated, which makes a line of Pericom-acquired PCIe products. You can safely download the documentation for their PCIe switches. You may also come across Broadcom switches, but as you might guess, their openness is typical of Broadcom products – that is, absent.

▍ The fruits of mining

It is also very interesting how ASMedia markings are painted over in some AliExpress product items

If you want to run a PCIe cable, and you are quite satisfied with the x1 channel, then you can get by with the USB3 standard – in fact, this is a pretty good means of transmission. It has two high-speed USB3 diffs and, given that USB3 is quite a demanding interface in terms of integrity requirements, the cables designed for it are excellent for PCIe. These cables also have a pair of USB2, which the 100MHz REFCLK pair pairs nicely with. In general, USB3 cables and PCIe x1 channels are literally made for each other. Realizing this, mining hardware manufacturers have flooded the market with all kinds of accessories that run PCIe channels through USB3 cables and connectors – slot expanders, PCIe switches, NVMe adapters, and more. And since the popularity of mining has fallen, all these accessories have become available at bargain prices.

A typical riser board consists of a x1 connector board with a USB3 socket that connects to the motherboard, a x16 slot board with a corresponding USB3 socket, and additional power inputs, sometimes with regulators. The third component is a USB3 cable with USB-A connectors on both ends. For an electronics designer designing an unusual device with PCIe, using these accessories can be tempting, as USB3 sockets and cables are easy to find in many variations.

And in practice, you’re sure to succeed – if there’s any proof of the ability to use PCIe in this way, it’s that similar mining expansion boards have worked well for countless miners 24/7/365. However, here you need to beware of some covers. For example, there is one nuance with the pinout, which is rarely found – the high-speed TX and RX diff pairs remain on the USB3 terminals, but there is no guarantee about which PCIe pair and which USB3 pair goes to which.

Although more importantly, the types of signals applied to the +5 V and GND wires (yes, this is a separate wire) of the USB2 part can differ – and this actually happened. A friend of mine assembled a device with these wires, and her PCIe risers turned out to have PEWAKE and PERST signals – as a result, this scheme worked perfectly. However, later in the course of a Russian miner forum investigation into riser compatibility, some riser and switch boards were found where the +5V line was not transmitting PERST or PEWAKE, which you might expect if designed for the particular riser you have. and 3, 3 or even 12 st.

If you’re designing something for risers and relying on either “slot x1 motherboard” or “slot x16” boards from a typical riser, you’d better double-check that the ones you’re using actually match your intended pinout. Using the wrong riser can cause a dangerous situation. It’s sad, but despite the fact that there is almost a certain standard for PCIe signal transmission for risers with convenient cables, no one really follows it.

Also, be sure to avoid the power adapters that sometimes come with such risers – all of these SATA power – PCIe 6-pin have a serious mismatch between input and output power. If there’s a problem with mining boards like this, it’s the use of these adapters, which can be an obvious source of ignition if your PCIe card suddenly draws a bit more power than a cheap SATA adapter’s power connector can handle.

▍ All relevant connectors

The approach of using mining accessories in the implementation of PCIe devices is not the only one. If you find an interesting chip that connects via PCIe, or want to extend the discovered PCIe line with suitable means, start designing your own PCB. Today, you have a choice between full-size slots for desktop applications or M.2 slots for mobile devices – and if you’re targeting older industrial hardware, mPCIe will also do.

For desktop applications, there are many PCIe connectors to choose from. To connect to such a connector, you will need a printed circuit board with a thickness of 1.6 mm. This thickness is fortunately the standard thickness of PCBs you can get from any factory and usually corresponds to the cheapest products. At the same time, you need to round the edge of the board, which is inserted into the socket – often this service can be ordered from the factory for a small fee. But you can do it yourself with a file or an x-acto knife with a spare blade.

As for mobile slots, I’ve covered the M.2 topic extensively before. A simple guide for M.2 is to use 0.8mm ENIG coated PCBs for the card and be sure to get a stencil if you are adding an M.2 slot to the board. Then choose a convenient connector form factor – A+E for WiFi cards, E-connector for connecting WiFi cards, M-key slots for PCIe cards up to x4 and B+M for PCIe cards up to x2 and for compatibility with B-key (WWAN) and M-key (SSD) slots. There are many M.2 sockets available, as well as slot and card templates. For mPCIe, a 1 mm printed circuit board is suitable, of course with an ENIG coating.

Whether you’re building a mobile/desktop PCIe card or a connector, the guidelines for disparity, required signals, and high compatibility between channel widths and generations remain the same. Whatever PCIe device you’re designing, it’s not that difficult – and with PCIe’s fault tolerance, which helps us in our high-speed runs, you’re sure to hit your project goals.

Telegram channel with prize draws, IT news and posts about retro games 🕹️

Related posts