1. Overview

This application note discusses the network backhaul found on Per Vices SDRs. This involves the constaints imposed on sending IQ samples in a packetized format/frame between the host system and SDR device containing the FPGA, DACs and ADCs. To see how we packetize data over our network backhaul (10/40/100 GBASE-R SFP/qSFP+ ports), see the application note on data formats. For information on host system’s appropriate for our SDRs, see the high data throughput considerations reference note.

2. Bandwidth, Sample Rates, and Networking

2.1 Backhaul Bandwidth

The performance of Per Vices radio products is constrained by the maximum backhaul bandwidth, which is limited to the line rate of the bus connecting the host machine and the Per Vices SDR. For Crimson, this comes in the form of two SFP+ ports that implement 10GBASE-R, with a maximum transfer rate of 10Gbps. For the stock 1GSPS Cyan product, this includes four qSFP+ ports each implementing 40GBASE-R, operating at 40Gbps each, and the high bandwidth 3GSPS Cyan product implements four 100GBASE-R ports at 100Gbps. We can say the following:

Crimson Backhaul bandwidth:

\[ BW_{bh, Crimson} = 10Gbps/ port \]

Cyan 1GSPS Backhaul bandwidth:

\[ BW_{bh, Cyan1G}= 40Gbps/ port \]

Cyan 3GSPS Backhaul bandwidth:

\[ BW_{bh, Cyan3G}= 100Gbps/ port \]

Note

The backhaul bandwidth for a given port is entirely determined by the link implementation (whether it’s 10/40/100GBASE-R) between the host system PC and SDR.

2.2 Converter Bandwidth

Each RFE channel includes its own convertor (DAC/ADC), which uses complex data. The digital bandwidth associated with each convertor is calculated by multiplying the sample rate of each convertor on each channel with the bit resolution of each sample (S). Thus, we can say:

Convertor Bandwidth:

\[ BW_{convertor} = f_{sampling} * S_{bitwidth} \]

2.3 Complex Samples

Within our SDRs, each RFE chain is fully independent, and sample using complex In-phase and Quadrature (IQ) sampling. For a brief background on these types of signals and sampling, see our reference note On IQ Signals. For our Crimson TNG and Cyan 1GSPS, the complex convertor sample bit width is 32bits (16 bits I and 16bit Q). For Cyan 3GSPS, we support 32-bit Tx samples (16-bits I and 16-bits Q), and 24-bit Rx samples (12-bits I and 12-bits Q).

Note

The section on Data Formats provides additional information on the sample and data format used by Cyan.

2.4. Maximum Sustained Bandwidth

To calculate the maximum sustained bandwidth that may be supported by each channel, we need to ensure that the aggregate sum of the total convertor BW supported by each BW is less than 98% of the total backhaul BW, as shown in subsection 2.1.

Note

The 98% figure accounts for protocol overhead. In particular, if sending packets with an average size of 8974 bytes, we expect to use around 78 bytes for header information (Ethernet/IP/UDP/VITA headers/trailers), with the remainder allocated for data payload (IQ signal data). In addition, routing networking overhead (ARP/etc), all conspire to reduce the total available BW to around 98% of the specified available BW.

Thus, we can say that the condition for our radios to sustain maximum capacity backhaul data flow is:

Sustained Backhaul Capacity Condition:

\[ BW_{convertor, sum} < 0.98 * BW_{bh} \]

For Crimson TNG, each backhaul supports 2 RFEs. For Cyan 1GSPS, each backhaul supports up to 4 RFEs, depending on the specific configuration. For Cyan 3GSPS, each backhaul supports up to 2 RFEs. For Cyan EC, each backhaul supports 1 RFE, which further supports up to 6 dedicated sub-bands channels (which are channelized on the convertor).

3. Specific Example Calculations of SDR Backhaul

For Crimson TNG (325MSps, 2 x 10GBASE-R), we are able to sustain a maxium backhaul bandwidth of 20Gbps. Similarly, each qSFP+ port on our Cyan products can sustain a throughput of 40 Gbps (Cyan 1GSps, 4 x 40GBASE-R) or 100 Gbps (Cyan 3GSps, 4 x 100GBASE-R) backhaul bandwidth, for a total of 160 or 400 Gbps of throughput bandwidth across the four qSFP+ ports, respectively. Essentially, we need to ensure all the RFEs/convertors that are mapped to each SFP+ or qSFP+ port does not exceed the maximum backhaul bandwidth. The examples below should make this clearer.

3.1 Crimson TNG

Crimson TNG uses DACs/ADCs at 325MSps (megasamples/second), and uses a complex sample bit widths of 32-bits/ sample (S). If we are to substitute these numbers into the convertor bandwidth equation, we have:

\[ BW_{convertor}= f_{sampling} * S_{bitwidth} = 325MSps * 32 bits/ S = 10.4 Gbps \]

Thus, this does not satisfy the sustained backhaul capacity condition, since we arrive at:

\[ BW_{convertor} = 10400 Gbps > 0.98 * BW_{bh, Crimson} = 9.8 Gbps/port → FAIL \]

If we wanted to ensure that our backhaul is sustained, we would need to decimate by at least 2 in order to reduce the convertor BW. Substituting into our equation:

\[ BW_{convertor} = 1/2 * f_{sampling} * S_{bitwidth}= 162.5 MSps *32 bits/ S = 5.2 Gbps \]

And this, of course, does satisfy the sustained backhaul capacity condition.

3.2 Cyan 1GSPS

Cyan 1GSPS uses DACs/ADCs at 1GSps (gigasamples/second) and uses a complex sample bit widths of 32-bits/ sample (S) sent over 4 x 40Gbps qSFP+ ports. If we are to substitute these numbers into the convertor bandwidth equation, we have:

\[ BW_{convertor}= f_{sampling} * S_{bitwidth} = 1GSps * 32 bits/ S = 32 Gbps \]

Thus, this satisfies the sustained backhaul capacity condition:

\[ BW_{convertor} = 32 Gbps < 0.98 * BW_{bh, Cyan1G} = 39.2 Gbps/ port → PASS \]

However, if we were to use 2 channels on one qSFP+ port, then we would have the following:

\[ BW_{convertor}= 2 * f_{sampling} * S_{bitwidth} = 2* 1GSps * 32 bits/ S = 64 Gbps \]

Thus, this does not satisfy our sustained backhaul capacity condition. Again, we could ensure that it does by decimating each channels sample rate by a factor of 2.

\[ BW_{convertor} = 1/2 * f_{sampling} * S_{bitwidth}= 1 GSps * 32 bits/ S = 32 Gbps \]

Alternatively, we could stream one channel at 1GSps (CH A), and the other channel at, say, 200 MSps (CH B), such that the sum of the two sample rates satisfies the sustained backhaul capacity condition:

\[ BW_{convertor, sum} = BW_{convertor, CH A} + BW_{convertor, CH B} = f_{sampling, CH A} * S_{bitwidth} + 1/5* f_{sampling, CH B} * S_{bitwidth} \]
\[ BW_{convertor, sum} = 1GSps * 32 bits/S + 200MSps * 32 bits/S = 38.4 Gbps \]

Thus, this satisfies the sustained backhaul capacity conditional as well:

\[ BW_{convertor} = 38.4 Gbps < 0.98 * BW_{bh, Cyan1G} = 39.2 Gbps/ port → PASS \]

3.3 Cyan 3 GSPS

The Cyan 3GSps product does not support sample rate decimation; the digitized data is sent directly from the convertor to the backhaul. As a result, we note the following:

For transmit (Tx), the DAC supports 32 bit complex samples (16-bit I, 16-bit Q). Thus when transmitting a signal at full rate (3GSps) on one channel, we have:

\[ BW_{convertor} = f_{sampling} * S_{bitwidth} = 3GSps * 32 bits/ S = 96 Gbps \]

Thus, this satisfies the sustained backhaul capacity condition relevant for Cyan 3GSps:

\[ BW_{convertor} = 96 Gbps < 0.98 * BW_{bh, Cyan3G} = 100Gbps/ port → PASS \]

For receive (Rx), the ADC supports 24-bit complex samples (12-bit I, 12-bit Q). Thus when receiving a signal at full rate on one channel, we have:

\[ BW_{convertor} = f_{sampling} * S_{bitwidth} = 3GSps * 24 bits/ S = 72 Gbps \]

This, of course, also satisfies the sustained backhaul capacity condition.

4. Note on Sample rate on Crimson TNG

The base sample rate for Crimson TNG Converters is 325MSPS. All of the Rx channels (A, B, C, D), Tx channel A and Tx channel B are sample at that rate. Due to FPGA limitations, the maximum supported sample rate on Tx channels C and D is \(\frac{1}{4}\) of this value; or 80.56640625MSPS (325 MSPS/4). Depending on your use case, custom FPGA images may be developed that allow full rate sending over all Tx channels, but this functionality is not included in the stock Crimson TNG image.

5. Note on Cyan sample rates

The default Cyan 1GSPS platform is capable of supporting 1GSPS streaming over qSFP+ channels. It also includes two 4GB DDR4 banks, to ensure ample buffering. However, the full duplex transfer rate of the DDR4 controller supports approximately 60Gbp transfer rate, which is slightly lower than what is required to simultaneously stream Tx data between two channels.

For full rate streaming across Tx channels we have FPGA images available that do not use the DDR4 banks. This results in a smaller available buffer, but lower latency and full rate streaming support. For streaming between deterministic devices, this is the preferred mode of operation.