The Impact of Hardware Architecture and Software

Netgate has a popular set of secure networking appliances ranging from the SG-1100 to the XG-1541. Buyers can quickly and easily differentiate them on the basis of physical interconnect, CPU, and price.

But, there is another purchase factor that is less obvious, yet quite important. That factor is the product’s internal architecture. You may (or may not) be aware that internal architecture differs significantly across the SG-1100, SG-3100, SG-5100, XG-7100 and XG-1537/XG-1541.

Architecture encompasses processor, memory, storage, interface types and counts, absence or presence of a switch fabric, and more. It becomes very important when considering questions like:

  • How many switch port connections do I need?
  • How much traffic needs to be passed locally between those connections?
  • How much traffic needs to leave the premises for the internet at large, or tunneled through the internet over a secure VPN connection?
  • What is the nature of my traffic - voice, data, video, or all of the above?
  • Will high speed connections run sustained or bursty traffic loads?
  • How much of my traffic is encrypted?

Let’s consider these questions by looking at a few secure networking use cases that routinely surface in customer conversations. And, to make it digestible, let’s break the use case responses into two stages:

  • First, we’ll just differentiate appliances based on a few key hardware architecture differences
  • Then, let’s see what changes if you were to equip these hardware architectures with TNSR™ software instead of pfSense® software

As a backdrop, the table below provides a quick and simple comparison of Netgate appliance architectures. Only base configurations are represented - and for the purpose of this blog, the diagrams are simplified to just processor, switch, and ethernet networking ports. There are, of course, other ports - USB, Mini PCIe, etc. For more detail, please visit our product pages within store.netgate.com.

Hardware Architecture

Use Case 1:

“I work from home. I need a VPN connection back to my main office. My Internet speed ranges from 100 Mbps to 300 Mbps.”

Our recommendation: get an SG-1100. It packs a punch as an internet access firewall for home or remote offices where 400-500 Mbps of firewall-protected traffic is sufficient. Many home and small office networks have interconnection speeds that realistically fall in this range. You’d be hard pressed to find a better desktop attractive, low heat, silent, feature-rich firewall / router at its price point.

Use Case 2:

“We work in a small office with under 50 employees. We need a VPN that supports 150 Mbps connection to our corporate office. Observed internet speeds are typically around 500 Mbps bidirectional. We also need a number of ethernet switch ports. Each connection doesn’t really require a lot of bandwidth, although a couple need more ‘guaranteed’ or, at least, ‘prioritized’ throughput. In the aggregate, they can comfortably share a 500 Mbps uplink.”

The SG-3100 is a great option here. Excellent port density for a small footprint desktop package, dual 1 Gbps WAN and solid inter-port traffic movement - for up to four switch ports that share a robust 2.5 Gbps switch fabric.

Use Case 3:

“We are on the receiving end of the work-at-home and small office users defined in Use Cases 1 and 2 above. Their respective SG-1100 and SG-3100 devices will connect to our corporate office device, which will need to support 800 Mbps and VPN termination.”

The SG-5100 is the ticket. With up to six fully-independent 1 Gbps Ethernet connections, you’re getting a very flexible 1 Gbps Layer 3 router / firewall at a bargain price. Provision each port in a LAN or WAN direction as you like. Additionally, the SG-5100’s Intel Atom C3558 2.2 GHz CPU - with QuickAssist, AES-NI, and SHA instructions (which helps in OpenSSL) - provides the boost needed to crank through encrypted traffic processing.

Use Case 4:

“I need a 1-3 Gbps WAN connection. I have a considerable number of switch ports that will need to move a fair amount of traffic between themselves locally. Local traffic can easily burst past 1 Gbps with packets of all sizes.”

For this use case, the XG-7100 is the starting point. Equipped with 10 Gbps SFP+ ports, it is easily differentiated from the SG-5100 (and below) products. Further, with eight ports tied to dual 2.5 Gbps switch links, burst potential across ports can peak up to 500% more than if they were tethered to a single 1 Gbps switch port. This is especially useful for large file transfer activity - common to development teams or IT departments responsible for large scale backups.

Looking for 3-10 Gbps and a significant increase in active connections? Move up to our XG-1537 or XG-1541. At these speeds, we also suggest a direct consultation with our sales team. They know our products inside and out. With a good understanding of your specific networking needs - they can help you design the right network at minimal cost.

So, there you have a run down of how to differentiate Netgate hardware appliances on the basis of internal product architectures, and how each architecture adds unique value.

Now, let’s bring software into the selection process to further pinpoint the right Netgate appliance for your needs. Hopefully, you’ve heard we can factory install our higher-end Netgate appliances with pfSense or TNSR software. That is a key distinction to consider - especially as bandwidth and/or encryption needs begin to rise.

Why? As popular, capable, and reliable as pfSense software is - it has its limits. Notably, as link speeds increase, packet sizes drop, and encryption escalates - pfSense software comes under strain, and ultimately throttles the full use of your hardware. TNSR has far more ability to push hardware to its limits (ours or anyone else’s), where pfSense can be a ‘software bottleneck’ that often holds users back from exploiting the full power of their hardware investment.

So where does pfSense start to become a bottleneck? You’ll see it 1) under true IMIX traffic load, 2) as the amount of encrypted traffic rises, and 3) especially if both occur. Let’s take IMIX first.

What is IMIX? Internet Mix (IMIX) is a mix of packet sizes used to emulate real-world traffic conditions experienced by network equipment - like routers, switches and firewalls - in traffic tests. The Simple IMIX is seven 40 byte packets, four 576 byte packets, and one 1500 byte packet. So how should it factor into a firewall or router purchase decision? That depends on your particular secure networking scenario. Let’s take two extremes:

Scenario 1:

“All I do read is my email, surf the net, and watch movies from Netflix.”

With the exception of message download (where most mail clients make you wait a bit, and it gets worse if there are attachments) email reading is largely gated by your reading speed, so we can largely throw that out, even if you speed read. Internet surfing? Whether browsing or searching, it mostly comes down to fetching pages of text, images, and some music and/or video streaming files - most of which can be managed either by your human consumption rate or buffering. Now, streaming a two hour film like Avengers: Infinity War? Well, that content is likely delivered to you in 1500 byte packets. That means the firewall / router has plenty of time to process a packet header, rest while a long packet payload follows, and get ready for the next header. So in this scenario, your firewall/router is not seeing a firehose of Simple IMIX traffic - so it is probably able to saturate a 1 Gbps link, and even a good chunk of a 10 Gbps link. We often hear users claim 6, 7, even 8 Gbps through pfSense software. That is entirely possible as long as packets remain large, unencrypted, and with a relatively simple firewall setup, e.g., no IDS/IPS or other traffic inspection rules activated.

Now, let’s go to the other extreme - and let’s keep the scenario pretty light from a security perspective. Increasing security workload just doubles down on the same argument.

Scenario 2:

“I run a relatively limited set of firewall access control lists (ACLs), no IDS/IP or web filtering. However, rather than the occasional two-hour stream of 1500 byte video frames, I see a widely varying mix of voice, data, and video traffic throughout the day.”

pfSense software (on inexpensive hardware) is likely to peak at around 200-300 Mbps of Simple IMIX traffic. If you are a home user content with around 300 Mbps of actual network bandwidth, you’ll be no worse for the wear. But, if you are a work at home enterprise-class organization employee - expected to get things done under time pressure - the last thing you want is network throughput congestion from your firewall, especially if your company paid for a faster network connection to keep your productivity high.

This is where TNSR pays big dividends. TNSR uses Vector Packet Processing (VPP) to push extreme packet performance through commercial off-the-shelf (COTS) hardware. In other words, what used to require high-priced ASIC-based hardware to achieve serious throughput can now be had for a fraction of the price of special-purpose hardware. You can learn more about TNSR here. Another key differentiator for TNSR is a RESTCONF API, but this blog is focused on hardware architecture and packet processing - so we’ll come back to that topic is a separate blog.

The first Netgate appliances to feature TNSR are the SG-5100, XG-1537, and XG-1541. We plan to share more extensive test results in a separate blog post, but for now - consider the following IPsec encrypted IMIX traffic numbers (based on AES-GCM-128 encoding) to understand the power of TNSR:

IMIX Traffic

OK, admittedly that’s a lot to digest. So let’s summarize a few key takeaways:

  • First. Remember all figures are based on Simple IMIX traffic
  • TNSR and pfSense are at parity with simple L3 forwarding (pf packet filter disabled), irrespective of whether TNSR single or multi-threaded
  • Step up the packet processing workload by adding firewall (pf packet filter enabled), and TNSR takes a 1.7-2.3X advantage as CPU and memory increases
  • Step up to an even tougher packet processing workload, IMIX traffic encrypted with AES-GCM-128, and allow both to take advantage of inherent SW or HW acceleration where they can, TNSR has a 2.2-9.4X advantage
    • In fairness, pfSense is able to exploit AES-NI, but not QAT. TNSR can take full advantage of QAT.
  • Note: TNSR is held to single threaded processing in all test cases except SG-5100 multi-interface. Adding threads takes the governor off and TNSR soars.
  • Also note: all figures are unidirectional, except TNSR on multi-interface SG-5100, which are bi-directional.

There you have it: each product shines under an appropriate light. But a purchase decision needs to consider use case, underlying product architecture, and software stack all three. If you are on the low- or high-end of the product line, the choices are quite clear. In the middle (SG-3100 to SG-5100 to XG-7100), think carefully about the ‘use case / switch / software stack interplay’ to get the biggest bang for your buck.

And if you’re running a business, think carefully about a technical assistance subscription. While both pfSense software and TNSR software are easy to use and robust, networks often throw curve balls. We’re there to field them - and our customer satisfaction rating is stellar. Also note, a technical assistance subscription is an additive charge with pfSense, whereas it is built in to either perpetual or subscription licensing with TNSR. As always, if you have questions, contact us. We love to talk about networking!