Most of TCP woes comes from high-bandwith latency-sensitive stuff like HFT and video, but TCP isn't particularly good for low-bandwidth high-latency networks either (e. g. NB-IoT with 10 seconds worst case RTT):
- TCP will waste roundtrips on handshakes. And then some extra on MTU discovery.
- TCP will keep trying to transmit data even if it's no longer useful (same issue as with real-time multimedia).
- If you move into a location with worse coverage, your latency increases, but TCP will assume packet loss due to congestion and reduce bandwidth. And in general, loss-based congestion control just doesn't work at this point.
- Load balancers and middleboxes (and HTTP servers, but that's another story) may disconnect you randomly because hey, you haven't responded for four seconds, you are probably no longer there anyway, right?
- You can't interpret the data you've got until you have all of it - because TCP will split packets with no regards to data structure. Which is twice as sad when all of your data would actually fit in 1200 bytes.
And TCP wants to pretend that all your data arrives as a linear stream one after the other; and keeps you from seeing package n+1, if you haven't received package n, yet.
That's useful for some things, but often you can make use of package n+1, even when package n hasn't arrived, yet.
For example, when you are transferring a large file, you could use erasure encoding to just automatically deal with 5% package loss. (Or you could use a fountain code to deal with variable packet loss, and the sender just keeps sending until the receiver says "I'm done" for the whole file, instead of ack-ing individual packages.
Fountain codes are how deep space probes send their data back. Latency is pretty terrible out to Jupiter or Mars.)
> And TCP wants to pretend that all your data arrives as a linear stream one after the other;
That’s one of the reasons people use TCP.
> For example, when you are transferring a large file, you could use erasure encoding to just automatically deal with 5% package loss.
It’s never that simple. You can’t just add some erasure coding and have it automatically solve your problems. You now have to build an entire protocol around those packets to determine and track their order. You also need mechanisms to handle the case where packet loss exceeds what you can recover, which involves either restarting the transfer or a retransmission mechanism.
The number of little details you have to handle quickly explodes in complexity. Even in the best case scenario, you’d be paying a price to handle erasure coding on one end, the extra bandwidth of the overhead, and then decoding on the receiving end.
That’s a lot of complexity, engineering, debugging, and opportunities to introduce bugs, and for what gain? In the file transfer example, what would you actually gain by rolling your own entire transmission protocol with all this overhead?
> Fountain codes are how deep space probes send their data back. Latency is pretty terrible out to Jupiter or Mars.)
Fountain codes and even general erasure codes are not the right tool for this job. The loss of a digital packetized channel across the internet is very different than a noisy analog channel sent through space.
> It’s never that simple. You can’t just add some erasure coding and have it automatically solve your problems. You now have to build an entire protocol around those packets to determine and track their order.
Yes, and you'd want to mostly build this extra protocol only once, and stick it into a library.
> You also need mechanisms to handle the case where packet loss exceeds what you can recover, which involves either restarting the transfer or a retransmission mechanism.
Or you can use a fountain code, and just keep transmitting.
> The number of little details you have to handle quickly explodes in complexity. Even in the best case scenario, you’d be paying a price to handle erasure coding on one end, the extra bandwidth of the overhead, and then decoding on the receiving end.
Well, you can also use a simpler mechanism: you transmit all the packages from A to B once, then at the end B tells A which packages get lost, and A sends those again. Repeat until you have everything.
That way needs more back-and-forth communication, but doesn't need any fancy error correcting code.
> Fountain codes and even general erasure codes are not the right tool for this job. The loss of a digital packetized channel across the internet is very different than a noisy analog channel sent through space.
Yes, the loss model is different. However you can eg interleave your bits to get something that close enough to work.
The main reason you don't need to use error correcting codes, is that transmitting feedback to the sender is typically a lot cheaper on the internet than in outer space. Not least because even bad latencies are typically measured in seconds at most, not minutes or hours.
(You could however use these codes when for some reason you have a very asymmetrical link. Eg if you have internet via geo-stationary satellite, or if one sender is broadcast to lots and lots of different receivers, and doesn't want to deal with each of them individually.)
Yes, that is convenient. But it's an abstraction you don't always need, but still always pay for.
That's why I would suggest that were possible the kernel should only expose something at most as high level as UDP, and we build the TCP abstractions on top of that in user level. Of course, you wouldn't want to re-write everything from scratch all the time: you'd use libraries.
The nice thing about user level libraries is that you can swap them out for better versions or for different abstractions without any privileged access.
> TCP will waste roundtrips on handshakes. And then some extra on MTU discovery.
This is especially true when using TLS (which is at least a major use case nowadays if not the default assumption); the TCP handshake has to complete before the TLS handshake can occur, whereas QUIC has protocol-level support for doing TLS negotiation as part of the initial handshake. Having the network protocol and the encryption be composable rather than tightly coupled feels more elegant, it's hard for me to convince myself that being able to save a round trip per connection isn't a more practical benefit in a world where we now expect pretty much everything to be encrypted in transit.
That kind of elegance ends up in application level implementations like STARTTLS and socket data becoming unusable on the middle of a connection while both parties scramble to decide what to throw away and what to interpret. It does really look like the real thing, but it's decoupling things that are very coupled by definition.
> If you move into a location with worse coverage, your latency increases, but TCP will assume packet loss due to congestion and reduce bandwidth. And in general, loss-based congestion control just doesn't work at this point.
Don't lower levels of wifi below TCP now do often their own retransmit, even if it is just from weak/noisy signal and not a sensed collision?
The biggest fucking problem with NB-IoT or LTE-M is the goddamn carriers--both managerial and technical.
1) Managerial--Just like back in the old days when the carriers stupidly fought the change from voice to data they are stupidly fighting the change from humans using the network to machines using the network. And it's just as stupid this time around and it will be just as stupidly profitable for them after the fact.
2) The carriers refuse to support older firmware so you have to be able to do multi-megabyte firmware updates every 3 months. This is anathema to anything running on a battery as your firmware updates are like 2 to 4 orders of magnitude(not exaggerating) larger than the total actual data your device will transmit over its lifetime.
The carrier cranio-rectal impaction is why LoRA still continues to exist and expand.
We've used Sigfox with success but then they went almost bankrupt and I honestly didn't check where they are now or if it's safe to use them going forward. I still want devices to function 10 years from now.
In my corner of the world (electric/water/gas utilities), we had been using proprietary techniques that matched the "NB-IoT" description for 20 years, and they weren't new then. It was fun to watch the public dialogue walk the same path for a while and rediscover some of the main tricks, but there were other important tricks left untouched (broadcast firmware downloads, highly compressed debug logs, low power clock synchronization so that you can do timeslots for ultra low power, mesh heuristics) probably because they were a massive pain in the rear and the people who would have risen to the occasion saw that the major ecological niches were already occupied by players who got in 30 years ago when this all became viable. (Disclaimer: I didn't actively chase down NB-IoT talks so I might just have been missing these things.)
If your market is large enough that managing your own infrastructure is attractive (vs cellular), you were also able to afford a few EEs and RF guys 30 years ago, and your needs are already met by a competitive ecosystem of players who have been walking this path for a long time. It doesn't leave much to compete on except price, and you don't want to compete on price.
That makes me envious. I've figured out the debug log compression part myself, but the rest is beyond my reach.
> If your market is large enough that managing your own infrastructure is attractive
This. I imagine proprietary is perfect when you are deploying in a single country with single RF plan. And your customer utility company that is willing to maintain a bit of extra infrastructure. But if you want to deploy on several sites scattered across all ITU regions, and each site is a huge plant with devices from maybe fifty different suppliers, it's just a no-go.
It definitely does, at least in Europe, not sure how things are on the other side of the pond. For the user equipment there is no real advantage over LTE-M, actually is LTE-M better for our case (send a single packet a day and go to sleep). But then I don't see the picture from the operator side, maybe the radio savings actually matter for them. Some operators either don't have LTE-M at all or don't allow PSM on LTE-M because screw you that's why. On NB-IoT PSM is mandatory at least.
And yes, LoRa/Sigfox/other non-cellular LPWAN might technically be better IF you provide the base stations. Nobody wants to do that. With cellular you can just mail the device and cross your fingers the operators haven't misconfigured roaming.
That's quite interesting. And the promised battery lifespans of NB-IoT device, did it hold some truth? In the beginning we heard about 10 year life spans for simple well designed devices.
You'll have to wait 10 years to see. But on paper it does (we aim for 15) and we have devices in field for some three years.
But for the "simple well-designed" - the problem is, the whole thing is a black box for application developer, and not a bugless one. So much of the "well-designed" part is to figure out when is the right time to pull modem's proverbial power cord. If you don't, it'll gladly eat up your battery in 48 hours after a cell handover goes wrong with no warning.
We have millions of devices in the field older than that on the same D-cell LiSOCl2 battery they shipped with. Some of them even take magnetic measurements with powered excitation!
The carriers require you to allow them to force a firmware upgrade on you. This kills your battery dead long before anything you would do on the device will.
Technically, however, 10 year battery life is generally unreasonable. Most batteries that exist self-discharge over 10 years. 5 years is doable, but requires quite solid engineering. 3 years is pretty reasonable for most things.
If you want 10 year lifetimes, I think you specifically need lithium thionyl chloride batteries.
We use straight UDP datagrams for streaming high-frequency sensor data. One of our R&D people built a new system that uses quic and solves most of our problems with out-of-order delivery. We still use datagrams over UDP for everything because we have to support some 3rd party sensors out of the box without adapters, and UDP is all they can do.
In general, the wireless handshake on G5/LTE/Wifi/Starlink will dwarf the traffic of sensors for power use. For old-style satellite mailboxes, the traffic is better batched in 45min chunks anyway (the 5W transmitters are usually only active for well under 20 seconds per message, and use a super-cap to handle the RF power draw spike.)
Naive UDP protocols have reachability problems in numerous scenarios, variable costs can balloon (concurrency with polling links is dumb), and can open a hole in your infrastructure at scale without special equipment mods. Only if you have a tier 3 or better WAN trunk or cloud-center should you even consider something silly like QUIC.
Thats fine for a simple cavity, get a filling and brush more often; but sometimes you have a three-holed sphere to put on one leg at a time, like anyone else.
This may seem like a minor nit, but I think there is a problem with using the term "unreliable" to describe UDP. The more commonly used term, and IMHO better term, is "best-effort" [1]. UDP makes its best effort to deliver the datagrams, but the datagrams may be dropped anyway. But it does not make UDP inherently unreliable.
IMO "best-effort" is euphemistic and confusing to people outside of this space, even (especially?) to native English speakers. (I would probably have described UDP as "reasonable effort" and TCP as "best effort", if not for the existing terminology.)
I recall my computer networking professor describing this as "Best-effort means never having to say you're sorry". In practice, best-effort does not mean you try your hardest to make sure the message gets from A to B, it means that you made an effort. Router in the path was congested? Link flap leading to blackholing on the order of 50ms before fast reroute kicks in? Oh well, we tried.
Meanwhile, TCP's reliable delivery will retry several times and will present an in-order data stream to the application.
Reliable vs unreliable might be bad terminology, but I don't think best-effort is any better.
My experience with unreliable systems is that they're great something like 95% of the time, and they're great for raw throughput, but there are many cases where that last 5% makes a huge difference.
The question is 'who deals with dropped packages'? In TCP, the answer is: 'the protocol'. In UDP the answer is 'the next layer of abstraction' (eg the app or some library).
You can build a 'reliable' protocol on top of UDP, and still not get TCP.
Eg if you want to transfer a large file that you know up front, then TCP's streaming mechanism doesn't make too much sense. You could use something like UDP to send the whole file from A to B in little chunks once, and at the end B can tell A what (numbered) chunks she's missing.
There's no reason to hold off on sending chunk n+1 of the file, just because chunk n hasn't arrived yet.
> There's no reason to hold off on sending chunk n+1 of the file, just because chunk n hasn't arrived yet.
Congestion control comes to mind -- you don't necessarily know what rate the network supports if you don't have a feedback mechanism to let you know when you're sending too fast. Congestion control is one of those things where sure, you can individually cheat and possibly achieve better performance at everyone else's expense, but if everyone does it, then you'll run into congestive collapse.
> You can build a 'reliable' protocol on top of UDP, and still not get TCP.
I agree -- there are reliable protocols running on top of UDP (e.g. QUIC, SCTP) that do not behave exactly like TCP. You don't need an in-order stream in the described use case of bulk file transfer. You certainly don't need head-of-line blocking.
But there are many details and interactions that you and I wouldn't realize or get right on the first try. I would rather not relearn all of those lessons from the past 50+ years.
> But there are many details and interactions that you and I wouldn't realize or get right on the first try. I would rather not relearn all of those lessons from the past 50+ years.
Oh, the model I had in mind was not that everyone should write their network code from scratch all the time, but rather that everything that's higher level than datagrams should be handled by unprivileged library code instead of privileged kernel level code.
If speed is an issue, modern Linux can do wonders with eBPF and io_uring, I guess? I'm taking my inspiration from the exokernel folks who believed that abstractions have no place in the operating system kernel.
interestingly, the exokernel approach is suited to particular cases. for instance, a single application (regardless of whether it's multiple sessions, diversity of RTT, etc). after all, "get the packets into userspace with as little fuss as possible" would be the right goal there.
the unix model is different: it's basically premised on a minicomputer server which would be running a diverse set of independent services where isolation is desired, and where it makes sense for a privileged entity to provide standardized services. services whose API has been both stable and efficient for more than a couple decades.
I think it's kind of like cloud: outsourcing that makes sense at the lower-scale of hosting alternatives. but once you get to a particular scale, you can and should take everything into your own hands, and can expect to obtain some greater efficiency, agility, autonomy.
> the unix model is different: it's basically premised on a minicomputer server which would be running a diverse set of independent services where isolation is desired, [...]
Exokernels provide isolation. Secure multiplexing is actually the only thing they do.
> and where it makes sense for a privileged entity to provide standardized services. services whose API has been both stable and efficient for more than a couple decades.
Yes, standardisation is great. Libraries can do that standardisation. Why do you need standardisation at the kernel level?
That kind of standardisation is eg what we are doing with libc: memcpy has a stable interface, but how it's implemented depends on the underlying hardware; the kernel does not impose an abstraction.
Is there a popular protocol that uses this scheme today? I've often thought that this would be a superior way to transfer/sync data. and have always wondered why it wasn't common.
The closest I can think of is the old FSP protocol, which never really saw wide use. The client would request each individual chunk by offset, and if a chunk got lost, it could re-request it. But that's not quite the same thing.
SACKs have been in TCP for 25-30 years now (Widely adopted as part of New Reno, although RFC 2018 proposed the TCP option and implementation back in 1996).
That said, the typical reason why TCP doesn't send packet N+1 is because its congestion window is full.
There is a related problem known as head-of-line blocking where the application won't receive packet N+1 from the kernel until packet N has been received, as a consequence of TCP delivering that in-order stream of bytes.
Sorry, I didn't literally mean n+1. What I mean is probably better described by 'head of line blocking'.
Basically, when transmitting a file, in principle you could just keep sending n+k, even if the n-th package hasn't been received or has been dropped. No matter how large k is.
You can take your sweet time fixing the missing packages in the middle, as long as the overall file transfer doesn't get delayed.
The term "best effort delivery" in networking is a weasel term that is no better than "unreliable". It should probably be burned.
"Effort" generally refers to some sort of persistence in the face of difficulty. Dropping a packet upon encountering a resource problem isn't effort, let alone best effort.
The way "best effort" is used in networking is quite at odds with the "best efforts" legal/business term, which denotes something short of a firm commitment, but not outright flaking off.
Separately from the delivery question, the checksums in UDP (and TCP!) also poorly assure integrity when datagrams are delivered. They only somewhat improve on the hardware.
I mean that in the same rhetorical sense of "standing around leaning on your shovel isn't effort, let alone best effort". Of course we all understand that what is effort is digging the ditch.
Anyway, so the question is, if typical IP forwarding is "best effort" ... what is an example of poor effort, and what exhibits it?
I think the issue is that there's a stigma around UDP, largely borne from not using it effectively. I'm not sure this is the right way to address the stigma though.
Ultimately, TCP vs UDP per se is rarely the right question to ask. But that is often the only configuration knob available, or at least the only way to get away from TCP-based protocols and their overhead is to switch over to raw UDP as though it were an application protocol unto itself.
Such naive use of UDP contributes to the stigma. If you send a piece of data only once, there's a nontrivial chance it won't get to its destination. If you never do any verification that the other side is available, misconfiguration or infrastructure changes can lead to all your packets going to ground and the sender being completely unaware. I've seen this happen many times and of course the only solution (considered or even available in a pinch) is to ditch UDP and use TCP because at least the latter "works". You can say "well it's UDP, what did you expect?" but unfortunately while that may have been meant to spur some deeper thought, it often just leads to the person who hears it writing off UDP entirely.
Robust protocol design takes time and effort regardless of transport protocol chosen, but a lot of developers give it short shrift. Lacking care or deeper understanding, they blame UDP and eschew its use even when somebody comes along who does know how to use it effectively.
Both? I've seen UDP and TCP treated as interchangeable. They're not, even though in many cases the APIs involved make it seem that way. Choosing datagrams and unreliable transport vs. streams and reliable transport affects the rest of the protocol design and behavior, whether the designer wants it to or not and whether the implementer considered the consequences or not.
i agree that "unreliable" isn't a good term for the transport. reliability is a property of the system, not of the transport. and you can make an unreliable system with TCP or a reliable one with UDP
but, "best-effort" implies that it's doing some effort to ensure delivery, when it's really dropping any packet that looks funny or is unlucky enough to hit a full buffer
i like "lossy", but this is definitely one of the two hard problems
In a way, TCP is just about as reliable as UDP, but at a different layer: TCP will either forward your entire stream of data in order and without gaps, or it won’t. UDP does the same, but on the per-datagram level.
Reliability is arguably more of a statement about the availability metrics of your underlying network; it doesn’t seem like a great summary for what TCP does. You can’t make an unreliable lower layer reliable with any protocol magic on top; you can just bundle the unreliability differently (e.g. by trading off an unreliability in delivery for an unreliability in timing).
TCP wont always deliver your packets anyway , but it does have a mechanism of timing out if a party believes the other party did not receive something. UDP just means that if one cares for their data to be received, they must verify it themselves.
That's in my opinion what reliable / unrealiable mean in this context.
reliable = it either succeeds or you get an error after some time, unreliable = it may or may not succeed
I concur with the people who think "best effort" is not a good term. But perhaps TCP streams are not reliable enough for TCP to be rightly called a reliable stream protocol. As it turned out, it's not really possible to use TCP without a control channel, message chunking, and similar mechanisms for transmitting arbitrary large files. If it really offered reliable streams that would its primary use case.
the designers of link layers and protocol implemntations, the army of operators who test signals, insteall repeaters, configure routers and hold conferences about how to manage the global routing system would disagree.
best effort implies 'no, we're not going to be climb that curve and get try to get to 100% reliability, because that would actually be counterproductive from an engineering perspective, but we're going to go to pretty substantial lengths to deliver your packet'
There is minimal effort. If the queue has room the packet gets forwarded. If the situation is any more difficult than that the effort ends and the packet is dropped.
This sounds like the problem is the term "best-effort" (hand wavy, what's the measure of effort? What's "the best" effort?).
In the end, best-effort is just saying "unreliable" in a fussier way.
>But it does not make UDP inherently unreliable.
Isn't that exactly what it does make it?
If that's not it, then what woud an actual "inherently unreliable" design for such a protocol be? Calling an RNG to randomly decide whether to send the next packet?
IMO stream abstractions make it too convenient to write fragile programs which are slow to recover from disconnections (if they do at all) and generally place too many restrictions on the transport layer. Congestion control is definitely needed but everything else seems questionable.
In a datagram-first world we would have no issue bonding any number of data links with very high efficiency or seamlessly roaming across network boundaries without dropping connections. Many types of applications can handle out-of-order frames with zero overhead and would work much faster if written for the UDP model.
> In a datagram-first world we would have no issue bonding any number of data links with very high efficiency or seamlessly roaming across network boundaries without dropping connections. Many types of applications can handle out-of-order frames with zero overhead and would work much faster if written for the UDP model.
So your argument is that software isn’t written well because TCP is too convenient, but we’re supposed to believe that a substantially more complicated datagram-first world would have perfectly robust and efficient software?
In practice, moving to less reliable transports doesn’t make software automatically more reliable or more efficient. It actually introduces a huge number of failure modes and complexities that teams would have to deal with.
> So your argument is that software isn’t written well because TCP is too convenient, but we’re supposed to believe that a substantially more complicated datagram-first world would have perfectly robust and efficient software?
I think you can make a better argument:
Your operating system should offer you something like UDP, but you can handle all the extra features you need on top of that to eg simulate something like TCP, at the level of an unprivileged library you link into your application.
That way is exactly as convenient for 'normal' programmers as the current world: you get something like TCP by default. But it's also easy for people to innovate and to swap out a different implementation, without needing access to the privileged innards of your kernel.
> Congestion control is definitely needed but everything else seems questionable.
Even congestion control can be optional for some applications with the right error correcting code. (Though if you have a narrow bottleneck somewhere in your connection, I guess it doesn't make too much sense to produce lots and lots more packages that will just be discarded at the bottleneck.)
As mentioned in the article, this doesn't handle the case of buffer bloat: packets all eventually arrive rather than being dropped, but increasingly late, and only backing off can help reduce the latency.
Any data block that must be fully received before processing comes to mind (e.g. websites’ html), just request retransmission of parts that didn’t make it in first pass. Funnily enough it’s (realtime) video and audio that already uses UDP due to preferring timely data to complete data.
On the contrary, for me it’s hard to imagine video game with missed packets as the state can get out of sync too easily, you’ll need eventual consistency via retransmission or some clever data structures (I know least about this domain though)
I'm not expert in this area, but my understanding is that for latency sensitive video games, on the client side, state is predicted between updates, e.g. the state may be updated via the network, at say, roughly 10Hz (probably faster, but probably less than 60Hz), but updated locally at 60Hz via interpolation, dead reckoning and other predictive/smoothing heuristics, a few missed updates just means a bit more prediction is used. "State" is not usually transmitted as one lump "frame" at a time, but rather per-game unit (per spaceship, asteroid, robot, or whatever is in the game) or per some-small-group of units. When some updates are delayed too long, you might get some visible artifacts, "rubber-banding", "teleportation" or "warping", etc. often lumped together by players under the umbrella term "lag". For out-of-order packets, older packets might be dropped as newer packets may already have been applied to the state of some game units (typically there's some timestamps or monotonically increasing counter associated with state updates for each unit used by the prediction/interpolation/smoothing heuristics) and the state is usually represented in absolute terms rather than relative terms (e.g. (x,y) = (100,100), rather than x+=10, y+=10, so that any update may be applied in isolation.)
Many video games just transmit the whole state the player should be able to see every tick. While some games send a diff of some kind, sometimes that turns out to not be worth it. This is particularly true in the games which care most about lag (think mario kart, or street fighter and friends).
One thing not mentioned often is that a lot of networks will drop UDP packets first when encountering congestion. The thinking is that those packets will not re-transmit, so it’s an effective means to shed excess traffic. Given we now have protocols that aggressively re-transmit on UDP, I wonder how that has changed things. I do seem to remember QUIC having re-transmit issues (vs HTTP1/2) years ago because of this.
I've attempted to replace the clickbait title* using terms from the article itself, but if someone can suggest a more representative phrase from the article, we can change it again.
>> The common wisdom is:
>> >> use TCP if you want reliable delivery
>> >> use UDP if you want unreliable delivery
>> What the *(& does that mean? Who wants unreliability?
I dont agree with the premise of this article, UDP isnt for unreliability, it provides a tradeoff which trades speed and efficiency and provides best-efforts instead of guarantees.
It makes sense depending on your application. For example, if I have a real-time multi-player video game, and things fall behind, the items which fell behind no longer matter because the state of the game changed. Same thing for a high-speed trading application -- I only care about the most recent market data in some circumstances, not what happened 100ms ago.
That isn’t the premise of the article, that is the “common wisdom” the author corrects as the article goes on. The author goes on to list video games as an example of where UDP makes sense, as well as live video.
Silly clickbait title, which the author even admits up front.
UDP and TCP have different behaviour and different tradeoffs, you have to understand them before choosing one for your use case. That's basically it. No need for "Never do X" gatekeeping.
My apologies if the star in front of "never" was added to the title in the last ten minutes after you posted your comment. But that star is clearly there to indicate "fine print ahead"; in a title, it's pretty obvious that it means that the article is not about to "gatekeep" people out of UDP. (In fact, at the end of the article, the author suggests to use QUIC which, you guessed it, is based on UDP.)
One missing from your list is real-time media. Retransmission, or even just buffering for reordering, adds latency so it's better to take the hit of a loss with error correction or packet-loss-concealment
Yeah exactly. With realtime, a dropped packet is often too old by the time it's re-transmitted. It's actually harmful to wait for it when you'd rather just skip it and move on and stay as close to realtime as possible. It's a very different usecase from most uses.
I do wish QUIC allowed carrying streams that were useful for realtime in conjunction with allowing reliable streams. Using MPEG-TS over SRT to have it just spam metadata to handle the unreliableness is janky. It would be far nicer to have a reliable stream for metadata, then an unreliable one for realtime streaming.
Hey I'm sure you're right, but as someone with less low-level networking expertise, I was wondering if you could explain why you'd want datagrams for these use-cases?
TCP is a back & forth between two hosts and thus cant do "shouting blindly into the network" things. That rules out local discovery and media broadcast.
Package encapsulation is bad over TCP because if the encapsulated data is TCP itself, you have congestion control twice. On congested networks, this results in extra slowdowns that can make the connection unusable.
The whole point of the article is that you don't need to go all the way down to raw datagrams to achieve the kind of low latency that is needed for things like games/VOIP/livestreaming
Nahhh. While most of the applications/cases will be using session-based connections, there are uses for using datagrams directly — don’t be afraid. Yes, you will have to take care of many more details yourself. And as a side line, it is a great way of learning the low level aspects of networking.
The SteamNetworkingMessages API for game development does a good job of making this available if you want it without caring about the internals for that use case.
Everything at some layer has to run over datagrams, since that's what IP is.
This is literally the point of the article: if you want to create a protocol over raw datagrams, you have to implement a lot of things that are very hard to get right, so you should just use QUIC instead, which does them for you.
"You" is doing all the hard work. Apps should use reliable sessions and transport, almost all the time? No disagree. The exceptions are understood. But don't pretend the substrate is that reliable data stream. "We" have to construct it almost always. We do reliable for you, over UDP.
I don't do this stuff any more, but I worked on OSI transport and remote operations service mapped to UDP and other protocols back in the 80s
> The bytes within each stream are ordered, reliable, and can be any size; it’s nice and convenient. Each stream could be a video frame […] But you can tell the QUIC stack to focus on delivering important streams first. The low priority streams will be starved, and can be closed to avoid wasting bandwidth.
Is the author saying that with QUIC I can send a "score update" for my game (periodic update) on a short-lived stream, and prevent retransmissions? I'll send an updated "score update" in a few seconds, so if the first one got lost, then I don't want it to waste bandwidth retransmitting. Especially I don't want it retransmitted after I've sent a newer update.
That doesn't quite provide the same experience, does it?
For something like a scoreboard you don't want the message to be discarded once any packet loss is detected. As long as message N is the most recent one, message N should be retried until it succeeds. You just want to stop retrying N once N+1 becomes available - or ideally even successfully delivered.
In a game context, where we're spewing state updates out 10-60x per second, there is always a newer packet in flight already, so discard-on-loss produces the same result.
In your scoreboard example, you probably just want an actually reliable stream for that - it's not like scores are rolling in so fast that it's beneficial to deal with it as unreliable.
It mentions that the video game industry uses UDP but then fails to further address that use case. So, should competitive shooter video games switch to QUIC? Is that even supported across all the various gaming platforms?
> It mentions that the video game industry uses UDP but then fails to further address that use case
Video games tend to use UDP for the same reason everyone else mentioned does: timeliness. You want the most recent position of the various game objects now, and you don't give a shit about where they were 100ms ago.
The proposed solution of segmenting data into QUIC streams and mucking with priorities should work just fine for a game.
> Is that even supported across all the various gaming platforms?
QUIC itself is implemented in terms of datagrams, so if you have datagrams, you can have QUIC.
But the proposed solution provides no value over datagrams, while it has plenty of downsides.
Let me rephrase the problem here for a simple FPS game:
The entire world state (or, the subset that the client is supposed to know about) is provided in each packet sent from the server. The last few actions with timestamps are in each packet sent from the client. You always want to have the latest of each, with lowest possible latency. Both sides send one packet per tick.
You do not want retransmission based on the knowledge that a packet was lost (the RTT is way too long for the information that a packet was lost to ever be useful, you just retransmit everything every tick), you do not want congestion control (total bandwidth is negligible, and if there is too much packet loss to maintain what is required, there is no possible solution to maintain sufficient performance and you shouldn't even try), and none of the other features talked about in the post add anything of value, either.
It reads like someone really likes QUIC, it fit well into their problems, and they are a bit too enthusiastic about evangelizing it.
In practice most games also need reliable side channels for control messages/chat/etc, and so they end up building optional reliable streams over UDP... and at the end of this path lies something that looks a lot like a (less thoroughly designed/tested) version of QUIC
My spidey sense is that almost all games used a custom UDP solution. I could be wrong! I made a Twitter poll to try and get info.
You’re right that a custom reliable UDP solution is going to wind up QUIC-like. On the other hand it’s what games have been doing for over 20 years. It’s not particularly difficult to write a custom layer that does exactly what a given project needs.
I don’t enough about QUIC to know if it adds unnecessary complexity or not.
NAT traversal, historically. Reliably punching one hole through the local NAT gateway is bad enough, adding a second protocol into the mix was asking for trouble.
Tribes, and various quake mods, used to use a TCP side channel for chat and control messages. Tribes famously used irc, and you could connect to a tribes server with any irc client to chat with your friends in game
> Video games tend to use UDP for the same reason everyone else mentioned does: timeliness. You want the most recent position of the various game objects now, and you don't give a shit about where they were 100ms ago.
This is only true for games that can replicate their entire state in each packet.
There are many situations where this is infeasible and so you may be replicating diffs of the state, partial state, or even replicating the player inputs instead of any state at all.
In those cases the "latest" packet is not necessarily enough, the "timliness" property does not quite cover the requirements, and like with most things, it's a "it depends".
With those requirements raw datagrams also don't fit the bill, so you'll need to build some sort of reliable stream abstraction over UDP (with optional unreliable delivery for the state which you can replicate quickly)... and now we're closing in on QUIC territory
I thought QUIC promised you get your data eventually? In that case it wouldn't be great for games, because you usually don't want packets that got dropped, as the next packet will replace it anyway?
> I thought QUIC promised you get your data eventually?
QUIC optionally promises you that, you are free to opt out. For example, take a look at the QUIC_SEND_FLAG_CANCEL_ON_LOSS flag on Microsoft's QUIC implementation.
I wonder if this guy thinks video GOPs should all be hundreds of frames long because P frames are so much smaller than I frames and since in his world we NEVER use an unreliable network you might as well.
I wonder if the DiffServ[0] bits in IPv6 could be another way to prevent bufferbloat from affecting real-time datagrams? Or are they like IPv4's ToS[1] bits, which I think were never implemented widely (or properly) enough for any software to bother with?
DSCP / ToS is used for traffic classification within a network (which can be used to select router policy), but generally not across autonomous systems. You'd have to get everyone to agree on what various traffic markings mean, and, more critically, which traffic deserves to be marked in certain ways.
For instance, suppose that you wanted a special traffic class that's prioritized over everything else. You really don't want just anyone to be able to send you that kind of traffic, since a DoS attack can easily starve all the other queues.
Your best bet is usually to match traffic based on other characteristics, e.g. port × protocol.
Still, it seems like it would be a useful idea to have an IP bit for "minimize buffering and drop instead, packet is time-sensitive", and that seems safe since the user is requesting fewer resources, not more.
You would think so, but buffers are actually a core part of regular router function. The problem is not inherently router buffers, but rather buffers that don't fully drain (bufferbloat) or that fill up (congestive drops).
Think about it this way: a 100Gbps link can process a 1500 byte packet in about 0.12 microsecond. If you have an average of 1,000 packets in that buffer at steady-state, that buffer is contributing a fraction of a millisecond to your overall latency. Meanwhile, if your home router's buffer has the same 1,000 packets for a 1Gbps link, that's 12 milliseconds of latency.
If you had a way to tell a router to never buffer packets, you'd encounter packet loss much sooner, and for no good reason. Buffers are great for smoothing out (small) bursts of traffic (although if you have large bursts on very short timescales, you can easily overwhelm these buffers -- this is why you typically have pacing on the OS level).
If instead you want the router to add no more than X amount of latency, that's suddenly much harder to dictate. (The "correct" value of X depends on your application requirements as well as the number of hops through the network between you and the server you're talking to.)
(Also, while you might think that this should be strictly beneficial for network operators since the user wants fewer resources, encoding exceptions like this into router policy ends up using more of a specialized and expensive kind of memory called TCAM. Assuming that it's even possible to do such a thing.)
This is a narrow way of looking at UDP applications. The whole HFT, low-latency fintech world is built on top of datagrams. Using TCP would be pure the worst choice possible.
You are correct, it's either point-to-point, like direct cross-connects with the exchange or it's a relatively small intranet.
Additionally, an overwhelming majority of packets flying around are market data. You don't want to re-transmit lost ones because by the time you do, the data is already outdated.
Imagine Ethernet was designed like this and you had to implement mandatory congestion control and other cruft. The layer of the stack that has knowledge of the content should be implementing the congestion control.
In fact, the link layer protocols that dealt with congestion and access to media more meticulously, eg Token Ring or FDDI, were generally much more efficient with the use of the media - using near 100% of the potential bandwidth, whereas Ethernet on a shared medium already gets quite inefficient at 30-40% utilization due to retries caused by collisions.
However, the trade off between additional complexity (and thus bigger cost and difficulty of troubleshooting) was such that the much simpler and dumber Ethernet has won.
However, a lot of similar principles are there in Fibre Channel family protocols, which are still used in some data center-specific protocols.
LAN can be oversubscribed too, even if your switches have sufficient capacity you can have short packet bursts that fill up the queues. You cannot assume reliable delivery.
Unless you are building software for a pre-established intranet, how do you predict ahead of time whether your software will run on congested networks?
Most end-user software ends up running on a variety of oversubscribed wifi and cell networks
You tell the user "hey, don't use on congested networks". And yeah, since most internet software has this exact disclaimer, it works kinda well.
But people do appreciate software that doesn't have this restriction, and even go out of their way to talk about it (more on mobile networks than on wired connections).
> how do you predict ahead of time whether your software will run on congested networks?
One thing you can do is build in "bad network emulation" into your software, allowing the various pieces to drop packets, delay packets, reorder packets, etc. and make sure it still behaves reasonably with all this garbage turned up.
I've seen UDP used for great effect in video streaming. Especially timely video streaming such as cloud gaming. When waiting a late packet is no longer useful.
RTSP is the control protocol. Some other protocol is needed for the actual audio/video streaming. That's usually RTP, these days.
RTP is a core part of WebRTC, for example.
When you're doing a video call in a web browser, you're using WebRTC, including RTP. In fact, this RTP-via-WebRTC is the only way to send UDP packets from JavaScript!
RTSP is still used by older streaming systems and hardware ecosystems that are slow to change, such as network-connected security cameras. But in newer applications, WebRTC has mostly replaced it. Of course, the QUIC effort is in part an attempt to replace WebRTC, so the wheel continues to turn!
Used high numbers of UDP packets over intercontinental internet links for mission critical applications for 15 years. 20mbit of UDP carrying RTP. Loss on a given flow is quite rare, and the application helps (via duplication, or retransmits)
As time has progressed increased from nothing to fec to dual-streaming and offset-streaming to RIST and SRT depending on the criticality.
On the other hand I've seen people try to use TCP (with rtmp) and fail miserably. Never* use TCP.
Or you know, use the right tool for the right job.
I feel like this article misses why people avoid TCP like the plague, and why people use UDP for many applications
1. Routers do all kinds of terrible things with TCP, causing high latency, and poor performance. Routers do not do this to nearly the same extent with UDP
2. Operating systems have a tendency to buffer for high lengths of time, resulting in very poor performance due to high latency. TCP is often seriously unusable for deployment on a random clients default setup. Getting caught out by Nagle is a classic mistake, its one of the first things to look for in a project suffering from tcp issues
3. TCP is stream based, which I don't think has ever been what I want. You have to reimplement your own protocol on top of TCP anyway to introduce message frames
4. The model of network failures that TCP works well for is a bit naive, network failures tend to cluster together making the reliability guarantees not that useful a lot of the time. Failures don't tend to be statistically independent, and your connection will drop requiring you to start again anyway
5. TCP's backoff model on packet failures is both incredibly aggressive, and mismatched for a flaky physical layer. Even a tiny % of packet loss can make your performance unusable, to the point where the concept of using TCP is completely unworkable
Its also worth noting that people use "unreliable" to mean UDP for its promptness guarantees, because reliable = TCP, and unreliable = UDP
QUIC and TCP actively don't meet the needs of certain applications - its worth examining a use case that's kind of glossed over in the article: Videogames
I think this article misses the point strongly here by ignoring this kind of use case, because in many domains you have a performance and fault model that are simply not well matched by a protocol like TCP or QUIC. None of the features on the protocol list are things that you especially need or even can implement for videogames (you really want to encrypt player positions?). In a game, your update rate might be 1KB/s - absolutely tiny. If more than N packets get dropped - under TCP or UDP (or quic) - because games are a hard realtime system you're screwed, and there's nothing you can do about it no matter what protocol you're using. If you use QUIC, the server will attempt to send the packet again which.... is completely pointless, and now you're stuck waiting for potentially a whole queue of packets to send if your network hiccups for a second, with presumably whatever congestion control QUIC implements, so your game lags even more once your network recovers. Ick! Should we have a separate queue for every packet?
Videogame networking protocols are built to tolerate the loss of a certain number of packets within a certain timeframe (eg 1 every 200ms), and this system has to be extremely tightly integrated into the game architecture to maintain your hard realtime guarantees. Adding quic is just overhead, because the reliability that QUIC provides, and the reliability that games need, are not the same kind of reliability
Congestion in a videogame with low bandwidths is extremely unlikely. The issue is that network protocols have no way to know if a dropped packet is because of congestion, or because of a flaky underlying connection. Videogames assume a priori that you do not have congestion (otherwise your game is unplayable), so all recoverable networking failures are 1 off transient network failures of less than a handful of packets by definition. When you drop a packet in a videogame, the server may increase its update rate to catch you up via time dilation, rather than in a protocol like TCP/QUIC which will reduce its update rate. A well designed game built on UDP tolerates a slightly flakey connection. If you use TCP or QUIC, you'll run into problems. QUIC isn't terrible, but its not good for this kind of application, and we shouldn't pretend its fine
For more information about a good game networking system, see this video: https://www.youtube.com/watch?v=odSBJ49rzDo, and it goes over pretty in detail why you shouldn't use something like QUIC
I've been having this "you don't want to use TCP" conversation a lot lately with people who are building real-time voice + LLM applications. Almost everybody who hacks together a voice + LLM prototype starts with WebSockets. Which I totally understand. WebSockets seem like they should do what you want.
And WebSockets do work fine while you're testing.
But WebSockets are TCP, so when you roll things out to real-world users, latency is higher than it should be a lot of the time. Connections randomly drop. You have to try to figure out why different OS configurations are behaving differently. You start building your own keepalive logic. You start trying to figure out how to build metrics and monitoring for audio-over-WebSockets.
The answer is to use UDP, and use a protocol built on top of UDP designed for real-time media (WebRTC).
It's been interesting to me that I've had this exact same conversation with several dozen engineers over the past few months. It's a good reminder that things that seem obvious when you've been doing something a long time aren't obvious to people new to a domain. Even if they are very experienced in some adjacent domain. (In this case, mobile app and web app development.) It's made me think about where my knowledge boundaries are and what "obvious" things I'm ignorant about. (Lots of them, I'm sure.)
If you're interested in WebSockets, WebRTC, and why UDP is the right low-level approach for real-time media, I wrote a a primer about that a few months ago here:
> 1. Routers do all kinds of terrible things with TCP, causing high latency, and poor performance. Routers do not do this to nearly the same extent with UDP
What things are you thinking of here?
> 2. Operating systems have a tendency to buffer for high lengths of time, resulting in very poor performance due to high latency.
Do they? I can only think of Nagle's algorithm causing any potential buffering delay on the OS level, and you can deactivate that via TCP_NODELAY on most OSes.
> 5. TCP's backoff model on packet failures is both incredibly aggressive, and mismatched for a flaky physical layer. Even a tiny % of packet loss can make your performance unusable, to the point where the concept of using TCP is completely unworkable
That's a property of your specific TCP implementation's congestion control algorithm, nothing inherent to TCP. TCP BBR is quite resistant to non-congestion-induced packet loss, for example [1].
> If you use QUIC, the server will attempt to send the packet again which.... is completely pointless
Isn't one of QUIC's features that you can cancel pending streams, which explicitly solves that issue? In your implementation, if you send update type x for frame n+1, you could just cancel all pending updates of type x if every update contains your complete new state.
> 3. TCP is stream based, which I don't think has ever been what I want. You have to reimplement your own protocol on top of TCP anyway to introduce message frames
I'm always curious where frame proponents are going to buffer in-progress frames. OS? I have huge memory concerns. Also take note you want to drop in-order requirement and it means multiple in-progress frames. It's a very nice DoS vector.
IMO one of the worst mistakes made in IP development was not having made a standard protocol for the one thing that pretty much everyone seems to want:
A reliable, unlimited-length, message-based protocol.
With TCP there's a million users that throw out the stream aspect and implement messages on top of it. And with UDP people implement reliability and the ability to transmit >1 MTU.
That's because TCP evolved as, essentially, abstraction of serial port.
And due to various political & cultural & industrial lobbying, more flexible but complex on their face approaches (OSI) were rejected.
TCP meant you could easily attach a serial terminal to a stream, a text protocol meant you could interact or debug a protocol by throwing together a sandwich, an undergrad, and a serial terminal (coffee too if they do a good job). You could also in theory use something like SMTP from a terminal attached over a TIP (which provided dial-in connection into ARPAnet where you told the TIP what host and what port you wanted).
If you look through some of the old protocol definitions you'll note that a bunch of them actually refer to simplest mode of TELNET as base.
Then in extension of arguably the same behaviour that blocked GOSIP mandate we have pretty much reduced internet to TCP, UDP, and bits of ICMP, because you can't depend on plain IP connectivity in a world of NAT and broken middleboxes.
Sorry for a useless comment but now I feel so stupid for never noticing that pretty much everything done on TCP is actually message exchange, not streams. The mismatch is indeed so unfortunate.
By numbers of communication actions, pretty much everything may actually be a message exchange, but by the amount of data that was transferred for many years the file downloads and uploads had been dominant on the Internet, which are the kind of thing for which TCP has been designed. For a big file more data may be transferred than for thousands or millions of message exchanges.
Nowadays perhaps video streaming, which may be lossy, so TCP is not needed for it (i.e. what is now called streaming does not have the reliability requirements of the TCP streams), might transfer more data on the Internet than file downloads, but that is not a desirable technical evolution, the replacement of downloading with streaming is just a means to extract more money from those who are satisfied by such a service.
> I feel so stupid for never noticing that pretty much everything done on TCP is actually message exchange, not streams.
This generalization isn’t true at all. Streaming data over TCP is extremely common in many applications.
When you download a large file, you’re streaming it to disk, for example.
I guess you could get pedantic and argue that an entire stream of something is actually one large message, but that’s really straining the definitions.
Because worse is better and if you try to be everything you (don't) get OSI. Also because everyone loves serial ports and dialup modems and TCP just emulates a serial port over packet network. And it wasn't supposed to be one-size-fits-all, rather «let us have telnet while we figuring out the rest» but then ossification happend.
Is it really that unfortunate? What’s the big downside?
There’s a bit of reinvention needed, sure, but adding a framing layer is about the easiest task in networking. If you want a standard way of doing it, these days you can use a WebSocket.
Edit to add: oh, I see, you want reliable but unordered messages. That would definitely be useful sometimes, but other times you do want ordering. If you don’t need ordering, isn’t that pretty much what QUIC does?
SCTP did both reliable/unreliable and ordered/unordered, with QUIC arguably being at least a bit inspired by it, but going over UDP because internet is broken.
EDIT: Internet being broken is also why there's a grease extension for QUIC
The difference is whether you get to process the ones which came after first or you must wait for the retransmission of the earlier ones. FWIW, SCTP provides for this difference.
Actually, the mistake in IP was in not encrypting the next layer, so that networks can block transport protocol innovation by filtering anything other than TCP and UDP. This is the mistake that quic has fixed.
I think that one can be forgiven for pure lack of processing power... We forget just how under powered even host machines were and how resource hungry encryption is...
This was absolutely not feasible at the time IP was being developed. There simply wasn’t enough computational power available or good encryption algorithms invented yet. And you can make a reasonable argument that that’s not the responsibility of that layer of the stack.
I should perhaps have said "works round" rather than fixes - yes, it has to run over UDP but the quic team had an explicit goal of not allowing the network to interfere further : see sec 3.3 and 7.5 of https://dl.acm.org/doi/pdf/10.1145/3098822.3098842
This now means that innovations based on Quic can occur
Since both QUIC and UDP are transport layers, does this mean that once switches and ISPs start supporting QUIC along side UDP and TCP, that a new version of QUIC can be released which doesn't require UDP?
Switches don't need to change anything to support QUIC, precisely because it uses UDP. So, the deployment of QUIC doesn't involve any technical changes that would enable a version of QUIC that worked directly over IP. It's possible that the defeat of middleboxes would change the motivation of ISPs to filter out alternate protocol numbers, meaning that there might be less resistance to getting them to allow that in the future - but doing so is still a big exercise.
Also, the overhead of a UDP packet is 8 bytes total, in 4 16-bit fields:
- source port
- dest port
- length
- checksum
So, we can save a max of 8 bytes per packet - how many of these can we practically save? The initial connection requires source and dest ports, but also sets up other connection IDs so theoretically you could save them on subsequent packets (but that's pure speculation, with zero due diligence on my part). It would require operational changes and would make any NATs tricky (so practically only for IPv6)
Length and checksum maybe you can save - QUIC has multiple frames per UDP datagram so there must be another length field there (anyway, IETF is busily inventing a UDP options format on the basis that the UDP length field is redundant to packet length, so can be used to point at an options trailer). QUIC has integrity protection but it doesn't seem to apply to all packet types - however I guess a checksum could be retained for those that don't only.
So in sum maybe you could save up to 8 bytes per packet, but it would still be a lot of work to do so (removing port numbers especially)
I tried using SCTP before QUIC. Unfortunately, while the API looks like you can mix reliable/unreliable messages, in reality the transport doesn't support it because of how FORWARD-TSN works. There were some other pretty fundamental blockers too.
Yes. When your messages need to be received and processed in the same order they are sent, then a stream is a good (or good enough) abstraction.
For most applications some re-orderings of messages don't matter, and others would need special handling. So as a one-size-fits-all-(but-badly) abstraction you can use a stream.
> Do we have any actual streaming use of TCP, with purely streaming protocol.
But to give you a proper answer: the stream of keyboard inputs from the user to the server in Telnet (or SSH).
Opens up the question that if you have "messages" are you a stream anymore? Can your stream of messages start mid message for example? Surely a stream can have this happen. Or are you instead messages send over a stream. In which case abstraction of stream instead of reliable message transportation is bit weird.
Wouldn't each input be a single albeit too short message? But this level of granularity really makes little sense...
If you have a reliable-byte-stream abstraction, you only need to add a way to serialise your messages into bytes, and you get a reliable-messages-in-order abstraction for 'free'. That's convenient! And you don't need to teach your lower level protocols about where your higher level messages start and end.
You can also stick your higher level messages into a structure that's more complicated than a stream, eg you can stick them into a tree. Anything you can serialise, you can send.
Of course, the downside to this is that when you don't need these strong guarantees, you are paying for stuff you don't need.
Think about what the feature (potentially) buys you:
(1) zero-copy, zero-allocation request processing
(2) up to a 2x latency reduction by intermingling networking and actual work
(3) more cache friendliness
(4) better performance characteristics on composition (multiple stages which all have to batch their requests and responses will balloon perceived latency)
If you have a simple system (only a few layers of networking), low QPS (under a million), small requests (average under 1KB, max under 1MB), and reasonable latency requirements (no human user can tell a microsecond from a millisecond), just batch everything and be done with it. It's not worth the engineering costs to do anything fancy when a mid-tier laptop can run your service with the dumb implementation.
As soon as those features start to matter, streaming starts to make more sense. I normally see it being used for cost reasons in very popular services, when every latency improvement matters for a given application, when you don't have room to buffer the whole request, or to create very complicated networked systems.
High quality VOD (ie streaming a 4K movie). HTTP block file systems, each block needs to be streamed reliably to fulfill the read() call plus read ahead.
For VOD can I just open connection and send single message and then stream will continue forever? And HTTP is message oriented protocol. I can't just send infinite length HTTP message. Which would be processed as it arrives or can I? Meaning can I upload something not that small like terabyte of video data over HTTP?
I've seen a few implementations of a reliable UDP messaging stream (using multicast or broadcast). It uses a sequencer to ensure all clients receive the correct message and order.
Much much faster, TCP is very slow and complicated (relatively). This can be sped up by not using the kernel using something talking directly to hardware.
I have implemented publish/subscribe middleware in the 90s using UDP and reliable IP multicast. It was not too hard to do and had taken me about two month. Yes I did reinvent the wheel but it had numerous positive sides:
a) It added yet another area of expertise to my portfolio.
b) I did it because it was needed to facilitate another product in the company where I worked. The alternative at the time would be paying $350,000 for middleware from Vendor-X. Not acceptable.
c) Vendor-X people noticed us, loved what we did and made us a partner to resell their middleware and we've made a ton of money on deployment, configuring and consulting services.
Where to keep in-progress frame data? I case of TCP it's an app and buffer for only a single frame or app even could make "stream processing" for in-progress frame.
WebSocket is over TCP, so still ordered. I think by message-based GP is talking not only about message framing (which WebSocket provides) but also reliable out-of-order delivery.
Other people have probably thought about it more than I have, but the way I see it working is that a set of unacked messages are stored on each end of the connection. When a connection receives a message, it acks it by ID. When acks come out of the order the client sent them in (or after a timeout period), it re-sends the packet that may have dropped.
If every message is getting dropped, the set of unacked messages fills up and the connection stops accepting messages from the application, similar to TCP in that situation.
I mostly work with TCP so haven't had to deal with unreliable channels generally, but I do use a similar approach at the application level for reliable delivery across reconnects.
I use UDP for my desktop game like software with clients all over the world to propagate anonymized state. It needs no encryption as the transmitted data has zero value to any third party. It needs no reliability since dropped packets would be handled by predictive filter with practical reliability that is way more than needed.
You might want some form of encryption or message signing to prevent spoofed packet injection. Just because the data might not have value to a third party, knowing that the data is legitimate has value to you.
Arguments like this should have practical value for business. For my business the value is zilch. It is simply not worth it for the hacker to mess with my packet server. I've been running it for 10 years with 0 complaints. And the worst they can do is to temporarily annoy few people. And yes I implement simple proprietary form packet of signing. It took me whole few minutes to develop.
Many games are popular targets for griefers or DDOS attacks, not because the attacker gains anything of value from the attack (and many of them aren't actually hackers, they just download the tools written by others), but instead they do it solely for the perverse happiness they get from knowing that they ruined someone else's day / hour / minute.
Possibly you just haven't seen this problem yet because your game isn't high-profile enough for them to bother with and it doesn't provide enough amusement for them to get a kick out of it.
It is game like by tech used but is not really a game. Unlikely to be noticed. When IF it ever does I'll worry about it the, Not too difficult as there are libs just for that. For now my simple signing works like a charm.
Because author of the article likes to be edgy and parrot the deep truth he learned yesterday - that TCP is "reliable" and UDP is not.
The reality, of course, is somewhat different and muddy. For example, if you have network outage following later by a software crash or a reboot, then all the TCP buffer worth of data (several kilobytes or upto some megabytes - depends on your tuning) is mercilessly dropped. And your application thinks that just because you used TCP the data must have been reliably delieved. To combat this you have to implement some kind of serializing and acking - but the article scoffs us that we are too dumb to implement anything besides basic stream /s
I'm not arguing that TCP is useless, just that UDP has its place and we - mere mortals - can use is too. Where appropriate.
> Because author of the article likes to be edgy and parrot the deep truth he learned yesterday
I will point out that "author of the article" is one of the core contributors in the IETF Media-over-QUIC working group (which is an effort to standardize how one might build these real-time applications over QUIC) and has been working in the real-time media protocols space for 10+ years.
The author recognizes that the title is clickbait, but the key point of the article is that you probably don't want to use raw UDP in most cases. Not that UDP is inherently bad (otherwise he wouldn't be working on improving QUIC).
The thing is, when you need that low latency, you don't want most of the features of more robust streams.
When you are using UDP the correct way to handle out of order delivery is to just ignore the older packet. It's old and consequently out of date.
Figuring out how to solve any mess caused by mistransmission necessarily has to be done at the application level because that's where the most up to date data is.
Most of TCP woes comes from high-bandwith latency-sensitive stuff like HFT and video, but TCP isn't particularly good for low-bandwidth high-latency networks either (e. g. NB-IoT with 10 seconds worst case RTT):
- TCP will waste roundtrips on handshakes. And then some extra on MTU discovery.
- TCP will keep trying to transmit data even if it's no longer useful (same issue as with real-time multimedia).
- If you move into a location with worse coverage, your latency increases, but TCP will assume packet loss due to congestion and reduce bandwidth. And in general, loss-based congestion control just doesn't work at this point.
- Load balancers and middleboxes (and HTTP servers, but that's another story) may disconnect you randomly because hey, you haven't responded for four seconds, you are probably no longer there anyway, right?
- You can't interpret the data you've got until you have all of it - because TCP will split packets with no regards to data structure. Which is twice as sad when all of your data would actually fit in 1200 bytes.
And TCP wants to pretend that all your data arrives as a linear stream one after the other; and keeps you from seeing package n+1, if you haven't received package n, yet.
That's useful for some things, but often you can make use of package n+1, even when package n hasn't arrived, yet.
For example, when you are transferring a large file, you could use erasure encoding to just automatically deal with 5% package loss. (Or you could use a fountain code to deal with variable packet loss, and the sender just keeps sending until the receiver says "I'm done" for the whole file, instead of ack-ing individual packages.
Fountain codes are how deep space probes send their data back. Latency is pretty terrible out to Jupiter or Mars.)
> And TCP wants to pretend that all your data arrives as a linear stream one after the other;
That’s one of the reasons people use TCP.
> For example, when you are transferring a large file, you could use erasure encoding to just automatically deal with 5% package loss.
It’s never that simple. You can’t just add some erasure coding and have it automatically solve your problems. You now have to build an entire protocol around those packets to determine and track their order. You also need mechanisms to handle the case where packet loss exceeds what you can recover, which involves either restarting the transfer or a retransmission mechanism.
The number of little details you have to handle quickly explodes in complexity. Even in the best case scenario, you’d be paying a price to handle erasure coding on one end, the extra bandwidth of the overhead, and then decoding on the receiving end.
That’s a lot of complexity, engineering, debugging, and opportunities to introduce bugs, and for what gain? In the file transfer example, what would you actually gain by rolling your own entire transmission protocol with all this overhead?
> Fountain codes are how deep space probes send their data back. Latency is pretty terrible out to Jupiter or Mars.)
Fountain codes and even general erasure codes are not the right tool for this job. The loss of a digital packetized channel across the internet is very different than a noisy analog channel sent through space.
> It’s never that simple. You can’t just add some erasure coding and have it automatically solve your problems. You now have to build an entire protocol around those packets to determine and track their order.
Yes, and you'd want to mostly build this extra protocol only once, and stick it into a library.
> You also need mechanisms to handle the case where packet loss exceeds what you can recover, which involves either restarting the transfer or a retransmission mechanism.
Or you can use a fountain code, and just keep transmitting.
> The number of little details you have to handle quickly explodes in complexity. Even in the best case scenario, you’d be paying a price to handle erasure coding on one end, the extra bandwidth of the overhead, and then decoding on the receiving end.
Well, you can also use a simpler mechanism: you transmit all the packages from A to B once, then at the end B tells A which packages get lost, and A sends those again. Repeat until you have everything.
That way needs more back-and-forth communication, but doesn't need any fancy error correcting code.
> Fountain codes and even general erasure codes are not the right tool for this job. The loss of a digital packetized channel across the internet is very different than a noisy analog channel sent through space.
Yes, the loss model is different. However you can eg interleave your bits to get something that close enough to work.
The main reason you don't need to use error correcting codes, is that transmitting feedback to the sender is typically a lot cheaper on the internet than in outer space. Not least because even bad latencies are typically measured in seconds at most, not minutes or hours.
(You could however use these codes when for some reason you have a very asymmetrical link. Eg if you have internet via geo-stationary satellite, or if one sender is broadcast to lots and lots of different receivers, and doesn't want to deal with each of them individually.)
> And TCP wants to pretend that all your data arrives as a linear stream one after the other;
This is actually very nice for an application developer. Probably a tiny bit less nice if you are running popular services taking 100k req/s.
Yes, that is convenient. But it's an abstraction you don't always need, but still always pay for.
That's why I would suggest that were possible the kernel should only expose something at most as high level as UDP, and we build the TCP abstractions on top of that in user level. Of course, you wouldn't want to re-write everything from scratch all the time: you'd use libraries.
The nice thing about user level libraries is that you can swap them out for better versions or for different abstractions without any privileged access.
> TCP will waste roundtrips on handshakes. And then some extra on MTU discovery.
This is especially true when using TLS (which is at least a major use case nowadays if not the default assumption); the TCP handshake has to complete before the TLS handshake can occur, whereas QUIC has protocol-level support for doing TLS negotiation as part of the initial handshake. Having the network protocol and the encryption be composable rather than tightly coupled feels more elegant, it's hard for me to convince myself that being able to save a round trip per connection isn't a more practical benefit in a world where we now expect pretty much everything to be encrypted in transit.
> feels more elegant
Yet, it's not.
That kind of elegance ends up in application level implementations like STARTTLS and socket data becoming unusable on the middle of a connection while both parties scramble to decide what to throw away and what to interpret. It does really look like the real thing, but it's decoupling things that are very coupled by definition.
maybe the lesson is that "composable" shouldn't mean "just nested".
> If you move into a location with worse coverage, your latency increases, but TCP will assume packet loss due to congestion and reduce bandwidth. And in general, loss-based congestion control just doesn't work at this point.
Don't lower levels of wifi below TCP now do often their own retransmit, even if it is just from weak/noisy signal and not a sensed collision?
Is NB-IoT actually being used these days? I remember it was hyped up but then the buzz disappeared.
The biggest fucking problem with NB-IoT or LTE-M is the goddamn carriers--both managerial and technical.
1) Managerial--Just like back in the old days when the carriers stupidly fought the change from voice to data they are stupidly fighting the change from humans using the network to machines using the network. And it's just as stupid this time around and it will be just as stupidly profitable for them after the fact.
2) The carriers refuse to support older firmware so you have to be able to do multi-megabyte firmware updates every 3 months. This is anathema to anything running on a battery as your firmware updates are like 2 to 4 orders of magnitude(not exaggerating) larger than the total actual data your device will transmit over its lifetime.
The carrier cranio-rectal impaction is why LoRA still continues to exist and expand.
3) Certified firmware*
* the telecom can and does blacklist IMEI numbers for unsigned firmware
* it is illegal to fix bugs that cause eSIM hardware lock-ups
* You can catch g05t5 on the carrier VPN network sandbox hitting each client
Sometimes it is better to wipe the slate clean, and start again... LoRAWAN certainly does make sense in some use-case applications.
https://www.youtube.com/watch?v=hMOwbNUpDQA
> g05t5
What does that mean?
Don't worry about it, as it is difficult to explain on a public forum...
Have a great day, =3
https://youtu.be/uhCRK_5P51E?si=OkRLqjJzDSFMqMXk&t=1
We've used Sigfox with success but then they went almost bankrupt and I honestly didn't check where they are now or if it's safe to use them going forward. I still want devices to function 10 years from now.
In my corner of the world (electric/water/gas utilities), we had been using proprietary techniques that matched the "NB-IoT" description for 20 years, and they weren't new then. It was fun to watch the public dialogue walk the same path for a while and rediscover some of the main tricks, but there were other important tricks left untouched (broadcast firmware downloads, highly compressed debug logs, low power clock synchronization so that you can do timeslots for ultra low power, mesh heuristics) probably because they were a massive pain in the rear and the people who would have risen to the occasion saw that the major ecological niches were already occupied by players who got in 30 years ago when this all became viable. (Disclaimer: I didn't actively chase down NB-IoT talks so I might just have been missing these things.)
If your market is large enough that managing your own infrastructure is attractive (vs cellular), you were also able to afford a few EEs and RF guys 30 years ago, and your needs are already met by a competitive ecosystem of players who have been walking this path for a long time. It doesn't leave much to compete on except price, and you don't want to compete on price.
That makes me envious. I've figured out the debug log compression part myself, but the rest is beyond my reach.
> If your market is large enough that managing your own infrastructure is attractive
This. I imagine proprietary is perfect when you are deploying in a single country with single RF plan. And your customer utility company that is willing to maintain a bit of extra infrastructure. But if you want to deploy on several sites scattered across all ITU regions, and each site is a huge plant with devices from maybe fifty different suppliers, it's just a no-go.
It definitely does, at least in Europe, not sure how things are on the other side of the pond. For the user equipment there is no real advantage over LTE-M, actually is LTE-M better for our case (send a single packet a day and go to sleep). But then I don't see the picture from the operator side, maybe the radio savings actually matter for them. Some operators either don't have LTE-M at all or don't allow PSM on LTE-M because screw you that's why. On NB-IoT PSM is mandatory at least. And yes, LoRa/Sigfox/other non-cellular LPWAN might technically be better IF you provide the base stations. Nobody wants to do that. With cellular you can just mail the device and cross your fingers the operators haven't misconfigured roaming.
That's quite interesting. And the promised battery lifespans of NB-IoT device, did it hold some truth? In the beginning we heard about 10 year life spans for simple well designed devices.
You'll have to wait 10 years to see. But on paper it does (we aim for 15) and we have devices in field for some three years.
But for the "simple well-designed" - the problem is, the whole thing is a black box for application developer, and not a bugless one. So much of the "well-designed" part is to figure out when is the right time to pull modem's proverbial power cord. If you don't, it'll gladly eat up your battery in 48 hours after a cell handover goes wrong with no warning.
We have millions of devices in the field older than that on the same D-cell LiSOCl2 battery they shipped with. Some of them even take magnetic measurements with powered excitation!
No. Because carriers are idiots.
The carriers require you to allow them to force a firmware upgrade on you. This kills your battery dead long before anything you would do on the device will.
Technically, however, 10 year battery life is generally unreasonable. Most batteries that exist self-discharge over 10 years. 5 years is doable, but requires quite solid engineering. 3 years is pretty reasonable for most things.
If you want 10 year lifetimes, I think you specifically need lithium thionyl chloride batteries.
We use straight UDP datagrams for streaming high-frequency sensor data. One of our R&D people built a new system that uses quic and solves most of our problems with out-of-order delivery. We still use datagrams over UDP for everything because we have to support some 3rd party sensors out of the box without adapters, and UDP is all they can do.
Also pretty much every media art system in the world just uses OSC over UDP
And ArtNet also over UDP.
Yeah, high frequency sensor data was on my mind too. Do you have any figures on the power saved wrt TCP?
In general, the wireless handshake on G5/LTE/Wifi/Starlink will dwarf the traffic of sensors for power use. For old-style satellite mailboxes, the traffic is better batched in 45min chunks anyway (the 5W transmitters are usually only active for well under 20 seconds per message, and use a super-cap to handle the RF power draw spike.)
Naive UDP protocols have reachability problems in numerous scenarios, variable costs can balloon (concurrency with polling links is dumb), and can open a hole in your infrastructure at scale without special equipment mods. Only if you have a tier 3 or better WAN trunk or cloud-center should you even consider something silly like QUIC.
YMMV, and I love the AstroTurf on YC... lol ;)
what language was the system build with? thats super cool!
hubris++
Thanks, I'll see myself out... lol =)
You could have used RoCE for reliability over UDP
"solves most of our problems"
Indeed, until the problems it creates begin to feature.
Have a great day =3
“Those are my problems and if you don’t like them . . . Well, I have others”
—G. Marx
https://en.wikipedia.org/wiki/Law_of_holes
;)
Thats fine for a simple cavity, get a filling and brush more often; but sometimes you have a three-holed sphere to put on one leg at a time, like anyone else.
https://en.wikipedia.org/wiki/Pair_of_pants_(mathematics)
This may seem like a minor nit, but I think there is a problem with using the term "unreliable" to describe UDP. The more commonly used term, and IMHO better term, is "best-effort" [1]. UDP makes its best effort to deliver the datagrams, but the datagrams may be dropped anyway. But it does not make UDP inherently unreliable.
[1] https://en.wikipedia.org/wiki/Best-effort_delivery
IMO "best-effort" is euphemistic and confusing to people outside of this space, even (especially?) to native English speakers. (I would probably have described UDP as "reasonable effort" and TCP as "best effort", if not for the existing terminology.)
I recall my computer networking professor describing this as "Best-effort means never having to say you're sorry". In practice, best-effort does not mean you try your hardest to make sure the message gets from A to B, it means that you made an effort. Router in the path was congested? Link flap leading to blackholing on the order of 50ms before fast reroute kicks in? Oh well, we tried.
Meanwhile, TCP's reliable delivery will retry several times and will present an in-order data stream to the application.
Reliable vs unreliable might be bad terminology, but I don't think best-effort is any better.
My experience with unreliable systems is that they're great something like 95% of the time, and they're great for raw throughput, but there are many cases where that last 5% makes a huge difference.
The question is 'who deals with dropped packages'? In TCP, the answer is: 'the protocol'. In UDP the answer is 'the next layer of abstraction' (eg the app or some library).
You can build a 'reliable' protocol on top of UDP, and still not get TCP.
Eg if you want to transfer a large file that you know up front, then TCP's streaming mechanism doesn't make too much sense. You could use something like UDP to send the whole file from A to B in little chunks once, and at the end B can tell A what (numbered) chunks she's missing.
There's no reason to hold off on sending chunk n+1 of the file, just because chunk n hasn't arrived yet.
> There's no reason to hold off on sending chunk n+1 of the file, just because chunk n hasn't arrived yet.
Congestion control comes to mind -- you don't necessarily know what rate the network supports if you don't have a feedback mechanism to let you know when you're sending too fast. Congestion control is one of those things where sure, you can individually cheat and possibly achieve better performance at everyone else's expense, but if everyone does it, then you'll run into congestive collapse.
> You can build a 'reliable' protocol on top of UDP, and still not get TCP.
I agree -- there are reliable protocols running on top of UDP (e.g. QUIC, SCTP) that do not behave exactly like TCP. You don't need an in-order stream in the described use case of bulk file transfer. You certainly don't need head-of-line blocking.
But there are many details and interactions that you and I wouldn't realize or get right on the first try. I would rather not relearn all of those lessons from the past 50+ years.
> But there are many details and interactions that you and I wouldn't realize or get right on the first try. I would rather not relearn all of those lessons from the past 50+ years.
Oh, the model I had in mind was not that everyone should write their network code from scratch all the time, but rather that everything that's higher level than datagrams should be handled by unprivileged library code instead of privileged kernel level code.
If speed is an issue, modern Linux can do wonders with eBPF and io_uring, I guess? I'm taking my inspiration from the exokernel folks who believed that abstractions have no place in the operating system kernel.
interestingly, the exokernel approach is suited to particular cases. for instance, a single application (regardless of whether it's multiple sessions, diversity of RTT, etc). after all, "get the packets into userspace with as little fuss as possible" would be the right goal there.
the unix model is different: it's basically premised on a minicomputer server which would be running a diverse set of independent services where isolation is desired, and where it makes sense for a privileged entity to provide standardized services. services whose API has been both stable and efficient for more than a couple decades.
I think it's kind of like cloud: outsourcing that makes sense at the lower-scale of hosting alternatives. but once you get to a particular scale, you can and should take everything into your own hands, and can expect to obtain some greater efficiency, agility, autonomy.
> the unix model is different: it's basically premised on a minicomputer server which would be running a diverse set of independent services where isolation is desired, [...]
Exokernels provide isolation. Secure multiplexing is actually the only thing they do.
> and where it makes sense for a privileged entity to provide standardized services. services whose API has been both stable and efficient for more than a couple decades.
Yes, standardisation is great. Libraries can do that standardisation. Why do you need standardisation at the kernel level?
That kind of standardisation is eg what we are doing with libc: memcpy has a stable interface, but how it's implemented depends on the underlying hardware; the kernel does not impose an abstraction.
Is there a popular protocol that uses this scheme today? I've often thought that this would be a superior way to transfer/sync data. and have always wondered why it wasn't common.
The closest I can think of is the old FSP protocol, which never really saw wide use. The client would request each individual chunk by offset, and if a chunk got lost, it could re-request it. But that's not quite the same thing.
Bittorrent?
BitTorrent is TCP; only the tracker announce protocol is UDP.
TCP kind of supports that with selective acknowledgements these days, I think.
SACKs have been in TCP for 25-30 years now (Widely adopted as part of New Reno, although RFC 2018 proposed the TCP option and implementation back in 1996).
That said, the typical reason why TCP doesn't send packet N+1 is because its congestion window is full.
There is a related problem known as head-of-line blocking where the application won't receive packet N+1 from the kernel until packet N has been received, as a consequence of TCP delivering that in-order stream of bytes.
Sorry, I didn't literally mean n+1. What I mean is probably better described by 'head of line blocking'.
Basically, when transmitting a file, in principle you could just keep sending n+k, even if the n-th package hasn't been received or has been dropped. No matter how large k is.
You can take your sweet time fixing the missing packages in the middle, as long as the overall file transfer doesn't get delayed.
The term "best effort delivery" in networking is a weasel term that is no better than "unreliable". It should probably be burned.
"Effort" generally refers to some sort of persistence in the face of difficulty. Dropping a packet upon encountering a resource problem isn't effort, let alone best effort.
The way "best effort" is used in networking is quite at odds with the "best efforts" legal/business term, which denotes something short of a firm commitment, but not outright flaking off.
Separately from the delivery question, the checksums in UDP (and TCP!) also poorly assure integrity when datagrams are delivered. They only somewhat improve on the hardware.
> Dropping a packet upon encountering a resource problem isn't effort, let alone best effort
The dropping part isn’t the effort; the forwarding part is.
> the checksums in UDP (and TCP!) also poorly assure integrity when datagrams are delivered.
That’s true, but it’s becoming less of a problem with ubiquitous encryption these days (at least on WAN connections).
I mean that in the same rhetorical sense of "standing around leaning on your shovel isn't effort, let alone best effort". Of course we all understand that what is effort is digging the ditch.
Anyway, so the question is, if typical IP forwarding is "best effort" ... what is an example of poor effort, and what exhibits it?
Should be "at most once" and "at least once", it's already used in other distributed systems with the same kind of problem
> but the datagrams may be dropped anyway
That's what I thought "unreliable" meant? I can't really tell what misconception you are trying to avoid.
I think the issue is that there's a stigma around UDP, largely borne from not using it effectively. I'm not sure this is the right way to address the stigma though.
Ultimately, TCP vs UDP per se is rarely the right question to ask. But that is often the only configuration knob available, or at least the only way to get away from TCP-based protocols and their overhead is to switch over to raw UDP as though it were an application protocol unto itself.
Such naive use of UDP contributes to the stigma. If you send a piece of data only once, there's a nontrivial chance it won't get to its destination. If you never do any verification that the other side is available, misconfiguration or infrastructure changes can lead to all your packets going to ground and the sender being completely unaware. I've seen this happen many times and of course the only solution (considered or even available in a pinch) is to ditch UDP and use TCP because at least the latter "works". You can say "well it's UDP, what did you expect?" but unfortunately while that may have been meant to spur some deeper thought, it often just leads to the person who hears it writing off UDP entirely.
Robust protocol design takes time and effort regardless of transport protocol chosen, but a lot of developers give it short shrift. Lacking care or deeper understanding, they blame UDP and eschew its use even when somebody comes along who does know how to use it effectively.
What kind of stigma is there around UDP? I have never heard of this and I have never seen anyone use UDP expecting it to be reliable.
I think the biggest mistake people make with UDP protocols is allowing traffic amplification attacks.
I don't think anyone expects it to be reliable, they just don't plan for packet loss properly.
Plan? Or implement?
Both? I've seen UDP and TCP treated as interchangeable. They're not, even though in many cases the APIs involved make it seem that way. Choosing datagrams and unreliable transport vs. streams and reliable transport affects the rest of the protocol design and behavior, whether the designer wants it to or not and whether the implementer considered the consequences or not.
i agree that "unreliable" isn't a good term for the transport. reliability is a property of the system, not of the transport. and you can make an unreliable system with TCP or a reliable one with UDP
but, "best-effort" implies that it's doing some effort to ensure delivery, when it's really dropping any packet that looks funny or is unlucky enough to hit a full buffer
i like "lossy", but this is definitely one of the two hard problems
We’ve been calling it “unreliable” transport since the 80’s, and that’s what it is. Want your packets to get there? TCP. Don’t care much? UDP.
Oversimplified.
Best effort is a dumb term. There’s no effort.
In a way, TCP is just about as reliable as UDP, but at a different layer: TCP will either forward your entire stream of data in order and without gaps, or it won’t. UDP does the same, but on the per-datagram level.
Reliability is arguably more of a statement about the availability metrics of your underlying network; it doesn’t seem like a great summary for what TCP does. You can’t make an unreliable lower layer reliable with any protocol magic on top; you can just bundle the unreliability differently (e.g. by trading off an unreliability in delivery for an unreliability in timing).
TCP wont always deliver your packets anyway , but it does have a mechanism of timing out if a party believes the other party did not receive something. UDP just means that if one cares for their data to be received, they must verify it themselves.
That's in my opinion what reliable / unrealiable mean in this context.
reliable = it either succeeds or you get an error after some time, unreliable = it may or may not succeed
I concur with the people who think "best effort" is not a good term. But perhaps TCP streams are not reliable enough for TCP to be rightly called a reliable stream protocol. As it turned out, it's not really possible to use TCP without a control channel, message chunking, and similar mechanisms for transmitting arbitrary large files. If it really offered reliable streams that would its primary use case.
Yes, it's not so much 'reliable' or 'best effort', rather the difference is 'which part of your software stack should deal with dropped packages'?
the designers of link layers and protocol implemntations, the army of operators who test signals, insteall repeaters, configure routers and hold conferences about how to manage the global routing system would disagree.
best effort implies 'no, we're not going to be climb that curve and get try to get to 100% reliability, because that would actually be counterproductive from an engineering perspective, but we're going to go to pretty substantial lengths to deliver your packet'
There is minimal effort. If the queue has room the packet gets forwarded. If the situation is any more difficult than that the effort ends and the packet is dropped.
As another reply points out, it has a useful and well-formed meaning, even if you don't like it.
UDP is at-most-once; TCP is exactly-once-in-order.
This sounds like the problem is the term "best-effort" (hand wavy, what's the measure of effort? What's "the best" effort?).
In the end, best-effort is just saying "unreliable" in a fussier way.
>But it does not make UDP inherently unreliable.
Isn't that exactly what it does make it?
If that's not it, then what woud an actual "inherently unreliable" design for such a protocol be? Calling an RNG to randomly decide whether to send the next packet?
Fire-and-forget?
There’s nothing intuitive about what “best effort” means, but if you know anything about udp and tcp you know which is “reliable.”
> UDP makes its best effort
TCP tries really hard, too, you know.
IMO stream abstractions make it too convenient to write fragile programs which are slow to recover from disconnections (if they do at all) and generally place too many restrictions on the transport layer. Congestion control is definitely needed but everything else seems questionable.
In a datagram-first world we would have no issue bonding any number of data links with very high efficiency or seamlessly roaming across network boundaries without dropping connections. Many types of applications can handle out-of-order frames with zero overhead and would work much faster if written for the UDP model.
> In a datagram-first world we would have no issue bonding any number of data links with very high efficiency or seamlessly roaming across network boundaries without dropping connections. Many types of applications can handle out-of-order frames with zero overhead and would work much faster if written for the UDP model.
So your argument is that software isn’t written well because TCP is too convenient, but we’re supposed to believe that a substantially more complicated datagram-first world would have perfectly robust and efficient software?
In practice, moving to less reliable transports doesn’t make software automatically more reliable or more efficient. It actually introduces a huge number of failure modes and complexities that teams would have to deal with.
It’s true that a stream-oriented abstraction puts constraints on the communication channel that don’t exist for a datagram oriented protocol.
> So your argument is that software isn’t written well because TCP is too convenient, but we’re supposed to believe that a substantially more complicated datagram-first world would have perfectly robust and efficient software?
I think you can make a better argument:
Your operating system should offer you something like UDP, but you can handle all the extra features you need on top of that to eg simulate something like TCP, at the level of an unprivileged library you link into your application.
That way is exactly as convenient for 'normal' programmers as the current world: you get something like TCP by default. But it's also easy for people to innovate and to swap out a different implementation, without needing access to the privileged innards of your kernel.
> Congestion control is definitely needed but everything else seems questionable.
Even congestion control can be optional for some applications with the right error correcting code. (Though if you have a narrow bottleneck somewhere in your connection, I guess it doesn't make too much sense to produce lots and lots more packages that will just be discarded at the bottleneck.)
As mentioned in the article, this doesn't handle the case of buffer bloat: packets all eventually arrive rather than being dropped, but increasingly late, and only backing off can help reduce the latency.
Does UDP suffer from buffer bloat? I thought it was only TCP?
Oh, it looks like UDP packages are also buffered?
Update: it looks like I learned something new today. UDP can also suffer from bufferbloat. (I thought UDP was done without buffers for some reason..)
Out of interest, what kind of applications are you thinking of? What systems are common written using TCP, that could switch to UDP?
Clearly websites, audio and video generally don't work with out-of-order frames -- most people don't want dropped audio and video.
Some video games are happy to ignore missed packets, but when they can they are already written in UDP.
Any data block that must be fully received before processing comes to mind (e.g. websites’ html), just request retransmission of parts that didn’t make it in first pass. Funnily enough it’s (realtime) video and audio that already uses UDP due to preferring timely data to complete data.
On the contrary, for me it’s hard to imagine video game with missed packets as the state can get out of sync too easily, you’ll need eventual consistency via retransmission or some clever data structures (I know least about this domain though)
I'm not expert in this area, but my understanding is that for latency sensitive video games, on the client side, state is predicted between updates, e.g. the state may be updated via the network, at say, roughly 10Hz (probably faster, but probably less than 60Hz), but updated locally at 60Hz via interpolation, dead reckoning and other predictive/smoothing heuristics, a few missed updates just means a bit more prediction is used. "State" is not usually transmitted as one lump "frame" at a time, but rather per-game unit (per spaceship, asteroid, robot, or whatever is in the game) or per some-small-group of units. When some updates are delayed too long, you might get some visible artifacts, "rubber-banding", "teleportation" or "warping", etc. often lumped together by players under the umbrella term "lag". For out-of-order packets, older packets might be dropped as newer packets may already have been applied to the state of some game units (typically there's some timestamps or monotonically increasing counter associated with state updates for each unit used by the prediction/interpolation/smoothing heuristics) and the state is usually represented in absolute terms rather than relative terms (e.g. (x,y) = (100,100), rather than x+=10, y+=10, so that any update may be applied in isolation.)
Many video games just transmit the whole state the player should be able to see every tick. While some games send a diff of some kind, sometimes that turns out to not be worth it. This is particularly true in the games which care most about lag (think mario kart, or street fighter and friends).
One thing not mentioned often is that a lot of networks will drop UDP packets first when encountering congestion. The thinking is that those packets will not re-transmit, so it’s an effective means to shed excess traffic. Given we now have protocols that aggressively re-transmit on UDP, I wonder how that has changed things. I do seem to remember QUIC having re-transmit issues (vs HTTP1/2) years ago because of this.
I've attempted to replace the clickbait title* using terms from the article itself, but if someone can suggest a more representative phrase from the article, we can change it again.
(* in keeping with the HN's title guideline: "Please use the original title, unless it is misleading or linkbait" - https://news.ycombinator.com/newsguidelines.html)
>> The common wisdom is: >> >> use TCP if you want reliable delivery >> >> use UDP if you want unreliable delivery >> What the *(& does that mean? Who wants unreliability?
I dont agree with the premise of this article, UDP isnt for unreliability, it provides a tradeoff which trades speed and efficiency and provides best-efforts instead of guarantees.
It makes sense depending on your application. For example, if I have a real-time multi-player video game, and things fall behind, the items which fell behind no longer matter because the state of the game changed. Same thing for a high-speed trading application -- I only care about the most recent market data in some circumstances, not what happened 100ms ago.
That isn’t the premise of the article, that is the “common wisdom” the author corrects as the article goes on. The author goes on to list video games as an example of where UDP makes sense, as well as live video.
That's basically what the article says if you keep on reading. It states the "common wisdom" at first to disagree with it.
Well then it's clickbait.
Silly clickbait title, which the author even admits up front.
UDP and TCP have different behaviour and different tradeoffs, you have to understand them before choosing one for your use case. That's basically it. No need for "Never do X" gatekeeping.
My apologies if the star in front of "never" was added to the title in the last ten minutes after you posted your comment. But that star is clearly there to indicate "fine print ahead"; in a title, it's pretty obvious that it means that the article is not about to "gatekeep" people out of UDP. (In fact, at the end of the article, the author suggests to use QUIC which, you guessed it, is based on UDP.)
What should be based on datagrams:
- Local discovery (DHCP, slaac, UPnP, mDNS, tinc, bittorrent)
- Broadcasts (Local network streaming)
- Package encapsulation (wireguard, IPSec, OpenVPN, vlan)
One missing from your list is real-time media. Retransmission, or even just buffering for reordering, adds latency so it's better to take the hit of a loss with error correction or packet-loss-concealment
Yeah exactly. With realtime, a dropped packet is often too old by the time it's re-transmitted. It's actually harmful to wait for it when you'd rather just skip it and move on and stay as close to realtime as possible. It's a very different usecase from most uses.
I do wish QUIC allowed carrying streams that were useful for realtime in conjunction with allowing reliable streams. Using MPEG-TS over SRT to have it just spam metadata to handle the unreliableness is janky. It would be far nicer to have a reliable stream for metadata, then an unreliable one for realtime streaming.
Hey I'm sure you're right, but as someone with less low-level networking expertise, I was wondering if you could explain why you'd want datagrams for these use-cases?
TCP is a back & forth between two hosts and thus cant do "shouting blindly into the network" things. That rules out local discovery and media broadcast.
Package encapsulation is bad over TCP because if the encapsulated data is TCP itself, you have congestion control twice. On congested networks, this results in extra slowdowns that can make the connection unusable.
Local discovery is itself based on broadcast/multicast, both of which only work over UDP (TCP doesn't provide any mechanism for broadcast streams).
And games.
The whole point of the article is that you don't need to go all the way down to raw datagrams to achieve the kind of low latency that is needed for things like games/VOIP/livestreaming
It totally depends on the kind of game.
Nahhh. While most of the applications/cases will be using session-based connections, there are uses for using datagrams directly — don’t be afraid. Yes, you will have to take care of many more details yourself. And as a side line, it is a great way of learning the low level aspects of networking.
The SteamNetworkingMessages API for game development does a good job of making this available if you want it without caring about the internals for that use case.
Quic is implemented over UDP. It's literally running over datagrams.
Everything at some layer has to run over datagrams, since that's what IP is.
This is literally the point of the article: if you want to create a protocol over raw datagrams, you have to implement a lot of things that are very hard to get right, so you should just use QUIC instead, which does them for you.
"You" is doing all the hard work. Apps should use reliable sessions and transport, almost all the time? No disagree. The exceptions are understood. But don't pretend the substrate is that reliable data stream. "We" have to construct it almost always. We do reliable for you, over UDP.
I don't do this stuff any more, but I worked on OSI transport and remote operations service mapped to UDP and other protocols back in the 80s
> The bytes within each stream are ordered, reliable, and can be any size; it’s nice and convenient. Each stream could be a video frame […] But you can tell the QUIC stack to focus on delivering important streams first. The low priority streams will be starved, and can be closed to avoid wasting bandwidth.
Is the author saying that with QUIC I can send a "score update" for my game (periodic update) on a short-lived stream, and prevent retransmissions? I'll send an updated "score update" in a few seconds, so if the first one got lost, then I don't want it to waste bandwidth retransmitting. Especially I don't want it retransmitted after I've sent a newer update.
I think you're looking for RFC 9221, "An Unreliable Datagram Extension to QUIC."
You don't need datagrams for this. You can set the stream to discard itself as soon as packet loss is detected
That doesn't quite provide the same experience, does it?
For something like a scoreboard you don't want the message to be discarded once any packet loss is detected. As long as message N is the most recent one, message N should be retried until it succeeds. You just want to stop retrying N once N+1 becomes available - or ideally even successfully delivered.
In a game context, where we're spewing state updates out 10-60x per second, there is always a newer packet in flight already, so discard-on-loss produces the same result.
In your scoreboard example, you probably just want an actually reliable stream for that - it's not like scores are rolling in so fast that it's beneficial to deal with it as unreliable.
It mentions that the video game industry uses UDP but then fails to further address that use case. So, should competitive shooter video games switch to QUIC? Is that even supported across all the various gaming platforms?
> It mentions that the video game industry uses UDP but then fails to further address that use case
Video games tend to use UDP for the same reason everyone else mentioned does: timeliness. You want the most recent position of the various game objects now, and you don't give a shit about where they were 100ms ago.
The proposed solution of segmenting data into QUIC streams and mucking with priorities should work just fine for a game.
> Is that even supported across all the various gaming platforms?
QUIC itself is implemented in terms of datagrams, so if you have datagrams, you can have QUIC.
But the proposed solution provides no value over datagrams, while it has plenty of downsides.
Let me rephrase the problem here for a simple FPS game:
The entire world state (or, the subset that the client is supposed to know about) is provided in each packet sent from the server. The last few actions with timestamps are in each packet sent from the client. You always want to have the latest of each, with lowest possible latency. Both sides send one packet per tick.
You do not want retransmission based on the knowledge that a packet was lost (the RTT is way too long for the information that a packet was lost to ever be useful, you just retransmit everything every tick), you do not want congestion control (total bandwidth is negligible, and if there is too much packet loss to maintain what is required, there is no possible solution to maintain sufficient performance and you shouldn't even try), and none of the other features talked about in the post add anything of value, either.
It reads like someone really likes QUIC, it fit well into their problems, and they are a bit too enthusiastic about evangelizing it.
In practice most games also need reliable side channels for control messages/chat/etc, and so they end up building optional reliable streams over UDP... and at the end of this path lies something that looks a lot like a (less thoroughly designed/tested) version of QUIC
My spidey sense is that almost all games used a custom UDP solution. I could be wrong! I made a Twitter poll to try and get info.
You’re right that a custom reliable UDP solution is going to wind up QUIC-like. On the other hand it’s what games have been doing for over 20 years. It’s not particularly difficult to write a custom layer that does exactly what a given project needs.
I don’t enough about QUIC to know if it adds unnecessary complexity or not.
Yeah, I don't know how many games actually use a fully custom layer, versus forking something like enet/RakNet
Why not make a second connection using another protocol (possibly TCP)?
You also don't want to share the routing performance, congestion treatment, OS queue, and probably a lot of other stuff.
NAT traversal, historically. Reliably punching one hole through the local NAT gateway is bad enough, adding a second protocol into the mix was asking for trouble.
Tribes, and various quake mods, used to use a TCP side channel for chat and control messages. Tribes famously used irc, and you could connect to a tribes server with any irc client to chat with your friends in game
> Video games tend to use UDP for the same reason everyone else mentioned does: timeliness. You want the most recent position of the various game objects now, and you don't give a shit about where they were 100ms ago.
This is only true for games that can replicate their entire state in each packet.
There are many situations where this is infeasible and so you may be replicating diffs of the state, partial state, or even replicating the player inputs instead of any state at all.
In those cases the "latest" packet is not necessarily enough, the "timliness" property does not quite cover the requirements, and like with most things, it's a "it depends".
With those requirements raw datagrams also don't fit the bill, so you'll need to build some sort of reliable stream abstraction over UDP (with optional unreliable delivery for the state which you can replicate quickly)... and now we're closing in on QUIC territory
I thought QUIC promised you get your data eventually? In that case it wouldn't be great for games, because you usually don't want packets that got dropped, as the next packet will replace it anyway?
> I thought QUIC promised you get your data eventually?
QUIC optionally promises you that, you are free to opt out. For example, take a look at the QUIC_SEND_FLAG_CANCEL_ON_LOSS flag on Microsoft's QUIC implementation.
QUIC is implemented in userspace, it doesn't need platform support beyond the basic networking stack.
No, QUIC is mostly for Google to enforce E2E advertisment.
I wonder if this guy thinks video GOPs should all be hundreds of frames long because P frames are so much smaller than I frames and since in his world we NEVER use an unreliable network you might as well.
I wonder if the DiffServ[0] bits in IPv6 could be another way to prevent bufferbloat from affecting real-time datagrams? Or are they like IPv4's ToS[1] bits, which I think were never implemented widely (or properly) enough for any software to bother with?
[0] https://en.wikipedia.org/wiki/Differentiated_services
[1] https://en.wikipedia.org/wiki/Type_of_service
DSCP / ToS is used for traffic classification within a network (which can be used to select router policy), but generally not across autonomous systems. You'd have to get everyone to agree on what various traffic markings mean, and, more critically, which traffic deserves to be marked in certain ways.
For instance, suppose that you wanted a special traffic class that's prioritized over everything else. You really don't want just anyone to be able to send you that kind of traffic, since a DoS attack can easily starve all the other queues.
Your best bet is usually to match traffic based on other characteristics, e.g. port × protocol.
Ah, I see.
Still, it seems like it would be a useful idea to have an IP bit for "minimize buffering and drop instead, packet is time-sensitive", and that seems safe since the user is requesting fewer resources, not more.
You would think so, but buffers are actually a core part of regular router function. The problem is not inherently router buffers, but rather buffers that don't fully drain (bufferbloat) or that fill up (congestive drops).
Think about it this way: a 100Gbps link can process a 1500 byte packet in about 0.12 microsecond. If you have an average of 1,000 packets in that buffer at steady-state, that buffer is contributing a fraction of a millisecond to your overall latency. Meanwhile, if your home router's buffer has the same 1,000 packets for a 1Gbps link, that's 12 milliseconds of latency.
If you had a way to tell a router to never buffer packets, you'd encounter packet loss much sooner, and for no good reason. Buffers are great for smoothing out (small) bursts of traffic (although if you have large bursts on very short timescales, you can easily overwhelm these buffers -- this is why you typically have pacing on the OS level).
If instead you want the router to add no more than X amount of latency, that's suddenly much harder to dictate. (The "correct" value of X depends on your application requirements as well as the number of hops through the network between you and the server you're talking to.)
(Also, while you might think that this should be strictly beneficial for network operators since the user wants fewer resources, encoding exceptions like this into router policy ends up using more of a specialized and expensive kind of memory called TCAM. Assuming that it's even possible to do such a thing.)
Great in depth article from what seems really a person who knows that stuff.
This is a narrow way of looking at UDP applications. The whole HFT, low-latency fintech world is built on top of datagrams. Using TCP would be pure the worst choice possible.
Is that because they mostly use dedicated links and intranets?
A lot of Enterprise messaging is based on UDP, I think on the presumption that corporate networks are just simply going to be a lot more reliable
You are correct, it's either point-to-point, like direct cross-connects with the exchange or it's a relatively small intranet. Additionally, an overwhelming majority of packets flying around are market data. You don't want to re-transmit lost ones because by the time you do, the data is already outdated.
Imagine Ethernet was designed like this and you had to implement mandatory congestion control and other cruft. The layer of the stack that has knowledge of the content should be implementing the congestion control.
Ethernet was designed like this :-)
https://en.m.wikipedia.org/wiki/IEEE_802.2
Back in the olden times, the Windows 3.11 and Windows 95 could even run services atop it using another protocol: https://en.m.wikipedia.org/wiki/NetBIOS_Frames
In fact, the link layer protocols that dealt with congestion and access to media more meticulously, eg Token Ring or FDDI, were generally much more efficient with the use of the media - using near 100% of the potential bandwidth, whereas Ethernet on a shared medium already gets quite inefficient at 30-40% utilization due to retries caused by collisions.
However, the trade off between additional complexity (and thus bigger cost and difficulty of troubleshooting) was such that the much simpler and dumber Ethernet has won.
However, a lot of similar principles are there in Fibre Channel family protocols, which are still used in some data center-specific protocols.
Maybe ... never use on a congested network.
or Never use on a network where congestion is above a certain level
or Never us on a network where this parameter is above a certain level - like network latency
or only use on a LAN not a WAN ....?
LAN can be oversubscribed too, even if your switches have sufficient capacity you can have short packet bursts that fill up the queues. You cannot assume reliable delivery.
Unless you are building software for a pre-established intranet, how do you predict ahead of time whether your software will run on congested networks?
Most end-user software ends up running on a variety of oversubscribed wifi and cell networks
You tell the user "hey, don't use on congested networks". And yeah, since most internet software has this exact disclaimer, it works kinda well.
But people do appreciate software that doesn't have this restriction, and even go out of their way to talk about it (more on mobile networks than on wired connections).
> how do you predict ahead of time whether your software will run on congested networks?
One thing you can do is build in "bad network emulation" into your software, allowing the various pieces to drop packets, delay packets, reorder packets, etc. and make sure it still behaves reasonably with all this garbage turned up.
I've seen UDP used for great effect in video streaming. Especially timely video streaming such as cloud gaming. When waiting a late packet is no longer useful.
Not as popular as it once was, but it is still in use:
https://en.wikipedia.org/wiki/Real-Time_Streaming_Protocol
Cheers =3
RTSP is the control protocol. Some other protocol is needed for the actual audio/video streaming. That's usually RTP, these days.
RTP is a core part of WebRTC, for example.
When you're doing a video call in a web browser, you're using WebRTC, including RTP. In fact, this RTP-via-WebRTC is the only way to send UDP packets from JavaScript!
RTSP is still used by older streaming systems and hardware ecosystems that are slow to change, such as network-connected security cameras. But in newer applications, WebRTC has mostly replaced it. Of course, the QUIC effort is in part an attempt to replace WebRTC, so the wheel continues to turn!
WebRTC still has its own set of issues, and I found it only slightly improved over other options compiling the ARM64 port:
https://github.com/mpromonet/webrtc-streamer.git
I remain unconvinced UDP based streams will ultimately remain in the long-term, but webRTC certainly made it easier to peer a connection. ;)
Hope this one gets captured by quackernews - can’t wait to see its description.
The choice is not UDP vs TCP.
UDP adds minimum over raw sockets so that you don't need root privileges. Other protocols are build on top of UDP.
It's better to use existing not-TCP protocols instead of UDP when the need arises instead of making your own. Especially for streaming.
Used high numbers of UDP packets over intercontinental internet links for mission critical applications for 15 years. 20mbit of UDP carrying RTP. Loss on a given flow is quite rare, and the application helps (via duplication, or retransmits)
As time has progressed increased from nothing to fec to dual-streaming and offset-streaming to RIST and SRT depending on the criticality.
On the other hand I've seen people try to use TCP (with rtmp) and fail miserably. Never* use TCP.
Or you know, use the right tool for the right job.
Network is nothing but datagrams.
I feel like this article misses why people avoid TCP like the plague, and why people use UDP for many applications
1. Routers do all kinds of terrible things with TCP, causing high latency, and poor performance. Routers do not do this to nearly the same extent with UDP
2. Operating systems have a tendency to buffer for high lengths of time, resulting in very poor performance due to high latency. TCP is often seriously unusable for deployment on a random clients default setup. Getting caught out by Nagle is a classic mistake, its one of the first things to look for in a project suffering from tcp issues
3. TCP is stream based, which I don't think has ever been what I want. You have to reimplement your own protocol on top of TCP anyway to introduce message frames
4. The model of network failures that TCP works well for is a bit naive, network failures tend to cluster together making the reliability guarantees not that useful a lot of the time. Failures don't tend to be statistically independent, and your connection will drop requiring you to start again anyway
5. TCP's backoff model on packet failures is both incredibly aggressive, and mismatched for a flaky physical layer. Even a tiny % of packet loss can make your performance unusable, to the point where the concept of using TCP is completely unworkable
Its also worth noting that people use "unreliable" to mean UDP for its promptness guarantees, because reliable = TCP, and unreliable = UDP
QUIC and TCP actively don't meet the needs of certain applications - its worth examining a use case that's kind of glossed over in the article: Videogames
I think this article misses the point strongly here by ignoring this kind of use case, because in many domains you have a performance and fault model that are simply not well matched by a protocol like TCP or QUIC. None of the features on the protocol list are things that you especially need or even can implement for videogames (you really want to encrypt player positions?). In a game, your update rate might be 1KB/s - absolutely tiny. If more than N packets get dropped - under TCP or UDP (or quic) - because games are a hard realtime system you're screwed, and there's nothing you can do about it no matter what protocol you're using. If you use QUIC, the server will attempt to send the packet again which.... is completely pointless, and now you're stuck waiting for potentially a whole queue of packets to send if your network hiccups for a second, with presumably whatever congestion control QUIC implements, so your game lags even more once your network recovers. Ick! Should we have a separate queue for every packet?
Videogame networking protocols are built to tolerate the loss of a certain number of packets within a certain timeframe (eg 1 every 200ms), and this system has to be extremely tightly integrated into the game architecture to maintain your hard realtime guarantees. Adding quic is just overhead, because the reliability that QUIC provides, and the reliability that games need, are not the same kind of reliability
Congestion in a videogame with low bandwidths is extremely unlikely. The issue is that network protocols have no way to know if a dropped packet is because of congestion, or because of a flaky underlying connection. Videogames assume a priori that you do not have congestion (otherwise your game is unplayable), so all recoverable networking failures are 1 off transient network failures of less than a handful of packets by definition. When you drop a packet in a videogame, the server may increase its update rate to catch you up via time dilation, rather than in a protocol like TCP/QUIC which will reduce its update rate. A well designed game built on UDP tolerates a slightly flakey connection. If you use TCP or QUIC, you'll run into problems. QUIC isn't terrible, but its not good for this kind of application, and we shouldn't pretend its fine
For more information about a good game networking system, see this video: https://www.youtube.com/watch?v=odSBJ49rzDo, and it goes over pretty in detail why you shouldn't use something like QUIC
I've been having this "you don't want to use TCP" conversation a lot lately with people who are building real-time voice + LLM applications. Almost everybody who hacks together a voice + LLM prototype starts with WebSockets. Which I totally understand. WebSockets seem like they should do what you want.
And WebSockets do work fine while you're testing.
But WebSockets are TCP, so when you roll things out to real-world users, latency is higher than it should be a lot of the time. Connections randomly drop. You have to try to figure out why different OS configurations are behaving differently. You start building your own keepalive logic. You start trying to figure out how to build metrics and monitoring for audio-over-WebSockets.
The answer is to use UDP, and use a protocol built on top of UDP designed for real-time media (WebRTC).
It's been interesting to me that I've had this exact same conversation with several dozen engineers over the past few months. It's a good reminder that things that seem obvious when you've been doing something a long time aren't obvious to people new to a domain. Even if they are very experienced in some adjacent domain. (In this case, mobile app and web app development.) It's made me think about where my knowledge boundaries are and what "obvious" things I'm ignorant about. (Lots of them, I'm sure.)
If you're interested in WebSockets, WebRTC, and why UDP is the right low-level approach for real-time media, I wrote a a primer about that a few months ago here:
https://www.daily.co/blog/how-to-talk-to-an-llm-with-your-vo...
> 1. Routers do all kinds of terrible things with TCP, causing high latency, and poor performance. Routers do not do this to nearly the same extent with UDP
What things are you thinking of here?
> 2. Operating systems have a tendency to buffer for high lengths of time, resulting in very poor performance due to high latency.
Do they? I can only think of Nagle's algorithm causing any potential buffering delay on the OS level, and you can deactivate that via TCP_NODELAY on most OSes.
> 5. TCP's backoff model on packet failures is both incredibly aggressive, and mismatched for a flaky physical layer. Even a tiny % of packet loss can make your performance unusable, to the point where the concept of using TCP is completely unworkable
That's a property of your specific TCP implementation's congestion control algorithm, nothing inherent to TCP. TCP BBR is quite resistant to non-congestion-induced packet loss, for example [1].
> If you use QUIC, the server will attempt to send the packet again which.... is completely pointless
Isn't one of QUIC's features that you can cancel pending streams, which explicitly solves that issue? In your implementation, if you send update type x for frame n+1, you could just cancel all pending updates of type x if every update contains your complete new state.
[1] https://atoonk.medium.com/tcp-bbr-exploring-tcp-congestion-c...
> 3. TCP is stream based, which I don't think has ever been what I want. You have to reimplement your own protocol on top of TCP anyway to introduce message frames
I'm always curious where frame proponents are going to buffer in-progress frames. OS? I have huge memory concerns. Also take note you want to drop in-order requirement and it means multiple in-progress frames. It's a very nice DoS vector.
IMO one of the worst mistakes made in IP development was not having made a standard protocol for the one thing that pretty much everyone seems to want:
A reliable, unlimited-length, message-based protocol.
With TCP there's a million users that throw out the stream aspect and implement messages on top of it. And with UDP people implement reliability and the ability to transmit >1 MTU.
So much time wasted reinventing the wheel.
That's because TCP evolved as, essentially, abstraction of serial port.
And due to various political & cultural & industrial lobbying, more flexible but complex on their face approaches (OSI) were rejected.
TCP meant you could easily attach a serial terminal to a stream, a text protocol meant you could interact or debug a protocol by throwing together a sandwich, an undergrad, and a serial terminal (coffee too if they do a good job). You could also in theory use something like SMTP from a terminal attached over a TIP (which provided dial-in connection into ARPAnet where you told the TIP what host and what port you wanted).
If you look through some of the old protocol definitions you'll note that a bunch of them actually refer to simplest mode of TELNET as base.
Then in extension of arguably the same behaviour that blocked GOSIP mandate we have pretty much reduced internet to TCP, UDP, and bits of ICMP, because you can't depend on plain IP connectivity in a world of NAT and broken middleboxes.
Sorry for a useless comment but now I feel so stupid for never noticing that pretty much everything done on TCP is actually message exchange, not streams. The mismatch is indeed so unfortunate.
By numbers of communication actions, pretty much everything may actually be a message exchange, but by the amount of data that was transferred for many years the file downloads and uploads had been dominant on the Internet, which are the kind of thing for which TCP has been designed. For a big file more data may be transferred than for thousands or millions of message exchanges.
Nowadays perhaps video streaming, which may be lossy, so TCP is not needed for it (i.e. what is now called streaming does not have the reliability requirements of the TCP streams), might transfer more data on the Internet than file downloads, but that is not a desirable technical evolution, the replacement of downloading with streaming is just a means to extract more money from those who are satisfied by such a service.
> I feel so stupid for never noticing that pretty much everything done on TCP is actually message exchange, not streams.
This generalization isn’t true at all. Streaming data over TCP is extremely common in many applications.
When you download a large file, you’re streaming it to disk, for example.
I guess you could get pedantic and argue that an entire stream of something is actually one large message, but that’s really straining the definitions.
Well, you often have some dependency between messages. Some re-orderings don't matter, but some do.
If you just keep everything in the same order, then you never have to worry about any re-ordering.
I can see why people picked streams as the one-size-fits-all-(but-badly) abstraction.
Because worse is better and if you try to be everything you (don't) get OSI. Also because everyone loves serial ports and dialup modems and TCP just emulates a serial port over packet network. And it wasn't supposed to be one-size-fits-all, rather «let us have telnet while we figuring out the rest» but then ossification happend.
Is it really that unfortunate? What’s the big downside?
There’s a bit of reinvention needed, sure, but adding a framing layer is about the easiest task in networking. If you want a standard way of doing it, these days you can use a WebSocket.
Edit to add: oh, I see, you want reliable but unordered messages. That would definitely be useful sometimes, but other times you do want ordering. If you don’t need ordering, isn’t that pretty much what QUIC does?
SCTP did both reliable/unreliable and ordered/unordered, with QUIC arguably being at least a bit inspired by it, but going over UDP because internet is broken.
EDIT: Internet being broken is also why there's a grease extension for QUIC
What would ‘reliable but unordered’ even mean? What’s the difference between a dropped message and one that’s just late?
The difference is whether you get to process the ones which came after first or you must wait for the retransmission of the earlier ones. FWIW, SCTP provides for this difference.
Here is a good message framing protocol built on TCP: https://www.nasdaq.com/docs/SoupBinTCP%204.0.pdf
If you want large messages just redefine the length from 16 to 32 bits.
It's been used for millions of messages per day for 25 years and hasn't been changed in a long time.
Admittedly it isn't as common as TCP and that's a shame. But it's out there, it's public, it's minimal and it makes sense.
> SoupBinTCP guarantees that the client receives each sequenced message generated by the server in the correct order, [...]
This protocol guarantees too much. It's still a stream.
You quoted the part about sequenced messages. Look at the part about unsequenced messages right after that. It has both.
Thanks!
Actually, the mistake in IP was in not encrypting the next layer, so that networks can block transport protocol innovation by filtering anything other than TCP and UDP. This is the mistake that quic has fixed.
I think that one can be forgiven for pure lack of processing power... We forget just how under powered even host machines were and how resource hungry encryption is...
This was absolutely not feasible at the time IP was being developed. There simply wasn’t enough computational power available or good encryption algorithms invented yet. And you can make a reasonable argument that that’s not the responsibility of that layer of the stack.
QUIC still runs over UDP. Are you referring to MASQUE?
I should perhaps have said "works round" rather than fixes - yes, it has to run over UDP but the quic team had an explicit goal of not allowing the network to interfere further : see sec 3.3 and 7.5 of https://dl.acm.org/doi/pdf/10.1145/3098822.3098842
This now means that innovations based on Quic can occur
Since both QUIC and UDP are transport layers, does this mean that once switches and ISPs start supporting QUIC along side UDP and TCP, that a new version of QUIC can be released which doesn't require UDP?
Switches don't need to change anything to support QUIC, precisely because it uses UDP. So, the deployment of QUIC doesn't involve any technical changes that would enable a version of QUIC that worked directly over IP. It's possible that the defeat of middleboxes would change the motivation of ISPs to filter out alternate protocol numbers, meaning that there might be less resistance to getting them to allow that in the future - but doing so is still a big exercise.
Also, the overhead of a UDP packet is 8 bytes total, in 4 16-bit fields:
- source port
- dest port
- length
- checksum
So, we can save a max of 8 bytes per packet - how many of these can we practically save? The initial connection requires source and dest ports, but also sets up other connection IDs so theoretically you could save them on subsequent packets (but that's pure speculation, with zero due diligence on my part). It would require operational changes and would make any NATs tricky (so practically only for IPv6) Length and checksum maybe you can save - QUIC has multiple frames per UDP datagram so there must be another length field there (anyway, IETF is busily inventing a UDP options format on the basis that the UDP length field is redundant to packet length, so can be used to point at an options trailer). QUIC has integrity protection but it doesn't seem to apply to all packet types - however I guess a checksum could be retained for those that don't only.
So in sum maybe you could save up to 8 bytes per packet, but it would still be a lot of work to do so (removing port numbers especially)
Encryption without authentication is rather useless. Just slows things down for no advantage.
You are looking for SCTP.
I tried using SCTP before QUIC. Unfortunately, while the API looks like you can mix reliable/unreliable messages, in reality the transport doesn't support it because of how FORWARD-TSN works. There were some other pretty fundamental blockers too.
Which you will absolutely not get past pretty much any middleboxes, unless you encapsulate it in UDP.
Now I wonder. Do we have any actual streaming use of TCP, with purely streaming protocol.
Yes. When your messages need to be received and processed in the same order they are sent, then a stream is a good (or good enough) abstraction.
For most applications some re-orderings of messages don't matter, and others would need special handling. So as a one-size-fits-all-(but-badly) abstraction you can use a stream.
> Do we have any actual streaming use of TCP, with purely streaming protocol.
But to give you a proper answer: the stream of keyboard inputs from the user to the server in Telnet (or SSH).
Opens up the question that if you have "messages" are you a stream anymore? Can your stream of messages start mid message for example? Surely a stream can have this happen. Or are you instead messages send over a stream. In which case abstraction of stream instead of reliable message transportation is bit weird.
Wouldn't each input be a single albeit too short message? But this level of granularity really makes little sense...
If you have a reliable-byte-stream abstraction, you only need to add a way to serialise your messages into bytes, and you get a reliable-messages-in-order abstraction for 'free'. That's convenient! And you don't need to teach your lower level protocols about where your higher level messages start and end.
You can also stick your higher level messages into a structure that's more complicated than a stream, eg you can stick them into a tree. Anything you can serialise, you can send.
Of course, the downside to this is that when you don't need these strong guarantees, you are paying for stuff you don't need.
Think about what the feature (potentially) buys you:
(1) zero-copy, zero-allocation request processing
(2) up to a 2x latency reduction by intermingling networking and actual work
(3) more cache friendliness
(4) better performance characteristics on composition (multiple stages which all have to batch their requests and responses will balloon perceived latency)
If you have a simple system (only a few layers of networking), low QPS (under a million), small requests (average under 1KB, max under 1MB), and reasonable latency requirements (no human user can tell a microsecond from a millisecond), just batch everything and be done with it. It's not worth the engineering costs to do anything fancy when a mid-tier laptop can run your service with the dumb implementation.
As soon as those features start to matter, streaming starts to make more sense. I normally see it being used for cost reasons in very popular services, when every latency improvement matters for a given application, when you don't have room to buffer the whole request, or to create very complicated networked systems.
High quality VOD (ie streaming a 4K movie). HTTP block file systems, each block needs to be streamed reliably to fulfill the read() call plus read ahead.
For VOD can I just open connection and send single message and then stream will continue forever? And HTTP is message oriented protocol. I can't just send infinite length HTTP message. Which would be processed as it arrives or can I? Meaning can I upload something not that small like terabyte of video data over HTTP?
Yes for everything. In HTTP1 it’s a chunked response, in H2+ it’s just a bunch of data frames. This is how low latency HLS video works.
Correct on the reliable UDP.
I've seen a few implementations of a reliable UDP messaging stream (using multicast or broadcast). It uses a sequencer to ensure all clients receive the correct message and order.
It can be extremely fast and reliable.
If it guarantees delivery and preserves order, what’s the difference vs. TCP?
Much much faster, TCP is very slow and complicated (relatively). This can be sped up by not using the kernel using something talking directly to hardware.
Delimited messages vs. stream.
I have implemented publish/subscribe middleware in the 90s using UDP and reliable IP multicast. It was not too hard to do and had taken me about two month. Yes I did reinvent the wheel but it had numerous positive sides:
> unlimited-length
Where to keep in-progress frame data? I case of TCP it's an app and buffer for only a single frame or app even could make "stream processing" for in-progress frame.
There’s Homa[1].
[1] https://github.com/PlatformLab/Homa
libfabric implements msg on top of TCP, among others. I love that I can just switch between TCP and RDMA without any code change.
WebSocket?
WebSocket is over TCP, so still ordered. I think by message-based GP is talking not only about message framing (which WebSocket provides) but also reliable out-of-order delivery.
When does a long out of order delay look like a packet loss?
Is it possible for an out of order packet to get delayed by a millisecond? A second? A minute? An hour?
Have you written code to be robust to this?
Other people have probably thought about it more than I have, but the way I see it working is that a set of unacked messages are stored on each end of the connection. When a connection receives a message, it acks it by ID. When acks come out of the order the client sent them in (or after a timeout period), it re-sends the packet that may have dropped.
If every message is getting dropped, the set of unacked messages fills up and the connection stops accepting messages from the application, similar to TCP in that situation.
I mostly work with TCP so haven't had to deal with unreliable channels generally, but I do use a similar approach at the application level for reliable delivery across reconnects.
My point is that a lost message looks the same as an arbitrary delay.
TL;DR - use QUIC (should have just looked at the domain name)
I use UDP for my desktop game like software with clients all over the world to propagate anonymized state. It needs no encryption as the transmitted data has zero value to any third party. It needs no reliability since dropped packets would be handled by predictive filter with practical reliability that is way more than needed.
So why the F.. would I bother with anything else?
You might want some form of encryption or message signing to prevent spoofed packet injection. Just because the data might not have value to a third party, knowing that the data is legitimate has value to you.
Arguments like this should have practical value for business. For my business the value is zilch. It is simply not worth it for the hacker to mess with my packet server. I've been running it for 10 years with 0 complaints. And the worst they can do is to temporarily annoy few people. And yes I implement simple proprietary form packet of signing. It took me whole few minutes to develop.
Many games are popular targets for griefers or DDOS attacks, not because the attacker gains anything of value from the attack (and many of them aren't actually hackers, they just download the tools written by others), but instead they do it solely for the perverse happiness they get from knowing that they ruined someone else's day / hour / minute.
Possibly you just haven't seen this problem yet because your game isn't high-profile enough for them to bother with and it doesn't provide enough amusement for them to get a kick out of it.
It is game like by tech used but is not really a game. Unlikely to be noticed. When IF it ever does I'll worry about it the, Not too difficult as there are libs just for that. For now my simple signing works like a charm.
Because author of the article likes to be edgy and parrot the deep truth he learned yesterday - that TCP is "reliable" and UDP is not.
The reality, of course, is somewhat different and muddy. For example, if you have network outage following later by a software crash or a reboot, then all the TCP buffer worth of data (several kilobytes or upto some megabytes - depends on your tuning) is mercilessly dropped. And your application thinks that just because you used TCP the data must have been reliably delieved. To combat this you have to implement some kind of serializing and acking - but the article scoffs us that we are too dumb to implement anything besides basic stream /s
I'm not arguing that TCP is useless, just that UDP has its place and we - mere mortals - can use is too. Where appropriate.
> Because author of the article likes to be edgy and parrot the deep truth he learned yesterday
I will point out that "author of the article" is one of the core contributors in the IETF Media-over-QUIC working group (which is an effort to standardize how one might build these real-time applications over QUIC) and has been working in the real-time media protocols space for 10+ years.
The author recognizes that the title is clickbait, but the key point of the article is that you probably don't want to use raw UDP in most cases. Not that UDP is inherently bad (otherwise he wouldn't be working on improving QUIC).
The author (disclaimer: former coworker) has likely learned about UDP before yesterday because he's been complaining about TCP for years.
The thing is, when you need that low latency, you don't want most of the features of more robust streams.
When you are using UDP the correct way to handle out of order delivery is to just ignore the older packet. It's old and consequently out of date.
Figuring out how to solve any mess caused by mistransmission necessarily has to be done at the application level because that's where the most up to date data is.