UDP vs. TCP
TCP vs. UDP
We have a decision to make here, do we use TCP sockets or UDP sockets?
Lets look at the properties of each:
- Connection based
- Guaranteed reliable and ordered
- Automatically breaks up your data into packets for you
- Makes sure it doesn’t send data too fast for the internet connection to handle (flow control)
- Easy to use, you just read and write data like its a file
- No concept of connection, you have to code this yourself
- No guarantee of reliability or ordering of packets, they may arrive out of order, be duplicated, or not arrive at all!
- You have to manually break your data up into packets and send them
- You have to make sure you don’t send data too fast for your internet connection to handle
- If a packet is lost, you need to devise some way to detect this, and resend that data if necessary
The decision seems pretty clear then, TCP does everything we want and its super easy to use, while UDP is a huge pain in the ass and we have to code everything ourselves from scratch. So obviously we just use TCP right?
Using TCP is the worst possible mistake you can make when developing a networked action game like an FPS! To understand why, you need to see what TCP is actually doing above IP to make everything look so simple!
How TCP really works
TCP and UDP are both built on top of IP, but they are radically different. UDP behaves very much like the IP protocol underneath it, while TCP abstracts everything so it looks like you are reading and writing to a file, hiding all complexities of packets and unreliability from you.
So how does it do this?
Firstly, TCP is a stream protocol, so you just write bytes to a stream, and TCP makes sure that they get across to the other side. Since IP is built on packets, and TCP is built on top of IP, TCP must therefore break your stream of data up into packets. So, some internal TCP code queues up the data you send, then when enough data is pending the queue, it sends a packet to the other machine.
This can be a problem for multiplayer games if you are sending very small packets. What can happen here is that TCP may decide that its not going to send data until you have buffered up enough data to make a reasonably sized packet (say more than 100 bytes or so). This is a problem because you want your client player input to get to the server as quickly as possible, if it is delayed or “clumped up” like TCP can do with small packets, the client’s user experience of the multiplayer game will be very poor. Game network updates will arrive late and infrequently, instead of on-time and frequently like we want.
TCP has an option you can set that fixes this behavior called TCP_NODELAY. This option instructs TCP not to wait around until enough data is queued up, but to send whatever data you write to it immediately. This is typically referred to as disabling Nagle’s algorithm.
Unfortunately, even if you set this option TCP still has serious problems for multiplayer games.
It all stems from how TCP handles lost and out of order packets, to present you with the “illusion” of a reliable, ordered stream of data.
How TCP implements reliability
Fundamentally TCP breaks down a stream of data into packets, sends these packets over unreliable IP, then takes the packets received on the other side and reconstructs the stream.
But what happens when a packet is lost? What happens when packets arrive out of order or are duplicated?
Without going too much into the details of how TCP works because its super-complicated (please refer to TCP/IP Illustrated) in essence TCP sends out a packet, waits a while until it detects that packet was lost because it didn’t receive an ack (acknowledgement) for it, then resends the lost packet to the other machine. Duplicate packets are discarded on the receiver side, and out of order packets are resequenced so everything is reliable and in order.
The problem is that if we were to attempt to synchronize this using TCP, whenever a packet is dropped it has to stop and wait for that data to be resent. Yes, even if more recent data arrives, that new data gets put in a queue, and you cannot access it until that lost packet has been retransmitted. How long does it take to resend the packet? Well, it is going to take at least round trip latency for TCP to work out that data needs to be resent, but commonly it takes 2*RTT, and another one way trip from the sender to the receiver for the resent packet to get there. So if you have a 125ms ping, you will be waiting roughly 1/5th of a second for the packet data to be resent at best, and in worst case conditions you could be waiting up to half a second or more (consider what happens if the attempt to resend the packet fails to get through?). What happens if TCP decides the packet loss indicates network congestion and it backs off? Yes it does this. Fun times!
Why you should never use TCP to network time critical data
The problem with using TCP for realtime games like FPS is that unlike web browsers, or email or most other applications, these multiplayer games have a real time requirement on packet delivery. For many parts of your game, for example player input and character positions, it really doesn’t matter what happened a second ago, you only care about the most recent data. TCP was simply not designed with this in mind.
Consider a very simple example of a multiplayer game, some sort of action game like a shooter. You want to network this in a very simple way. Every frame you send the input from the client to the server (eg. keypresses, mouse input controller input), and each frame the server processes the input from each player, updates the simulation, then sends the current position of game objects back to the client for rendering.
So in our simple multiplayer game, whenever a packet is lost, everything has to stop and wait for that packet to be resent. On the client game objects stop receiving updates so they appear to be standing still, and on the server input stops getting through from the client, so the players cannot move or shoot. When the resent packet finally arrives, you receive this stale, out of date information that you don’t even care about! Plus, there are packets backed up in queue waiting for the resend which arrive at same time, so you have to process all of these packets in one frame. Everything is clumped up!
Unfortunately, there is nothing you can do to fix this behavior with TCP, nor would you want to, it is just the fundamental nature of it! This is just what it takes to make the unreliable, packet-based internet look like a reliable-ordered stream.
Thing is we don’t want a reliable ordered stream.
We want our data to get as quickly as possible from client to server without having to wait for lost data to be resent.
This is why you should never use TCP for networking time-critical data!
Wait? Why can’t I use both UDP and TCP?
For realtime game data like player input and state, only the most recent data is relevant, but for other types of data, say perhaps a sequence of commands sent from one machine to another, reliability and ordering can be very important.
The temptation then is to use UDP for player input and state, and TCP for the reliable ordered data. If you’re sharp you’ve probably even worked out that you may have multiple “streams” of reliable ordered commands, maybe one about level loading, and another about AI. Perhaps you think to yourself, “Well, I’d really not want AI commands to stall out if a packet is lost containing a level loading command – they are completely unrelated!”. You are right, so you may be tempted to create one TCP socket for each stream of commands.
On the surface, this seems like a great idea. The problem is that since TCP and UDP are both built on top of IP, the underlying packets sent by each protocol will affect each other. Exactly how they affect each other is quite complicated and relates to how TCP performs reliability and flow control, but fundamentally you should remember that TCP tends to induce packet loss in UDP packets. For more information, read this paper on the subject.
Also, it’s pretty complicated to mix UDP and TCP. If you mix UDP and TCP you lose a certain amount of control. Maybe you can implement reliability in a more efficient way that TCP does, better suited to your needs? Even if you need reliable-ordered data, it’s possible, provided that data is small relative to the available bandwidth to get that data across faster and more reliably that it would if you sent it over TCP. Plus, if you have to do NAT to enable home internet connections to talk to each other, having to do this NAT once for UDP and once for TCP (not even sure if this is possible…) is kind of painful.
My recommendation then is not only that you use UDP, but that you only use UDP for your game protocol. Don’t mix TCP and UDP, instead learn how to implement the specific pieces of TCP that you wish to use inside your own custom UDP based protocol.