Table of Content
The internet, a vast and intricate network connecting billions of devices, relies on a complex set of protocols to ensure seamless data transmission. Underneath this seamless communication lies a sophisticated framework of protocols, among which Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) stand out as key mechanisms for packaging and delivering data.
While both fulfill the critical function of transporting information, they differ in how they prioritize speed, reliability, and overall efficiency. These contrasting approaches make TCP and UDP uniquely suited to a broad range of applications—from real-time gaming and live video streaming to chat services, financial data feeds, and the expanding Internet of Things (IoT).
Building on this foundation, the purpose of this paper is to clarify how TCP and UDP operate in practical software contexts—bridging the gap between conceptual understanding and hands-on implementation.
By providing concise explanations, illustrative code examples, and real-world use cases, it aims to cover the knowledge necessary to design and optimize networked solutions. The intended audience includes software developers, network engineers, and tech enthusiasts who seek a straightforward introduction to writing and deploying custom communication protocols, particularly those who want to go beyond high-level abstractions and gain a deeper grasp of data transport fundamentals.
TCP
This protocol prioritizes accuracy and order by ensuring reliable, ordered, and error-checked delivery of data between applications. Operating at the transport layer of the OSI model, TCP is often used alongside the Internet Protocol (IP) as part of the TCP/IP suite, the backbone of the internet. Its emphasis on reliability makes TCP ideal for applications where data integrity is essential, such as web browsing, file transfers, and email. For instance, when downloading a file, TCP verifies that every byte of data arrives accurately, preventing corruption and ensuring that you receive the complete file. However, this reliability comes at the cost of speed, as the overhead of connection establishment and packet tracking adds some latency.
UDP
In contrast, UDP prioritizes speed and efficiency over guaranteed delivery. Think of it as a carefree messenger that sends packets of data quickly, hoping they reach their destination without establishing a connection or tracking individual packets. This makes UDP the protocol of choice for applications where speed is critical and occasional data loss is acceptable, such as video streaming, online gaming, and voice chat. For example, while streaming a video, a few lost packets might cause a brief flicker or a skipped frame, but the overall viewing experience remains largely unaffected. The focus with UDP is on delivering data as quickly as possible, even if it means sacrificing some accuracy.
Key DifferencesÂ
The main distinction between TCP and UDP lies in their approach to data delivery: TCP offers a connection-oriented, reliable service, whereas UDP provides a connectionless, best-effort service. TCP establishes a dedicated connection between the client and server, ensuring that data is delivered in the correct order and checked for errors. UDP, on the other hand, simply sends data packets without any guarantee of delivery or order.
Networking in Software
Choosing between TCP and UDP depends entirely on the specific application requirements.But how do we access the network stack in our software?
The operating system manages networking by acting as a manager between the application and the network hardware. It provides a set of Application Programming Interfaces (APIs) that enable programming languages to interact with the network stack. These APIs offer functions to establish an endpoint for sending or receiving data across a network, typically using sockets.
In IoT projects, for example, you might need to interact directly with the address bus of a microchip like the ESP32 to set up a bare-metal network stack. This includes low-level Wi-Fi initialization, Ethernet frame management, ARP resolution, and full handling of the IP and TCP protocols, including connection management.
Sockets
A socket is an endpoint for sending or receiving data over a network. Essentially, it is an abstraction that allows applications to communicate with each other, acting like a “door” through which data flows in and out of a program. Sockets provide a standardized way for programs to access the network stack, enabling them to establish connections, send and receive data, and manage the flow of information.
Practical Example: Building a TCP Server
Now, let’s move on to some practical code. We’ll write a TCP server in Rust to demonstrate how to work with sockets and implement a custom communication protocol.
use std::{
io::{Read, Write},
net::TcpListener,
thread,
};
fn main() {
let listener = TcpListener::bind("127.0.0.1:8080").expect("fail to bind to 127.0.0.1:8080 address");
for socket in listener.incoming() {
let mut socket = socket.expect("fail to adquire socket");
thread::spawn(move || loop {
let mut read_buff = [0u8; 1024];
let bytes = socket.read(&mut read_buff).expect("fail to read");
if bytes == 0 {
// connection ended. EOF
break;
}
let bytes = socket.write(b"hello world!").expect("fail to write");
if bytes == 0 {
// connection ended. EOF
break;
}
});
}
}
Overview of a TCP Server in Rust
First, the server must open a port on the localhost, for example, port 8080. It then listens for incoming connections. When a connection occurs, the operating system returns a socket, signaling that a connection has been established. To keep the socket alive, an infinite loop is employed to continually perform read and write operations. It is important to note that operations such as listening, reading, and writing are blocking; if there is no new connection, the server will block. To prevent this from affecting the entire program, a new thread is launched for each connection. Within each thread, the socket can perform blocking operations independently without interfering with other connections.
Working with Sockets
When working with sockets, you must provide a fixed-size buffer for the read operation. Because internet data arrives over time, the operating system fills this buffer with the first incoming packet and returns the number of bytes read. If a read or write operation returns zero bytes, that indicates the connection has ended (EOF).
This approach can be inconvenient. For example, you might allocate a 1KB buffer, but on an IoT device like a GPS tracker, the data—comprising GPS information and metadata—could be larger or smaller than 1KB. Additionally, you have to determine the data’s format (JSON, CBOR, Protobuf, etc.) and schema (user schema, GPS schema, etc.). These challenges often necessitate creating a custom communication protocol.
Write a Communication Protocol
Writing your own communication protocol might seem intimidating at first, but it can be surprisingly simple and straightforward. Most communication protocols are divided into two sections: the header and the body. The header contains metadata about the package—such as the body size, data type, and version—while the body carries the actual data.
To design a general and agnostic header, you need to account for four variables: the size of the package, the version of the API, the encoding protocol (e.g., JSON, CBOR), and the data type (i.e., schemas). The challenge is to encode this header in a way that is easy to process with sockets. The solution is straightforward: use a fixed-size header. Here’s how it can be done:
As shown in the illustration, only 8 bytes are needed for the header:
- The first 4 bytes represent an int32 value that indicates the size (in bytes) of the body.
- The next 2 bytes indicate the major and minor version of the API.
- The following byte specifies the encoding of the body. This is useful because you might use a human-readable encoding during development and switch to a binary encoding in production.
- The final byte represents the schema of the body data.
That’s it! Easy as cake and the best part is that it is very flexible each part to ensure that it fits your application requirement.
In code, this protocol could be used as follows:
- Create an 8-byte header buffer.
- Read 8 bytes from the socket.
- Parsing the header. Getting the body size, the version, body encoding and the body schema
- Check if the API version is supported.
- Create a new buffer with the size of the body and instruct the socket to read the specified number of bytes.
- Match case the schema type.
- Match case the encoding type and decode the body.
let mut read_buff = [0u8; 8];
let bytes = socket.read(&mut read_buff).expect("fail to read");
if bytes == 0 {
// connection ended. EOF
break;
}
let mut body_size: u32 = 0;
let major_version: u8;
let minor_version: u8;
let encoding_type: u8;
let schema: u8;
body_size = (read_buff[0] as u32)
| (read_buff[1] as u32) << 8
| (read_buff[2] as u32) << 16
| (read_buff[3] as u32) << 24;
major_version = read_buff[4];
minor_version = read_buff[5];
println!("Version {}.{}", major_version, minor_version);
encoding_type = read_buff[6];
println!("encoding type {}", encoding_type);
schema = read_buff[7];
println!("schema {}", schema);
let mut body = Vec::<u8>::new();
body.resize(body_size as usize, 0u8);
socket.read(&mut body).unwrap();
let bytes = socket
.write(encode_msg_to_bytes(
"{"data": "hellow world"}",
EncodingType::Json,
major_version,
minor_version,
))
.expect("fail to write");
if bytes == 0 {
// connection ended. EOF
break;
}
This simple design provides a robust framework for building reliable communication protocols.
Building a TCP client.
So far, we’ve only covered the server side. But what about the client side? On the client side, the process begins by creating a socket and connecting to the server, typically by specifying the server’s IP address and port number. The client then performs a handshake to establish the connection. Once it’s established, the client can enter a read/write cycle to exchange data with the server. The communication protocol remains the same throughout the process.
use std::{
io::{Read, Write},
net::TcpStream,
};
fn main() {
let mut socket = TcpStream::connect("127.0.0.1:8080").expect("");
loop {
let gps = GpsTracker::get_point();
let msg_bytes = encode_msg_to_bytes(gps.point, generate_metadata_from_gps(&gps));
let n = socket.write(msg_bytes).expect("");
if n == 0 {
break;
}
let mut read_buff = [0; 8];
let n = socket.read(&mut read_buff).expect("");
if n == 0 {
break;
}
let mut body_size: u32 = 0;
for i in 0..4 {
body_size |= (read_buff[i] as u32) << (i * 8);
}
let encoding_type = read_buff[6];
let schema = read_buff[7];
let mut response_body = Vec::<u8>::new();
response_body.resize(body_size as usize, 0u8);
socket.read(&mut response_body).unwrap();
}
}
Disadvantages of the Communication Protocol
 Although this protocol is simple and efficient—requiring only 8 bytes to encode the necessary data, which makes it easy to decode—it is not without drawbacks. Some of the notable issues include:
- Header sanitization: An incoming package might contain an invalid 32-bit body size (e.g., a huge number). Allocating that much memory is unrealistic, and while the operating system may ignore the request and allocate less, the protocol could still end up with garbage values in the remaining header fields.
- Not suitable for medium-to-large file transfers: The header’s 32-bit limit on body size is 4,294,967,295 bytes (about 4.3 GB). This limit is impractical in many real-world scenarios and wastes resources on both the client and server sides.
- Lack of human readability: It can be difficult to inspect what’s in the header using network tools, unlike more verbose human-readable protocols such as BitTorrent or HTTP.
- A related suggestion (rather than a disadvantage) is to use a 24-bit value for body size, which caps it at 8,388,607 bytes (about 8 MB). This smaller limit is safer for memory allocation and often more than sufficient for typical use cases.
Conclusion
Understanding the different communication protocols is essential to designing robust network applications. Whether you choose TCP for its reliable, ordered delivery or UDP for its lean, fast transmission depends entirely on your application’s needs. By leveraging the operating system’s network APIs and utilizing sockets, developers can create flexible and scalable connections tailored to specific use cases.
Furthermore, designing your own communication protocol doesn’t have to be daunting. By adopting a simple, fixed-size header format that includes key metadata, such as package size, API version, encoding type, and schema, you can streamline both the exchange and parsing of data across networks. This approach not only simplifies the integration process but also allows for greater control and customization in handling diverse data types and communication scenarios.
I gotta say, this post hit the nail on the head! Leveraging communication protocols in networking is crucial, and understanding TCP/IP vs UDP is essential. I’d take it a step further and emphasize the importance of standardizing metadata fields in headers for seamless integration & customization. Good stuff!
I appreciate the effort in explaining communication protocols in networking environments! However, I’d like to suggest a few tweaks to enhance the clarity and accuracy of this blog post. For instance, including examples of real-world applications that utilize these protocols would make the content more engaging. Additionally, a section on secure communication protocols, such as TLS, could be beneficial for users seeking to integrate security features into their networking environments.