The Anatomy of an HTTP Request: Headers, Methods, and What cURL Actually Does

Most people learn HTTP backward: fetch first, then curl, then — maybe — the actual bytes. That order works until something breaks in a way none of the abstractions explain. A request that succeeds in curl but fails in the browser with a CORS error. A POST that silently gets parsed as form data instead of JSON. A client that hangs forever because the server never sent a length the client could trust. All three have the same root cause: nobody looked at what is actually sitting on the wire. This guide does that.

The wire format

An HTTP/1.1 request is plain text (unless you're on HTTP/2 or HTTP/3, which frame things differently but preserve the same semantics). It has four parts, in this exact order: a request-line, one or more header lines, a blank line, and an optional body. Every line — including the blank one — ends in a carriage return followed by a line feed, written as CRLF or \r\n. Not just \n. This is a real distinction: a parser that only splits on \n will usually work by accident, but a strict HTTP/1.1 parser is entitled to reject a message that uses bare LF.

Here is a real request, exactly as it appears on the socket:

POST /api/orders HTTP/1.1
Host: api.example.com
Content-Type: application/json
Content-Length: 25
Connection: keep-alive

{"item":"widget","qty":3}

Line by line: POST /api/orders HTTP/1.1 is the request-line — method, space, request target (path plus query string), space, HTTP version. Every line after it up to the blank line is a header: Name: value, one per line. The blank line — just \r\n with nothing before it — is the header/body separator, and it is mandatory even when there is no body. Everything after that blank line, for exactly Content-Length bytes, is the body. If you count the JSON payload above, it's 25 bytes, which is why Content-Length: 25 is correct — get that number wrong (by hand-building a request string, say) and the receiving side will either truncate your body or hang waiting for bytes that never come.

With explicit line endings shown, the same request looks like this — this is closer to what a packet capture actually shows you:

POST /api/orders HTTP/1.1\r\n
Host: api.example.com\r\n
Content-Type: application/json\r\n
Content-Length: 25\r\n
\r\n
{"item":"widget","qty":3}

What curl's flags actually do to that text

curl is a thin, honest wrapper over exactly this format — every flag maps to one specific piece of the request above, with no magic in between.

• -X POST writes the method token in the request-line. Without it, curl infers GET unless you also pass -d, in which case it infers POST — so -X POST is often redundant when -d is already present, though it's harmless to be explicit.
• -H "Header: value" inserts that exact string as a header line, verbatim, wherever curl places it in the header block. There's no validation beyond basic syntax — curl will happily send a header your server has never heard of.
• -d '{"a":1}' sets the body to that literal string and switches the method to POST if you haven't set one. This is the flag that causes the most conversion bugs. If you don't pass an explicit -H "Content-Type: ...", curl adds Content-Type: application/x-www-form-urlencoded for you — not application/json, even though the body you typed looks exactly like JSON. curl doesn't parse the body to guess its shape; it just applies the historical default for -d, which predates JSON as a common wire format by a couple of decades. A server that does strict content negotiation will either 415 that request or, worse, silently try to parse {"a":1} as a URL-encoded form and get nothing useful out of it. This is the single most common reason a curl example "just works" in a terminal but the naive fetch translation (assuming JSON was implied) fails until you add the header explicitly.

Content-Length vs. Transfer-Encoding: chunked

A body has to end somewhere, and TCP doesn't tell you where — it's a byte stream with no message boundaries of its own. HTTP solves this with exactly two mechanisms, and a request or response with a body must use one of them (or the body must be empty).

Content-Length: N is the simple case: read exactly N bytes after the blank line and stop. It requires the sender to know the full body size up front, which is easy for a small JSON payload and awkward for a response being generated on the fly (say, a database export streaming out row by row before the total size is known).

Transfer-Encoding: chunked solves that case. Instead of one length up front, the body is split into chunks, each prefixed by its own size in hex followed by CRLF, ending in a zero-length chunk:

7\r\n
Mozilla\r\n
9\r\n
Developer\r\n
0\r\n
\r\n

That decodes to MozillaDeveloper — two chunks of 7 and 9 bytes, then a terminating zero-length chunk. The receiver stops reading when it sees that 0\r\n\r\n, never needing a total length in advance.

Without either header, there is no reliable way to know where the body ends short of the connection closing — and relying on connection-close as a boundary is exactly what HTTP/1.1 keep-alive was designed to avoid, since it forces a new TCP connection per request. This is also a security-relevant detail: request smuggling exploits arise specifically when a front-end proxy and a back-end server disagree about which of Content-Length or Transfer-Encoding governs a given message, and each ends the body at a different byte.

Safe and idempotent are not the same thing

RFC 9110 (formerly 7231) defines two separate properties per method, and conflating them is a common source of bad retry logic.

"Safe" means the client isn't asking for anything beyond retrieval — a well-behaved GET shouldn't create a row or charge a card, even though nothing stops a badly-written server from doing so. "Idempotent" is a stronger, independent claim: N identical requests leave the resource in the same state as one request. DELETE /orders/42 is idempotent because the order is gone whether you call it once or five times; the second through fifth calls just return 404 instead of 204, but nothing extra gets deleted.

This is precisely why HTTP clients, browsers, and proxies feel free to auto-retry a GET or PUT that timed out, but never auto-retry a POST. If POST /orders times out after the server already created the order and the response is just lost on the way back, retrying blindly creates a second order. There's no generic way for the client to tell "no response" apart from "no effect." If you need retry-safe writes, use a client-supplied idempotency key the server can deduplicate against, or model the operation as an idempotent PUT to a specific resource ID instead of a POST to a collection.

Why CORS never shows up in curl but always shows up in the browser

CORS (Cross-Origin Resource Sharing) is not a property of HTTP the protocol — it's a policy the browser enforces on top of it, and curl has no browser inside it, so none of this applies when you run a request from the terminal.

A cross-origin request counts as "simple" — no preflight needed — only if it's a GET, HEAD, or POST, uses only a small allow-listed set of headers, and its Content-Type is one of text/plain, application/x-www-form-urlencoded, or multipart/form-data. The moment you add a custom header (say, an Authorization bearer token or an X-Requested-With), use Content-Type: application/json, or use a method like PUT or DELETE, the request is "non-simple." For a non-simple request, the browser doesn't send your request first — it sends an OPTIONS request to the same URL, with no body, carrying Access-Control-Request-Method (the method you actually intend to use) and Access-Control-Request-Headers (the custom headers you're about to send). The server has to answer that preflight — usually before your route handler code even runs, if you're using CORS middleware — with matching Access-Control-Allow-Origin, Access-Control-Allow-Methods, and Access-Control-Allow-Headers. Only if those line up does the browser fire off your real request at all. If they don't line up, the browser never sends the real request and hands your JS a generic network error — the server response, if there even was one, is invisible to your code.

curl sends none of this dance. There's no origin to compare against, no policy engine deciding whether a script "from" one page is allowed to read a response "from" another — that entire concept only exists because a browser is running script from an untrusted page and mediating what it can read back. curl just opens a socket and sends the one request you asked for. So a curl command from an API doc that works perfectly on the command line, converted line-for-line into a browser fetch(), can fail with a CORS error that has nothing to do with your JavaScript being wrong — it's the server's Access-Control-Allow-* configuration that curl never had to satisfy in the first place.

Try it

Paste a real API doc's curl example into the DevBox cURL Converter and check the generated headers against what's described here — specifically whether it had to add a Content-Type you didn't see in the original command, and whether the method and headers it produces would trigger a preflight if that generated code ran in a browser tab against a cross-origin API.