After upgrade to 3.10.3 header with Content-Type was omitted at request

MSlaski · January 10, 2024, 11:10am

Hello,
I have Java 11 + Maven project with Gatling. Today I upgraded gatling from version 3.9.5 to 3.10.3. Necessary changes for DSL were done (like exitBlockOnFail)
Also I remove tens of .exec() blocks inside my scenarios thanks to changes at 3.10 version

After that I found an issue. For one of the request the Content-Type header somehow was omitted. Rest of headers was properly added. Please, check example of request and also output from 3.9.5 and 3.10.3 versions

Request:

public static final ActionBuilder myRequest =
    http("Some request")
      .post(someEndpoint)
      .ignoreProtocolHeaders()
      .header("Content-Type", "multipart/form-data")
      .header("apiKey", apiKey)
      .header("User-Agent", userAgent)
      .check(status().is(200));

Here it is necessary to ignore headers provided by protocolBuilder cuz I have to use different API Key to start a user journey, for rest of requests Im using headers from protocolBuilder set globally.

Output from version 3.10.3

=========================
HTTP request:
POST someUrl
headers:
        apiKey: <<<<>>>>>>
        User-Agent:  <<<<>>>>>>
        accept: */*
        host:  <<<<>>>>>>
        content-length: 0
=========================
HTTP response:
version:
        HTTP/1.1
status:
        415 Unsupported Media Type
headers:
        Date: Wed, 10 Jan 2024 10:50:05 GMT
        Content-Length: 0
        Connection: keep-alive
        access-control-allow-headers: Content-Type,User-Agent,Origin,Accept,Accept-Encoding,Authorization
        access-control-allow-methods: GET,POST,DELETE,OPTIONS
        access-control-allow-origin: *
        server:  <<<<>>>>>>
        x-envoy-upstream-service-time: 36
        x-ipvproxy-from-cluster:  <<<<>>>>>>
        x-ratelimit-limit-year: 100000000
        x-ratelimit-remaining-year: 99987948
        api-correlation-id:  <<<<>>>>>>

<<<<<<<<<<<<<<<<<<<<<<<<<

As you can use, sent request does not contain Content-Type header, but apiKey and User-Agent were properly added

Output from version 3.9.5

=========================
HTTP request:
POST someUrl
headers:
	Content-Type: multipart/form-data
	apiKey: <<<<>>>>>>
	User-Agent: <<<<>>>>>>
	accept: */*
	host: <<<<>>>>>>
	content-length: 0
=========================
HTTP response:
status:
	200 OK
headers:
	Date: Wed, 10 Jan 2024 10:51:50 GMT
	Content-Type: application/json
	Transfer-Encoding: chunked
	Connection: keep-alive
	access-control-allow-headers: Content-Type,User-Agent,Origin,Accept,Accept-Encoding,Authorization
	access-control-allow-methods: GET,POST,DELETE,OPTIONS
	access-control-allow-origin: *
	server: <<<<>>>>>>
	x-envoy-upstream-service-time: 36
	x-ratelimit-limit-year: 100000000
	x-ratelimit-remaining-year: 99987946
	api-correlation-id: <<<<>>>>>>

body:
SOME BODY
<<<<<<<<<<<<<<<<<<<<<<<<<

As you can see here the Content-Type header was properly added.

I can also show you the value of globally set headers:

http
          .baseUrl(baseUrl)
          .header("apiKey", apiKey)
          .userAgentHeader(userAgent);

As you can see by ignoring protocolHeaders I want to override apiKey value and userAgent

Do you have any idea what’s going on here?

slandelle · January 10, 2024, 12:11pm

Your 3.9.5 and 3.10.3 codes don’t do the same thing.

Your request is a POST with the request body missing, so it shouldn’t have a content-type. This is also what we see in your 3.10.3 logs.

Your 3.9.5 logs show a request body.

I guess you’ve removed the line defining the request body by mistake.

MSlaski · January 10, 2024, 8:14pm

Thanks @slandelle for response. I double check this particular request and it was not changed during version update.
This particular request doesn’t contain any body. I also checked it at project with functional/API tests and requests are the same (here in gatling, and at API test projects)

slandelle · January 10, 2024, 8:40pm

My bad, I was seeing the body of the response in your logs.

Then, my comment is still valid: I don’t think a user-agent is supposed to set a Content-Type request header for a request that doesn’t have a content.

Unless someone can prove me wrong, I think this is an issue with the design of your API and that the new behavior we have with Gatling 3.10 is correct.

slandelle · January 10, 2024, 8:42pm

Pinging @GeMi who reported the content-type + missing body issue in HTTP: Should not be send Content-Type when is used http method without payload when use asXXX() · Issue #4429 · gatling/gatling · GitHub

GeMi · January 10, 2024, 10:02pm

As I understood rfc7231 and rfc9110 there are some things that connected in this case:

" The POST method requests that the target resource process the
representation enclosed in the request according to the resource’s
own specific semantics." - 4.3.3. POST
“A sender that generates a message containing a payload body SHOULD
generate a Content-Type header field in that message unless the
intended media type of the enclosed representation is unknown to the
sender.” - 3.1.1.5. Content-Type
“Representation header fields provide metadata about the
representation. When a message includes a payload body, the
representation header fields describe how to interpret the
representation data enclosed in the payload body.” - 3.1. Representation Metadata
“A “representation” is information that is intended to reflect a past, current, or desired state of a given resource, in a format that can be readily communicated via the protocol. A representation consists of a set of representation metadata and a potentially unbounded stream of representation data” - 3.2. Representations

So:

POST without representation data enclosed is not correct because target don’t have “thing” to process, and respresentation must have representation metadata and representation data
sender should (not must) add Content-Type
representation header (Content-Type) can’t provide metadata about the representation because there is missing representation data

To sum up, Gatling from version 3.10.X (After fix → HTTP: Should not be send Content-Type when is used http method without payload when use asXXX()) behaves correctly in this situation - previously wasn’t.

If I am wrong, please correct me.

slandelle · January 11, 2024, 7:10am

IMO, a server receiving a POST request with no body nor content-type should respond with 400: Bad Request.

What are the semantics of your server for this use case? How does it make sense for your server to have this POST request without a body?

sbrevet · January 11, 2024, 8:14am

IMO, a server receiving a POST request with no body nor content-type should respond with 400: Bad Request .
@slandelle

Why? A common pattern in the wild internet exists that you should use the less values for non-idempotent requests.
The lowest non-idempotent request may (some say “should”) be POST /resources that answers with a new id (the only generated value) and with default values. Then the client send a PUT /resources/{new-id} with the wanted values.

Query parameters are another way of providing values to the request without needs for body and its content-type header.

What are the semantics of your server for this use case? How does it make sense for your server to have this POST request without a body?

Some server (based on popular frameworks) craft the response format (JSON, XML, …) based on Accept (of course) but default to the Content-type value of the request (try to speak the same language). If there is no preferences, the server fall in the indecision state (developer should have configure a preferred server format).

IMO, Gatling should default not to send the Content-Type header when there is no body, but should let the user to define one if they want (as defined here). (Easier to tell than to code for sure)

MSlaski · January 11, 2024, 8:24am

@slandelle The responsibility of this POST request is to create new transaction, and server should return default details about newly created transaction

@sbrevet Accroding to this:

IMO, Gatling should default not to send the Content-Type header when there is no body, but should let the user to define one if they want (as defined here). (Easier to tell than to code for sure)

should I raise an issue at GitHub for such functionality?

GeMi · January 12, 2024, 6:07pm

I have read a lot of topics for this case and I’m confused.
The most substantive statement is POST with empty body from Darrel Miller on 2010-09-02 (ietf-http-wg@w3.org from July to September 2010)
I think that Gatling should give choice to user if he want to set some header or not by
.headers(foo)
.header("foo", "bar")
but for .asXXX should respect if body exist or not.

@slandelle @sbrevet WDYT?

slandelle · January 13, 2024, 8:47pm

2 different things:

we should revert the whole change.

Depending on the encoding, an empty body could be perfectly valid.

For example, a browser would post a empty form as an body with a application/x-www-form-urlencoded content-type header.

<html>
  <body>
    <form method="post">
      <input type="submit" value="Submit">
    </form>
  </body>
</html>

On the contrary, an empty string is neither valid JSON nor XML.

decide what to do with asXXX

As they target JSON and XML where an empty body is not valid, I’d be also in favor of not setting the content-type when there’s no body defined (but not trying to handle an empty one, too complicated).

slandelle · January 15, 2024, 12:05pm

github.com/gatling/gatling

HTTP: revert aggressively removing requst content-type header when request body is empty

opened 10:55AM - 15 Jan 24 UTC

closed 11:17AM - 15 Jan 24 UTC

slandelle

Module - HTTP Type - Bug

As of #4429, we're aggressively removing request content-type header when reques…t body is empty. This is wrong, some content-types have valid empty string value, such as `application/x-www-form-urlencoded`. Let's only not add a request content-type when using `asJson` and `asXml` when there's no body as an empty string is not a valid content for these encodings.

GeMi · January 15, 2024, 7:02pm

I think it’s good decision.

system · February 14, 2024, 7:02pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Possible regression - http requests without explicit headers are ignored Gatling (Open-Source)	5	138	August 15, 2014
HTTP 307 doesn't keep headers Gatling (Open-Source)	6	150	January 21, 2015
How to Troubleshoot IO.Exception: Premature Close? Gatling (Open-Source)	6	169	August 20, 2019
Scenario is reporting server timeout when receiving gzip response from the server Gatling (Open-Source)	4	385	March 11, 2023
Trying to understand why we are getting this error HTTP header is larger than 8192 bytes when the request header does not appear to be Gatling (Open-Source)	6	241	March 25, 2019

After upgrade to 3.10.3 header with Content-Type was omitted at request

Related Topics