# Stress a server with big file uploads over a slow tcp connection

**URL:** <https://community.gatling.io/t/stress-a-server-with-big-file-uploads-over-a-slow-tcp-connection/1963>\
**Category:** Gatling (Open-Source)\
**Created:** [February 3, 2015, 6:23pm UTC](https://community.gatling.io/t/stress-a-server-with-big-file-uploads-over-a-slow-tcp-connection/1963 "2015-02-03T18:23:40Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![leire.retana](https://avatars.discourse-cdn.com/v4/letter/l/eb8c5e/32.png) [@leire.retana](https://community.gatling.io/u/leire.retana)\
**Post date:** [February 3, 2015, 6:23pm UTC](https://community.gatling.io/t/stress-a-server-with-big-file-uploads-over-a-slow-tcp-connection/1963/1 "2015-02-03T18:23:40Z")

</div>

Hi

I am trying to stress a server with big uploads (10MB file each time scenario) over slow connections (tcp delay set to 100 ms and tcp buffer size set to 4k) with gatling on the following setup:

setUp(  
scn.inject(  
constantUsersPerSec(1) during(30),  
constantUsersPerSec(2) during(30),  
constantUsersPerSec(3) during(30),  
constantUsersPerSec(4) during(30),  
constantUsersPerSec(5) during(30),  
nothingFor(30)  
)  
)

Because of the slow tcp connections, each 10MB file takes several tens of seconds to send completely, which is making the test run out of memory every time (java heap space). Is there any way in gatling to make the test only store in memory the amount of data it sends at each time as opposed to the whole file size?

Thanks!

Leire

---

<div class="post-metadata">

**Author:** ![Abhinav\_Gogna](https://avatars.discourse-cdn.com/v4/letter/a/c0e974/32.png) [@Abhinav\_Gogna](https://community.gatling.io/u/Abhinav_Gogna)\
**Post date:** [February 3, 2015, 7:29pm UTC](https://community.gatling.io/t/stress-a-server-with-big-file-uploads-over-a-slow-tcp-connection/1963/2 "2015-02-03T19:29:55Z")

</div>

Have you tried increasing the heap size?

---

<div class="post-metadata">

**Author:** ![slandelle](https://dub1.discourse-cdn.com/flex013/user_avatar/community.gatling.io/slandelle/32/4_2.png) [@slandelle](https://community.gatling.io/u/slandelle)\
**Post date:** [February 3, 2015, 9:00pm UTC](https://community.gatling.io/t/stress-a-server-with-big-file-uploads-over-a-slow-tcp-connection/1963/3 "2015-02-03T21:00:07Z")

</div>

You have to realize that you don’t provide any kind of information that would help on this.  
First, in the [Group’s rules](https://groups.google.com/forum/#!forum/gatling), we ask that you at least provide the Gatling version (amongst other things to check).

Then, you should share your simulation so we know how you perform your uploads.  
And finally, as you have an OOM, providing a head dump would also help.

---

<div class="post-metadata">

**Author:** ![leire.retana](https://avatars.discourse-cdn.com/v4/letter/l/eb8c5e/32.png) [@leire.retana](https://community.gatling.io/u/leire.retana)\
**Post date:** [February 4, 2015, 11:20am UTC](https://community.gatling.io/t/stress-a-server-with-big-file-uploads-over-a-slow-tcp-connection/1963/4 "2015-02-04T11:20:53Z")

</div>

Sorry about the missing information, I thought the use case itself might spring some ideas to mind.

Gatling version is 2.1.3 and this is how I am performing the updloads:

_val fileSize = 10000000_

_var scn = scenario(“Upload 10M Benchmark”)_  
_exec(http(“Upload all”)_  
_.put("${URI}")_  
_.body(InputStreamBody( \_ =\> new InputStream() {_  
_var byteNumber = 0_

_override def read(): Int = {_  
_byteNumber = byteNumber + 1_  
_if (byteNumber \< (fileSize + 1) ) random nextInt(255) else -1_  
_}_  
_}))_  
_.header(“Content-Range”, s"bytes 0-$fileSize/$fileSize")_  
_.check(status.is(204))_

I have tried tried to increase the heap size up to 3G, which is as high as it can go in that box, but it still runs out of memory. And even if it had worked, it would only hide the problem: as soon as the tcp delay or the number of users were increased the test would fall over again.

One other thing to note is that this works with smaller file sizes (and fails earlier with large file sizes). I assume Gatling is building up the entire file upload in memory which (due to the really low transmission speeds we are forcing) are building up over time. The initial reason we went with an InputStream generator was that we were hoping Gatling would be able to steam the content on demand rather than prebuild the body but from what we can see that does not seem to be happening.

We analysed the heap dump and it shows a huge number of byte arrays of 8216 bytes, Attached is an example of the path to root of one of these arrays.

Thanks again!  
Leire

[path-to-root.txt](https://community.gatling.io/uploads/short-url/w190YveQdDRlO8xGvSQYq3mdp18.txt) (536 KB)

---

<div class="post-metadata">

**Author:** ![slandelle](https://dub1.discourse-cdn.com/flex013/user_avatar/community.gatling.io/slandelle/32/4_2.png) [@slandelle](https://community.gatling.io/u/slandelle)\
**Post date:** [February 4, 2015, 12:30pm UTC](https://community.gatling.io/t/stress-a-server-with-big-file-uploads-over-a-slow-tcp-connection/1963/5 "2015-02-04T12:30:21Z")

</div>

I’m not sure we can do anything about this.  
The underlying Netty component (ChunkedInput/ChunkedWriteHandler) doesn’t seem to have any sort of back-pressure mechanism.  
Asking on the Netty ICQ.

---

<div class="post-metadata">

**Author:** ![slandelle](https://dub1.discourse-cdn.com/flex013/user_avatar/community.gatling.io/slandelle/32/4_2.png) [@slandelle](https://community.gatling.io/u/slandelle)\
**Post date:** [February 10, 2015, 9:20am UTC](https://community.gatling.io/t/stress-a-server-with-big-file-uploads-over-a-slow-tcp-connection/1963/6 "2015-02-10T09:20:15Z")

</div>

@Leire: FYI, I opened an issue on Netty: [https://github.com/netty/netty/issues/3413](https://github.com/netty/netty/issues/3413). Don’t expect anything fast though.

---

<div class="post-metadata">

**Author:** ![Stefano\_Sgorlon](https://avatars.discourse-cdn.com/v4/letter/s/73ab20/32.png) [@Stefano\_Sgorlon](https://community.gatling.io/u/Stefano_Sgorlon)\
**Post date:** [October 27, 2016, 2:16pm UTC](https://community.gatling.io/t/stress-a-server-with-big-file-uploads-over-a-slow-tcp-connection/1963/7 "2016-10-27T14:16:41Z")

</div>

This use-case is very similar on what I am trying to do now.  
Was anyone able to implement it?  
For the **_InputStream_** it makes sense to use this implementation:  
[https://commons.apache.org/proper/commons-io/javadocs/api-2.5/org/apache/commons/io/input/NullInputStream.html](https://commons.apache.org/proper/commons-io/javadocs/api-2.5/org/apache/commons/io/input/NullInputStream.html)