Go back to previous page
Go back to home page


Last update: 25-Dec-2013
Author: R. Koucha

Thread's stack memory consumption









s

Introduction

This paper focuses on the virtual memory consumed by the stack of the threads inside a Linux process even when threads are terminated.

Example of multi-threaded process

The following program creates or terminates a thread each time the operator enters respectively "c" or "t":

#include <stdio.h>
#include <pthread.h>

#define NB_MAX_THD 50

static pthread_t tid[NB_MAX_THD];
static unsigned int idx_create;
static unsigned int idx_terminate;


static void *thd(void *p)
{
  pause();

  return NULL;
} // thd


static void create_thd(void)
{
int rc;

  if (0 == tid[idx_create])
  {
    rc = pthread_create(&(tid[idx_create]), NULL, thd, NULL);
    if (0 == rc)
    {
      printf("Created thread %p\n", (void *)(tid[idx_create]));
      idx_create = (idx_create + 1) % NB_MAX_THD;
    }
  }
  else
  {
    fprintf(stderr, "Too many threads\n");
  }

} // create_thd


static void terminate_thd(void)
{
int rc;

  if (tid[idx_terminate])
  {
    rc = pthread_cancel(tid[idx_terminate]);
    if (0 == rc)
    {
      rc = pthread_join(tid[idx_terminate], NULL);
      if (0 == rc)
      {
        printf("Terminated thread %p\n", (void *)(tid[idx_terminate]));
        tid[idx_terminate] = 0;
        idx_terminate = (idx_terminate + 1) % NB_MAX_THD;
      }
      else
      {
        fprintf(stderr, "Unable to join thread %p\n", (void *)(tid[idx_terminate]));
      }
    }
    else
    {
      fprintf(stderr, "Unable to cancel thread %p\n", (void *)(tid[idx_terminate]));
    }
  }
  else
  {
    fprintf(stderr, "No running threads\n");
  }

} // terminate_thd



int main(void)
{
int c;

  while(1)
  {
    c = fgetc(stdin);
    switch(c)
    {
      case 'c' :
      {
        create_thd();
      }
      break;

      case 't' :
      {
        terminate_thd();
      }
      break;
    }
  }

  return 0;
} // main

Here is an example of compilation and execution of the preceding program:

$ gcc thd.c -lpthread
$ ./a.out
c
Created thread 0x7f5e45b72700
c
Created thread 0x7f5e45371700
c
Created thread 0x7f5e44b70700
t
Terminated thread 0x7f5e45b72700
t
Terminated thread 0x7f5e45371700
c
Created thread 0x7f5e45371700

Between each thread creation and termination, we display the memory map of the process (through "cat /proc/<pid>/maps).

Startup
[...]
7fff7afcf000-7fff7aff0000 rw-p 00000000 00:00 0                          [stack]
[...]

After creation of first thread: 0x7f5e45b72700
[...]
7f5e45372000-7f5e45373000 ---p 00000000 00:00 0
7f5e45373000-7f5e45b73000 rw-p 00000000 00:00 0                          [stack:3649]
[...]

After creation of second thread: 0x7f5e45371700
[...]
7f5e44b71000-7f5e44b72000 ---p 00000000 00:00 0
7f5e44b72000-7f5e45372000 rw-p 00000000 00:00 0                          [stack:3651]
7f5e45372000-7f5e45373000 ---p 00000000 00:00 0
7f5e45373000-7f5e45b73000 rw-p 00000000 00:00 0                          [stack:3649]
[...]

After creation of third thread: 0x7f5e44b70700
[...]
7f5e44370000-7f5e44371000 ---p 00000000 00:00 0
7f5e44371000-7f5e44b71000 rw-p 00000000 00:00 0                          [stack:3653]
7f5e44b71000-7f5e44b72000 ---p 00000000 00:00 0
7f5e44b72000-7f5e45372000 rw-p 00000000 00:00 0                          [stack:3651]
7f5e45372000-7f5e45373000 ---p 00000000 00:00 0
7f5e45373000-7f5e45b73000 rw-p 00000000 00:00 0                          [stack:3649]
[...]

After termination of first thread: 0x7f5e45b72700
[...]
7f5e44370000-7f5e44371000 ---p 00000000 00:00 0
7f5e44371000-7f5e44b71000 rw-p 00000000 00:00 0                          [stack:3653]
7f5e44b71000-7f5e44b72000 ---p 00000000 00:00 0
7f5e44b72000-7f5e45372000 rw-p 00000000 00:00 0                          [stack:3651]
7f5e45372000-7f5e45373000 ---p 00000000 00:00 0
7f5e45373000-7f5e45b73000 rw-p 00000000 00:00 0
[...]

After termination of second thread: 0x7f5e45371700
[...]
7f5e44370000-7f5e44371000 ---p 00000000 00:00 0
7f5e44371000-7f5e44b71000 rw-p 00000000 00:00 0                          [stack:3653]
7f5e44b71000-7f5e44b72000 ---p 00000000 00:00 0
7f5e44b72000-7f5e45372000 rw-p 00000000 00:00 0
7f5e45372000-7f5e45373000 ---p 00000000 00:00 0
7f5e45373000-7f5e45b73000 rw-p 00000000 00:00 0
[...]

After creation of fourth thread: 0x7f5e45371700
[...]
7f5e44370000-7f5e44371000 ---p 00000000 00:00 0
7f5e44371000-7f5e44b71000 rw-p 00000000 00:00 0                          [stack:3653]
7f5e44b71000-7f5e44b72000 ---p 00000000 00:00 0
7f5e44b72000-7f5e45372000 rw-p 00000000 00:00 0                          [stack:3720]
7f5e45372000-7f5e45373000 ---p 00000000 00:00 0
7f5e45373000-7f5e45b73000 rw-p 00000000 00:00 0
[...]


For each created thread, two lines appear (memory zones): one for the stack and TCB space and the other one for the red zone on top of the stack used to detect the stack overflows. For example, the first thread has the following allocated memory spaces:

The memory map also points out the fact that the termination of a thread does not free the virtual memory space belonging to it: the stack as well as the red zone are still present after the end of the thread.

Moreover,  when threads are created after some terminated threads, the new threads get the memory spaces of the previously terminated thread if any. For example, the fourth thread with TCB 0x7f5e45371700 got the memory spaces of the terminated second thread (hence the same TCB value!).

Behaviour of the GLIBC

When we look at the sources of the pthread library in the GLIBC, we can see that a 40 MB stack cache is kept. At process startup, this cache is empty. Then it is filled with the stacks of the terminated threads during the process life. The management of this cache can be found in .../nptl/allocatestack.c.

The size of the cache is configured as follow:

/* Maximum size in kB of cache.  */
static size_t stack_cache_maxsize = 40 * 1024 * 1024; /* 40MiBi by default.  */

About the author

The author is an engineer in computer sciences located in France. He can be contacted here or you can have a look at his WEB home page.




Go back to previous page
Go back to home page