Design of an API for FTP clients

The include directory contains the file roof.h in which are defined the public services and data structures. This file must be included by any user of the library.
The lib directory contains the implementation of the libroof.so library which will be dynamically linked with the user program. The files are roof.c for the API and roof_p.h for the internal definitions.
The client directory contains an example of FTP client based on the API: roof.
The fs directory contains the implementation of a file system based on FUSE and ROOF as an alternative to file systems like NFS.
The man directory contains the online manuals of the library and the client. The manual is spreaded in the section 1 (roof command), 3 (API) and 7 (general description).
The build process uses a script called roof_install.sh based on cmake which must be installed on the system. Here we show how to build and install directly with cmake command in the default directory /usr/local (the installation requires the super user rights):

$ cd roofxxx $ cmake . -- Check for working C compiler: /usr/bin/gcc [...] -- Build files have been written to: [...] $ make [...] Linking C executable roof [...] $ sudo make install [...] Linking C executable CMakeFiles/CMakeRelink.dir/roof Install the project... [...]

To check that the installation succeeded, read the online manual of roof (this may need to update the environment variable MANPATH with /usr/local/man):

$ man 3 roof ROOF(3) Linux Programmer’s Manual ROOF(3) NAME roof - API for Remote Operations On Files [...]

It is possible to test the roof executable (this may need to update the LD_LIBRARY_PATH environment variable with /usr/local/lib or to call "ldconfig -n /usr/local/lib" if the dynamic linking with libroof.so fails):

$ roof roof 1.5 [...] Type 'help' or '?' for the list of available commands. ftp> quit $

In the following chapters of this article, we describe the main parts of the library to understand in delail the step from the RFC959 recommendation to the implementation. The code snippets are summaries to make this article as short as possible. The reader is advised to download the sources from sourceforge to get the complete implementation.

2.2. Initialization

To be easy to use and robust, an API is supposed to be reentrant to run multiple instances concurrently. This precaution is very important if the software which uses it, is multithreaded (multiple thread running in parallel may use the library concurrently). A mutex is created and initialized in the entry point of the library. To be identified by the dynamic linker at loading time, the entry point is declared as follow (cf. this page for more details about dynamic libraries entry and exit points):

void __attribute__ ((constructor)) roof_initialize(void); void roof_initialize(void) { int rc; // Creation of the mutex (unlocked) rc = pthread_mutex_init(&roof_mtx, NULL); [...] } // roof_initialize

Two macros are defined to lock and unlock the mutex:

#define ROOF_LOCK() (pthread_mutex_lock(&roof_mtx)) #define ROOF_UNLOCK() (pthread_mutex_unlock(&roof_mtx))

2.3. The context

The context is a data structure which identifies an instance. There is one context of type roof_ctx_t per user:

typedef struct {   void *ctx;         // User context } roof_ctx_t;

The field ctx points on the user's private data. The library does not do anything with this field. This only makes possible to associate arbitrary data to the user context. In the internals of the library, the user context is stored in the field ctx of the structure roof_context_t defined in roof_p.h as follow:

typedef struct {   unsigned int  debug_level;    // Niveau de debug   char         *iobuf;          // Buffer d'E/S   unsigned int  l_iobuf;        // Taille du buffer d'E/S   void         *ctx;            // Données utilisateur   int           busy;           // 1, si contexte occupé   int           internal_iobuf; // 1, si buffer d'E/S alloué en interne   int           ctrl;           // Socket de la cnx de contrôle   unsigned int  timeout_ms;     // Timeout avec le serveur en ms   char          type;           // Type pour la commande TYPE   char          code;           // Code pour la commande TYPE } roof_context_t;

If this structure was in the file roof.h, the users of the library would be tempted to use the fields of the structure in their code. This would make their programs incompatible with future versions of the library (if the fields are renamed or disappear) and it may cause the library to fail (if the fields are modified by the user without calling the services of the API).
In this article, roof_ctx_t will be called "external context" (the one seen by the user) and roof_context_t will be called "internal context" (the one seen by the library). To get the latter from the first, the library uses the macro ROOF_CTX() which retrieves the address of the internal context from the address of the external context and the offset of the field ctx in the structure roof_context_t:

#define ROOF_CTX(p) ((roof_context_t *) ((char *)p - offsetof(roof_context_t, ctx)))

Conversely, the external context is retrieved from the internal context through the macro ROOF_EXT_CTX() which merely returns the address of the field ctx in roof_context_t:

#define ROOF_EXT_CTX(p) ((roof_ctx_t *)&(p->ctx))

The library defines a maximum number of contexts with the constant ROOF_NB_MAX_CTX which is the dimension of the table of contexts roof_context[] in roof.c.
The first thing that the user must do, is to allocate a context by calling roof_new():

roof_ctx_t *roof_new(                      unsigned int  timeout_ms, // Timeout (ms) to interact with the server                                                // 0 = Infinite wait                      char         *iobuf,      // I/O buffer (default if NULL)                      unsigned int  l_iobuf,    // Lenght of the I/O buffer (default if 0)                      void         *ctx         // User context                     ) { unsigned int i;   ROOF_LOCK();   // Look for a free context   for (i = 0; i < ROOF_NB_MAX_CTX; i ++)   {     if (0 == roof_context[i].busy)     {       roof_context[i].busy = 1;       break;     }   } // End for   ROOF_UNLOCK();   if (i >= ROOF_NB_MAX_CTX)   {     errno = ENOSPC;     return NULL;   }   roof_context[i].debug_level = 0;   // If the buffer has been allocated by the user   if (iobuf && l_iobuf)   {     [...]     roof_context[i].iobuf = iobuf;     roof_context[i].l_iobuf = l_iobuf;     roof_context[i].internal_iobuf = 0;   }   else // Buffer allocated internally   {     roof_context[i].iobuf = (char *)malloc(ROOF_IO_BUF_SIZE);     [...]     roof_context[i].l_iobuf = ROOF_IO_BUF_SIZE;     roof_context[i].internal_iobuf = 1;   }   roof_context[i].ctrl = -1;   roof_context[i].timeout_ms = timeout_ms;   roof_context[i].ctx = ctx;   return (roof_ctx_t *)&(roof_context[i].ctx); } // roof_new

A free context (busy field = 0) is looked for in the table of contexts. The search is done in a critical section protected by the mutex (ROOF_LOCK()/ROOF_UNLOCK()) to be reentrant. The found context is initialized with the parameters passed by the user. The first parameter timeout_ms is the maximum time in milliseconds to wait for a answer from the server. It is advised to set it to some seconds to avoid to have a program hanging when the server does not answer. The following parameters (iobuf and l_iobuf) are respectively the address and the length of the I/O buffer to interact with the server. If the caller passes NULL for the address or 0 for the length, the buffer is allocated internally with ROOF_IO_BUF_SIZE bytes as default length (this is defined in roof_p.h). The service returns NULL on error or the address of the external context if OK (this is the address of the field ctx in the structure roof_context_t in the table roof_context[]). From the user point of view, this context is an identifier that he will pass to all the subsequent calls to the library in order to identify its instance.
When the user does no longer need the context, he calls roof_delete() to free the resources:

void roof_delete(roof_ctx_t *pContext) // external context { roof_context_t *pCtx = ROOF_CTX(pContext);   // Deallocation of the I/O buffer if allocated internally   if (pCtx->internal_iobuf)   {     free(pCtx->iobuf);   }   // Close the control socket if opened   if (pCtx->ctrl >= 0)   {     shutdown(pCtx->ctrl, SHUT_RDWR);     close(pCtx->ctrl);   }   ROOF_LOCK();   // Free the context   pCtx->busy = 0;   ROOF_UNLOCK(); } // roof_delete

This function is the counterpart of roof_new(). The I/O buffer is freed if it has been allocated internally (field internal_iobuf != 0), the control socket is closed properly with a call to shutdown() then close() if it is opened and finally, the field busy is reset to mark the context as being free.

2.4. Read/Write on the network

The communication on the network uses the TCP/IP protocol through the socket library of Linux. This makes possible to receive and send data over the network via a file descriptor using the read() and write() system calls as if we were writing or reading a file (this is the application of the famous concept of Unix: "everything is file").
In the library, two functions roof_read() and roof_write() encapsulate the read() and write() system calls to mainly use the context concept where are located useful information like the debug level and to handle the EINTR error code. As signals are asynchronous, they can be received at any moment interrupting the running code to trigger a handler that the user may have defined. Under Linux, most of the system calls return in error (e.g. the value -1) and set the global variable errno to EINTR if they are interrupted by a signal. So, roof_read() and roof_write() check this error code to relaunch the system calls.
Here is the roof_write() function:
 

static int roof_write(                       roof_context_t *pCtx, // Internal context                       int             fd,   // Output file descriptor                       const void     *buf,  // Writing buffer                       size_t          len   // Number of bytes to write ) { int          rc; unsigned int l;   l = len;   do   {     rc = write(fd, ((const char *)buf) + (len - l), l);     if (rc < 0)     {       if (EINTR == errno)       {         // Reiterate the read()         rc = 0;       }     }     if (rc > 0)     {       assert(l >= (unsigned)rc);       l -= rc;     }   } while (l && (rc >= 0));   if (-1 == rc)   {   int saved_errno = errno;     ROOF_ERR(pCtx, "Error '%s' (%d) on write()\n", strerror(errno), errno);     errno = saved_errno;   }   else   {     rc = len;   }   return rc; } // roof_write

Here is the roof_read() function:

static int roof_read(                      roof_context_t *pCtx, // Internal context                      int             fd,   // Input descriptor                      char           *buf,  // Read buffer                      unsigned int    len   // Maximum number of bytes to read                     ) { int rc; int saved_errno;   do   {     rc = read(fd, buf, len);     if (-1 == rc)     {       if (EINTR == errno)       {         continue;       }       saved_errno = errno;       ROOF_ERR(pCtx, "Error '%s' (%d) on read(%d)\n", strerror(errno), errno, fd);       errno = saved_errno;       return -1;     }   } while (rc < 0);   return rc; } // roof_read

In the preceding examples, the errno variable is saved before the calls to the macros displaying the error messages and it is restored after because we need to preserve the error value at the return of roof_read() and roof_write(). The saving is mandatory because the error macro calls functions from the C library like printf() which may alter the value of errno. This is advised by the online manual of errno (man 3 errno).
Those functions merely could use the address of the I/O buffers stored in the internal context instead of receiving the address as parameter. But in some cases, those functions are used with a buffer which is not from the context or with an address at a given offset in the I/O buffer.

2.5. Command sending

One of the main functions of the client is to send commands to the server. As seen in the § 1.4, the commands are composed with a keyword eventually followed by a parameter and are terminated by the end of line characters CR and LF. The commands are sent by the client on the control channel. It is the work of the roof_send_cmd() function:

static int roof_send_cmd(                          roof_context_t *pCtx,   // Internal context                          const char     *format, // Command to send                          ...                         ) { int     rc = 0; va_list args_list; int     sz;   va_start(args_list, format);   sz = vsnprintf(pCtx->iobuf, pCtx->l_iobuf, format, args_list);   va_end(args_list);   [...]   rc = roof_write(pCtx, pCtx->ctrl, pCtx->iobuf, sz);   [...]   return 0; } // roof_send_cmd

The function is passed a command described with a format and a variable number of arguments like printf(). vsnprintf() is used to decode the parameters. This interface makes the function generic enough to be able to send any FTP command (with or without parameters). For example, to send a parameter less command like PASV:

rc = roof_send_cmd(pCtx, "PASV\r\n");

And to send a command with parameters like TYPE:

rc = roof_send_cmd(pCtx, "TYPE %c %c\r\n", type, code);

2.6. Reception of the responses

The other main function of the client is to receive the responses from th server. As specified in § 1.5, the answers are formatted and may contain one or multiple lines. This is the work of the roof_get_reply() function:

int roof_get_reply(                    roof_ctx_t  *pContext, // External context                    const char **reply     // Response                   ) { roof_context_t *pCtx = ROOF_CTX(pContext); fd_set          fdset; int             rc; struct timeval  to; unsigned int    i; int             first = 1; unsigned int    offset = 0; unsigned int    lreply; char            code[4];   *reply = NULL;   lreply = pCtx->l_iobuf; one_more_time:   FD_ZERO(&fdset);   FD_SET(pCtx->ctrl, &fdset);   if (pCtx->timeout_ms)   {     to.tv_sec = pCtx->timeout_ms / 1000;     to.tv_usec = (pCtx->timeout_ms % 1000) * 1000;     rc = select(pCtx->ctrl + 1, &fdset, NULL, NULL, &to);   }   else   {     rc = select(pCtx->ctrl + 1, &fdset, NULL, NULL, NULL);   }   switch(rc)   {     case -1: // Error or signal     {       if (EINTR == errno)       {         goto one_more_time;       }       ROOF_ERR(pCtx, "Error '%s' (%d) on select()\n", strerror(errno), errno);       return -1;     }     break;     case 0: // Timeout     {       ROOF_ERR(pCtx, "Timeout on read\n");       errno = ETIMEDOUT;       return -1;     }     break;     case 1 : // Data from the connection     {     char *p;       rc = roof_read_line(pCtx, pCtx->ctrl, pCtx->iobuf + offset, lreply);       [...]       // If the connection is closed       if (0 == rc)       {         // Overwrite the buffer with a dummy code         strcpy(pCtx->iobuf, "600");         lreply = pCtx->l_iobuf - 3;         // Add additional information if enough room         strncat(pCtx->iobuf, " End of connection", lreply);         pCtx->iobuf[pCtx->l_iobuf - 1] = '\0';         *reply = pCtx->iobuf;         return 0;       }       [...]       // Look for the end of line       p = pCtx->iobuf + offset;       i = 0;       while (p < (pCtx->iobuf + offset + rc))       {         if (('\r' == *p) && ('\n' == *(p + 1)))         {           // This is the first line           if (first)           {             // We must have 3 digits at the beginning of the line             [...]             for (i = 0; i < 3; i ++)             {               [...]               code[i] = pCtx->iobuf[i];             } // End for             // Is it a multipline answer ?             if ('-' == pCtx->iobuf[i])             {               first = 0;               // Remaining space in the input buffer               lreply -= rc;               // New beginning of buffer when the response               // is multiline               offset += rc;               [...]               // Read the following line               goto one_more_time;             }             // This is a single line response             // Terminate the line by overwriting the last <CR> with NUL             *p = '\0';             *reply = pCtx->iobuf;             return 0;           }           else // This is a new line of a multiline response           {             // According to the specification, a line may begin with a             // number. So, to check if it is the last line, we check             // the code value             if ((rc > 3) &&                 (' ' == ((pCtx->iobuf + offset)[3])) &&                 (code[0] == (pCtx->iobuf + offset)[0]) &&                 (code[1] == (pCtx->iobuf + offset)[1]) &&                 (code[2] == (pCtx->iobuf + offset)[2]))             {               // End of multiline answer               // Terminate the line by overwriting the last <CR> with NUL               *p = '\0';               *reply = pCtx->iobuf;               return 0;             }             else // This is not the end of a multiline response             {               // Remaining space in the input buffer               lreply -= rc;               // New beginning of buffer for a multiline response               offset += rc;               // Read the following line               goto one_more_time;             }           }         }         i ++;         p ++;       } // End while       [...]     }     break;     default : // Impossible ???!!!???     {       return -1;     }   } // End switch   return -1; } // roof_get_reply

Although quite big, the function is straightforward. It is based on the select() system call to wait for data from the server on the control socket (ctrl field in the internal context). By the way we can notice the use of the timeout if it has been specified by the user in roof_new() call (cf. § 2.3). This avoids to wait for a response which would not come if the server is out of service. The function is able to read responses of one or more lines by testing the presence of the minus sign behind the 3-digit code. To read the data lines from the server, the internal function roof_read_line() is called to get the input characters up to the apparition of the characters CR and LF. This is not useful to describe it in detail.
roof_get_reply() is part of the API because not only it is used internally but also it is used externally by the user to get the spontaneous messages from the server. That is why it is passed an external context instead of an internal context as parameter.
We will see in the following functions that the responses are handled by checking the first digit of the 3-digit code because it is sufficient to know which action to trigger (cf. § 4.2 of RFC959).

2.7. Connection to the server

The connection to the server consists before all to establish the control channel as specified at the beginning of the § 5.4 of the RFC959: the channel is opened and the server sends a response with a code equal to 220 to specify that it is ready. If the server is not ready to receive commands, it sends a response with a code equal to 120 and the client is supposed to wait until it receives a code equal to 220.
The establishment of the control channel is done by roof_open_ctrl() which is passed the address of the server (name or IP address) and the TCP port number. If the server is listening on the standard port (general situation), the caller can pass the constant ROOF_DEF_PORT instead of the hardcoded value 21. The return code of this service is the socket descriptor of the connection or -1 if an error occured. The ctrl field of the internal context stores the value of this socket. We can note the use of the SO_LINGER socket option to make the TCP protocol wait for the remote side to get all the pending data at disconnection time.

int roof_open_ctrl(                    roof_ctx_t   *pContext, // External context                    const char   *host,     // Server's address (name or IP address)                    unsigned int  port      // Server's port number                   ) { roof_context_t     *pCtx = ROOF_CTX(pContext); int                 rc; int                 sd = -1; struct sockaddr_in  addr; struct hostent     *pHost; char               *code; struct linger       opt_linger; int                 err_sav; [...]   // Get a TCP socket descriptor   sd = socket(PF_INET, SOCK_STREAM, 0);   [...]   // Set SO_LINGER option to make the caller of close() on the socket   // wait until the remote part has got all its data   opt_linger.l_onoff = 1;  // Activate the LINGER   opt_linger.l_linger = 2; // Persistence time in 100ms units   rc = setsockopt(sd, SOL_SOCKET, SO_LINGER, &opt_linger, sizeof(opt_linger));   [...]   // Translate the hostname into address   pHost = gethostbyname(host);   [...]   // Populate the address   memset(&addr, 0, sizeof(addr));   addr.sin_family = AF_INET;   addr.sin_port = htons(port);   addr.sin_addr.s_addr = (in_addr_t)(*(unsigned long *)(pHost->h_addr_list[0]));   // Connection to the remote host   rc = connect(sd, (struct sockaddr *)&addr, sizeof(addr));   [...]   // We populate the context right now bacause roof_get_reply() need the ctrl field   pCtx->ctrl = sd;   // Loop until we receive a 2yz response or timeout   do   {     // Wait for a response from the server     rc = roof_get_reply(pContext, &code);     [...]     if ((code[0] != '1') &&         (code[0] != '2'))     {       ROOF_ERR(pCtx, "Negative reply code '%s'\n", pCtx->iobuf);       errno = EIO;       rc = -1;       goto error;     }   } while (code[0] != '2');   rc = sd;   goto end; error:   err_sav = errno;   if (sd >= 0)   {     close(sd);     sd = -1;   }   pCtx->ctrl = -1;   errno = err_sav; end:   return rc; } // roof_open_ctrl

The second and last step of the connection is the identification procedure. The RFC959 proposes a diagram reproduced in figure 5 (with some adaptations). The diagram shows that depending on the server, the identification may consist only of a USER command or USER followed by PASS or USER followed by PASS and ACCT. In practice, the procedure consists most of the time of USER followed by PASS.




The identification diagram is run by the roof_service() service. We describe this service to show that the commands are sent by the roof_send_cmd() internal routine introduced above, the responses are received by roof_get_reply() introduced above and the diagrams are implemented with a suite of switch/case to test the values of the responses and goto to change the states.

int roof_login(                roof_ctx_t *pContext, // External context                const char *login,    // Login name                const char *passwd,   // Password                const char *account   // Account information               ) { roof_context_t *pCtx = ROOF_CTX(pContext); int             rc; const char     *code;   assert(NULL != pCtx);   assert(pCtx->busy);   if (!login || !(login[0]))   {     ROOF_ERR(pCtx, "NULL login parameter\n");     errno = EINVAL;     return -1;   }   rc = roof_send_cmd(pCtx, "USER %s\r\n", login);   [...]   rc = roof_get_reply(ROOF_EXT_CTX(pCtx), &code);   [...]   switch(code[0])   {     case '1' : // Preliminary positive response     case '4' : // Transient negative completion     case '5' : // Permanent negative completion     {       ROOF_ERR(pCtx, "Error '%s'\n", pCtx->iobuf);       errno = EIO;       return -1;     }     break;     case '3' : // Intermediate positive reponse     {       goto send_passwd;     }     break;     case '2' : // Positive completion     {       goto end;     }     break;     default : // Normally impossible     {       ROOF_ERR(pCtx, "Unexpected reply code '%s'\n", pCtx->iobuf);       errno = EIO;       return -1;     }     break;   } // End switch send_passwd:   if (!passwd || !(passwd[0]))   {     ROOF_ERR(pCtx, "Password parameter is required by server\n");     errno = EINVAL;     return -1;   }   rc = roof_send_cmd(pCtx, "PASS %s\r\n", passwd);   [...]   rc = roof_get_reply(ROOF_EXT_CTX(pCtx), &code);   [...]   switch(code[0])   {     case '1' : // Positive Preliminary reply     case '4' : // Transient negative completion     case '5' : // Permanent negative completion     {       ROOF_ERR(pCtx, "Error '%s'\n", pCtx->iobuf);       errno = EIO;       return -1;     }     break;     case '3' : // Intermediate positive response     {       goto send_account;     }     break;     case '2' : // Positive termination     {       goto end;     }     break;     default : // Normally impossible     {       ROOF_ERR(pCtx, "Unexpected reply code '%s'\n", pCtx->iobuf);       errno = EIO;       return -1;     }     break;   } // End switch send_account:   if (!account || !(account[0]))   {     ROOF_ERR(pCtx, "Account parameter is required by server\n");     errno = EINVAL;     return -1;   }   rc = roof_send_cmd(pCtx, "ACCT %s\r\n", account);   [...]   rc = roof_get_reply(ROOF_EXT_CTX(pCtx), &code);   [...]   switch(code[0])   {     case '1' : // Positive preliminary reply     case '3' : // Intermediate positive response     case '4' : // Transient negative completion     case '5' : // Permanent negative completion     {       ROOF_ERR(pCtx, "Error '%s'\n", pCtx->iobuf);       errno = EIO;       return -1;     }     break;     case '2' : // Positive completion     {       goto end;     }     break;     default : // Normally impossible     {       ROOF_ERR(pCtx, "Unexpected reply code '%s'\n", pCtx->iobuf);       errno = EIO;       return -1;     }     break;   } // End switch end:   return 0; } // roof_login

2.8. Diagram number 1

The diagram in figure 6 is the first one presented in the § 6 of the RFC959 recommendation. It concerns the commands ABOR, ALLO, DELE, CWD, CDUP, SMNT, HELP, MODE, NOOP, PASV, QUIT, SITE, PORT, SYST, STAT, RMD, MKD, PWD, STRU and TYPE.

2.9. Diagram number 2

The diagram in figure 7 is the second one presented in the § 6 of the RFC959 recommendation. It concerns the data transfer commands APPE, LIST, NLST, REIN, RETR, STOR and STOU.