Good design from the ground up – how I/O library design can make or break IoT solutions

Recently, as more and more players are entering the Internet of Things (IoT) field, there has been an influx of not-so-well-thought-out networking or general I/O libraries. Even certain large vendors which claim to be focusing on small embedded systems have been missing the mark significantly by providing libraries that are at best cumbersome and memory intensive, and at worst not really fit for purpose.

Rather than naming and shaming, I wanted to put together a short post on what it is that so many are missing, because I suspect it simply comes down to a lack of awareness of the alternative. It’s one of those things which is obvious once you’ve been exposed to it, but until then might not cross your mind. 

So what am I talking about? It’s the iovec concept, known both as “vectored I/O” and “scatter/gather I/O”.

What this means is simply that instead of assuming that all the data to be sent resides in contiguous memory, it needs to be gathered from several separate buffers. Conversely, for receiving incoming data, the destination buffer is not assumed to be contiguous but rather the read data needs to be scattered to several buffers. Instead of passing a single buffer to the read or write function, a vector of buffers is passed (hence the term “vectored I/O”). In short it allows a single call to read or write data to and from many buffers at once; useful for bunching together fields of different data structures to form one I/O transaction, or slicing protocol headers.

A similar but slightly alternative approach is to design the library with an interface which allows incremental composing of requests, though in practice this in turn often requires extra memory within the library for buffering there instead. The gains are not as easy as with an iovec style interface.

When it comes to I/O library interfaces then, the lowest building block should be one which takes a vector of buffers, rather than a single buffer. There are two big reasons for this. The first is that it is trivial to implement an interface for sending a single buffer on top of an iovec interface, but often impossible to implement an iovec interface on top of a single buffer interface. The second reason cuts to the heart of the matter – an iovec interface can allow for vastly reduced memory usage, depending on the protocol/situation.

Typically, when you’re sending network protocol data you’re having to include a few logically separate parts – some sort of header followed by a body payload, with perhaps also a trailer for checksums or such. And even within each of those, there may be logical separation, such as HTTP request which has the request line followed by a number of header field lines, and an optional body after that. Some of these parts are often constant and do not change.

As a somewhat contrived and seriously cut down example, consider a case where you have a small, memory-limited IoT device collecting some sort of sensor data, and uploading that data using HTTP. A request might look like:

    POST /upload/device/0242c41f0116/sensor HTTP/1.1
    Host: www.iot.example.com
    Connection: close
    User-Agent: cool-iot/1.3.1312
    Accept: application/json
    X-SharedSecretAuth: bmV2ZXIgZ29ubmEgZ2l2ZSB5b3UgdXAsIG5ldmVyIGdvbm5hIGxldCB5b3UgZG93bi4uLgo
    Content-Length: 28
    
    {"t":1623906557,"mV":4.16}

In that request, only a few parts are actually variable (between devices and/or times):

  • the device identifier in the request line
  • the timestamp and sensor reading value in the body JSON
  • possibly the authentication token

Note that I’m deliberately overlooking the possibility of a varying content-length, because it doesn’t add anything of value to the discussion but would make the examples less readable. These examples are not for real world use, only for demonstrating concepts. With that said, back to the topic at hand.

If the http I/O library only presents a send function which takes a single buffer, you have to put the full request into RAM. On the other hand, if it presents an iovec style send function, only the variable parts have to reside in RAM – the rest can be kept in flash (assuming memory-mapped flash, of course).

To illustrate with some code, if we have:

    int http_send_request(http_conn *conn, const uint8_t *buf, uint16_t sz);

then to send that request we have to do something like:

    int do_upload(http_conn *conn, const char *device_id, uint32_t t, double mv)
    {
      // Caution: assumes t and mv to be within well-defined value range
      static const char *fmt =
        "POST /upload/device/%s/sensor HTTP/1.1\r\n"
        "Host: www.iot.example.com\r\n"
        "Connection: close\r\n"
        "User-Agent: cool-iot/1.3.1312\r\n"
        "Accept: application/json\r\n"
        "X-SharedSecretAuth: bmV2ZXIgZ29ubmEgZ2l2ZSB5b3UgdXAsIG5ldmVyIGdvbm5hIGxldCB5b3UgZG93bi4uLgo\r\n"
        "Content-Length: 28\r\n"
        "\r\n"
        "{\"t\":%u,\"mV\":%.2f}"; // in flash
    
      // Too big for our stack, so dynamically allocate space
      char *buffer = 0;
      int n = asprintf(&buffer, fmt, device_id, t, mv);
      if (n < 0)
        return n;
    
      // Here n is ~300
      n = http_send_request(conn, buffer, n);
    
      free(buffer);
      return n;
    }

As we have to have everything available in contiguous memory in order to call http_send_request(), there’s no avoiding allocating a memory block of significant size. Depending on circumstances, it might be possible to statically allocate it, but the memory demand is still there.

Compare that to if there’s an iovec interface available:

    int http_send_request(http_conn *conn, iovec *iov);

in which case we could keep most things in flash:

    int do_upload(http_conn *conn, const char *device_id, uint32_t t, double mv)
    {
      // Caution: assumes t and mv to be within well-defined value range
      static const char req_a[] = "POST /upload/device/"; // in flash
      static const char req_b[] = "/sensor HTTP/1.1\rn"
        "Host: www.iot.example.com\r\n"
        "Connection: close\r\n"
        "User-Agent: cool-iot/1.3.1312\r\n"
        "Accept: application/json\r\n"
        "X-SharedSecretAuth: bmV2ZXIgZ29ubmEgZ2l2ZSB5b3UgdXAsIG5ldmVyIGdvbm5hIGxldCB5b3UgZG93bi4uLgo\r\n"
        "Content-Length: 28\r\n"
        "\r\n"; // in flash
      static const char body_fmt[] = "{\"t\":%u,\"mV\":%.2f}"; // in flash
      char body[29]; // in RAM (on stack)
      snprintf(body, sizeof(body), body_fmt, t, mv);
      // Set up a gather-read iovec, interleaving flash and RAM contents
      iovec iov[] = {
        { req_a, sizeof(req_a) -1 },
        { device_id, strlen(device_id) },
        { req_b, sizeof(req_b) -1 },
        { body, 28 },
        { 0, 0 },
      }; // in RAM (on stack)
      return http_send_request(conn, iov);
    }

Here you can see the difference. Even with this trivial example, we went from needing around ~300 bytes in RAM to ~70 (assuming each iovec entry is 8 bytes), effectively relieving 75% of the memory pressure. For real world scenarios the gain can be even bigger, especially where the body payload contains a lot of boilerplate formatting. More importantly, it can be difference between being able to compose the request or not when you are working on a severely RAM constrained device, where your total memory is measured in kilobytes. Having to spend big chunks of precious RAM on temporary buffers just because the underlying communications library wasn’t sufficiently well designed is at best a downer, and at worst a show stopper.

My exhortation is thus: Don’t be the person who lets your company ship non-iovec style I/O libraries for small embedded systems! Pretty please with a cherry on top :).

Want to know more about how DiUS can help you?

Offices

Melbourne
Level 3, 31 Queen St Melbourne, Victoria, 3000

Phone: 03 9008 5400

Sydney
The Commons

32 York St Sydney,

New South Wales, 2000

DiUS wishes to acknowledge the Traditional Custodians of the lands on which we work and gather at both our Melbourne and Sydney offices. We pay respect to Elders past, present and emerging and celebrate the diversity of Aboriginal peoples and their ongoing cultures and connections to the lands and waters of Australia.

Subscribe to updates from DiUS

Sign up to receive the latest news, insights and event invites from DiUS straight into your inbox.

© 2024 DiUS®. All rights reserved.

Privacy  |  Terms