Always memset C structures and check memory with Valgrind

Few days ago I faced with a task of refactoring my utility receiving messages from my netfilter kernel module. This utility is very simple, it connects to the kernel via netlink socket, set up it’s own logging mechanism, then waiting for new messages from kernel and sending further these messages or just printing them out to console. There is an example of connecting function.

Full code of kernel module and it’s client you can find here.

I changed just several lines after sendmsg() and sendmsg() started to fail every time with ENOBUFS error.

send pid to kernel failed No buffer space available

That seemed very strange. But you should always keep in mind that if there is something strange in your C/C++ app behaviour, if you add some code and it affects code above, you need to check memory. And Valgrind tool will help you here. Switch on track-origins option, it will help to identify where this variable comes from. -s option is for show an error list.

valgrind --track-origins=yes -s /home/keyrnk/Projects/linux_system_dev/kernel_module_netlink_client_server_example/kernel_client/build/kernel_module_client

After Valgrind launch we can see that indeed there is an uninitialized variable msg passing to sendmsg() which leads to some garbage in this variable’s fields and sendmsg() ENOBUFS error :

==80279== 1 errors in context 1 of 2:
==80279== Syscall param sendmsg(msg) points to uninitialised byte(s)
==80279==    at 0x4CD5B87: sendmsg (in /usr/lib64/libc-2.30.so)
==80279==    by 0x4014D3: KernelModuleClient::Run(std::atomic<bool>&) (in /home/keyrnk/Projects/linux_system_dev/kernel_module_netlink_client_server_example/kernel_client/build/kernel_module_client)
==80279==    by 0x4012E7: main (in /home/keyrnk/Projects/linux_system_dev/kernel_module_netlink_client_server_example/kernel_client/build/kernel_module_client)
==80279==  Address 0x1ffefffc60 is on thread 1's stack
==80279==  in frame #1, created by KernelModuleClient::Run(std::atomic<bool>&) (???:)
==80279==  Uninitialised value was created by a stack allocation
==80279==    at 0x40136C: KernelModuleClient::Run(std::atomic<bool>&) (in /home/keyrnk/Projects/linux_system_dev/kernel_module_netlink_client_server_example/kernel_client/build/kernel_module_client)

This is original code:

struct msghdr msg;
msg.msg_name = (void *)&dest_addr;  
msg.msg_namelen = sizeof(dest_addr);  
msg.msg_iov = &iov;  
msg.msg_iovlen = 1;

This is full msghdr definition:

struct msghdr {
    void         *msg_name;       /* optional address */
    socklen_t     msg_namelen;    /* size of address */
    struct iovec *msg_iov;        /* scatter/gather array */
    size_t        msg_iovlen;     /* # elements in msg_iov */
    void         *msg_control;    /* ancillary data, see below */
    size_t        msg_controllen; /* ancillary data buffer len */
    int           msg_flags;      /* flags on received message */
};

We can see that we didn’t initialize all fields, because we didn’t need them. But because of some garbage in uninitialized structure, behaviour was incorrect. By occasion in previous build code worked fine because of some luck and specific frames order. But after changing some lines code stopped working.

And this is the fix with zeroing msghdr structure:

struct msghdr msg;
memset(&msg, 0, sizeof(msg));
msg.msg_name = (void *)&dest_addr;  
msg.msg_namelen = sizeof(dest_addr);  
msg.msg_iov = &iov;  
msg.msg_iovlen = 1;

After initializing msghdr struct valgrind is happy and code works fine.

==80502== HEAP SUMMARY:
==80502==     in use at exit: 0 bytes in 0 blocks
==80502==   total heap usage: 1 allocs, 1 frees, 72,704 bytes allocated
==80502== All heap blocks were freed -- no leaks are possible
==80502== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

So the conclusion is simple: memset all structures with zero and use Valgrind to detect the reason of program’s mysterious behaviour. See you.