Best Python memcache client: Python-memcached v/s Pylibmc

Recently, I tried to access memcached server on local and remote setup in a Python application. I am sharing my experience with two of most popular clients.

Python-memcached

• The first choice was a pure  Python solution to access memcached, that is python-memcached. It is a simple to install, understand and use solution.
• It do not offer many customization setting, at least on its README 🙂
• Very less or non-existent, useful documentation
• Fails consistently for a high frequency request to memcached server. Failures are for simultaneous read on a file xxx.  I could not find an easy fix for this problem.
• I do not suggest using it for a highly scalable application.

Pylibmc

• Bare minimum Python wrapper to C/C++ API to memcached
• Installation was simple similar to Python-memcached
• Offers many useful options during connection setup (such as non-blocking requests, TCP no-delay disable)
• Shows no problem in demanding and highly scalable application
• Shows better performance than Python-memcached

My verdict: Pylibmc is a clear winner 🙂

Potential pitfall of ‘new’ operator in C++: Missing ‘NULL’

I faced a very interesting problem in a C++ code. The code logic was as follows:

#include<stdio.h>

int main()
{

int *p = new int();
int* B[10];

for ( int i =0; i<10; i++)
{
delete p;
//p = NULL;
printf(“Address1 =  %x\n”, p);

int *q = new int();
printf(“Address2 =  %x\n”, q);

B[i] = q;

i++;
}

}

Output:

kanaujia@ubuntu:~/Desktop/ToKeep/cprogs$ ./a.out
Address1 =  96f8008
Address2 =  96f8008
Address1 =  96f8008
Address2 =  96f8008
Address1 =  96f8008
Address2 =  96f8008
Address1 =  96f8008
Address2 =  96f8008
Address1 =  96f8008
Address2 =  96f8008

This code looks pretty simple and with no bug, right? But, it has an interesting problem. The first memory allocation of integer will give us an address from the heap (say, it is 0x12345).

Now inside the loop, with first iteration, we free this memory. ‘new’ will mark this address 0x12345 in free-list. Next , we again ask an integer memory allocation. And ‘new’ returned me same address 0x12345 for this allocation. I save this address in array B. Next and henceforth forth iterations will call ‘delete’ on 0x12345, and again ask an allocation. This request again returns 0x12345. So, we end up with single value of 0x12345 for *all* elements of arrayB.

How to fix this:

Always mark the pointer to NULL after calling ‘delete’. Just mark p as NULL here.

9         for ( int i =0; i<10; i++)
10         {
11                 delete p;
12                 p = NULL;
13                 printf(“Address1 =  %x\n”, p);
14
15                 int *q = new int();
16                 printf(“Address2 =  %x\n”, q);
17
18                 B[i] = q;
19
20                 i++;
21         }

Output:

kanaujia@ubuntu:~/Desktop/ToKeep/cprogs$ ./a.out
Address1 =  0
Address2 =  9265008
Address1 =  0
Address2 =  9265018
Address1 =  0
Address2 =  9265028
Address1 =  0
Address2 =  9265038
Address1 =  0
Address2 =  9265048

That’s it folks! Hope you enjoyed it.

Thoughts on malloc()

* Always check for equal number of malloc()/free() calls.
• Use calloc() instead of malloc() + memset()
• Instead of allocation memory in a piece-meal manner, try to allocate a bigger chunk with malloc() and keep on resizing it with realloc() if needed.
• malloc() over-commits the memory i.e. it gives you an address which actually does not exist!
• malloc() grouping & reallocing with “more than you want”

One interesting question: What is the behavior of following program?

int main()
{
    int *p = (int*)malloc(10);
    p[10] = 1024;
}