

Talloc: The Tutorial - papaf
https://talloc.samba.org/talloc/doc/html/libtalloc__tutorial.html

======
xyzzy123
It's cool, but a seldom recognised downside to custom allocators is having to
reinvent heap protection.

[http://www.void.at/exploits/hoagie_samba_packetchaining.c](http://www.void.at/exploits/hoagie_samba_packetchaining.c)

[https://lists.samba.org/archive/samba-
technical/2013-October...](https://lists.samba.org/archive/samba-
technical/2013-October/095546.html)

------
girvo
So, I've been learning C (through "Learn C The Hard Way") to be able to
contribute to various projects (PHP being one), and build my own system tools,
as well as learning it so that I can understand Vala and Nimrod's underlying
architecture and functionality better (Vala's GObject system seems like a
leaky abstraction at times, and I get stuck as I'm no good with C...).

For my projects and the like, the other stuff I want to play with, is it worth
using libs like these? They seem to "bring C into 2013", if you will, in terms
of the features possible without re-inventing the wheel.

Speaking of re-inventing the wheel, I wish I knew where to look to get
specific libraries for C, something like Ruby's Gems, and PHP's Composer. For
now, I've just been using Google. Is there a better way? And does anyone have
a good resource for explaining how to set-up a project with dependencies that
are both binary and need to be compiled? I know a little bit of Makefiles now
(I love it, and use it across all my languages now for various automation
tooling), but feel like I'm not sure when to "expect a .so in the system" vs,
brining the dependency with my archive and compiling locally/statically or
dynamically, etc. There's a lot behind it all, and I don't know where to
start, so I've been winging it!

~~~
slashdotaccount
> where to look to get specific libraries for C

[http://ccodearchive.net/](http://ccodearchive.net/)

------
abcd_f
Talloc borrows the block hierarchy idea from halloc and extends it with
destructors and some fluff. So if you need just the hierarchy, take a look at
[http://www.swapped.cc/halloc](http://www.swapped.cc/halloc)

EDIT -
[https://github.com/apankrat/halloc](https://github.com/apankrat/halloc) too

~~~
papaf
I've used (and like) halloc but I found talloc interesting because it also
supports memory pools and is well documented.

------
jbert
Apache has a nice system where all allocations come out of pools.
[http://apr.apache.org/docs/apr/2.0/group__apr__pools.html](http://apr.apache.org/docs/apr/2.0/group__apr__pools.html)

You can then create a new pool to handle a single http request, then discard
the whole pool with one free when the request has finished.

~~~
csmuk
Spot on. This and APR make stuff a lot simpler. Have used pool allocation on a
number of projects.

------
esneider
Hierarchical memory allocations are a great concept. But Samba's talloc is
kind of bloated, with all the reference counting, destructors and stuff.

Here is a simple, 200-line version of just the hierarchical allocator:
[https://github.com/esneider/talloc](https://github.com/esneider/talloc)

~~~
huhtenberg
Too simple ;)

    
    
      void * p;
    
      p = talloc(1, NULL);
      talloc_set_parent(p, p);
      tfree(p);

~~~
esneider
Yep, you can also do

    
    
      void *p;
      free(p);
      free(p);
    

Don't get me wrong, I get where you are going. But C philosophy is that
libraries shouldn't do expensive checking to detect programmers' errors.

~~~
huhtenberg
I'm not going anywhere. Double-free is a known pitfall. People watch for it
and a lot of standard libraries have safeguards against it.

On the other hand tfree() hangs if a prior call to talloc_set_parent() is
passed certain arguments. Just more ammunition to shoot one's own foot. (p, p)
might be a degenerate case, but real code _will_ create cyclic references. In
such case blaming programmer is the least productive way to handle it.

~~~
esneider
Fair enough... I thought it would be more difficult/inefficient to check for
cyclic hierarchies, but it wasn't.

Fixed! Thanks.

------
rkangel
I've been looking into things that might help manage memory in some larger
applications. I've been working with GObject recently, which has some benefits
but using it is a bit of a full on lifestyle choice, and I'm not sure how
suitable it is for some of the smaller embedded environments I work in.

This might be quite a good middle ground. Need to go and have a look at the
impact on performance and memory usage.

------
kyberias
This seems to be a very useful concept and implementation. I wonder whether it
was possible to make this more transparent for C programmers.

~~~
Someone
I think C programmers would be better of moving to a small subset of C++ with
destructors. This library tracks the link between a struct and its child
objects for each instance of the struct (both the application and this library
know about the link between a user and its username field). In C++, the
knowledge 'free the pointer pointed to by field X' is kept only once. If you
have lots of objects, that will add up.

Even if you don't want to move to C++, I don't think this is easier than
writing separate functions 'allocAnX' and 'freeAnX', and being disciplined to
always use them.

The 'reference counting' mentioned in the introduction may be a reason to use
this instead, but it could not find it in the documentation elsewhere. Did I
overlook that?

~~~
JoachimSchipper
Reference counting is implemented by
[https://talloc.samba.org/talloc/doc/html/group__talloc__ref....](https://talloc.samba.org/talloc/doc/html/group__talloc__ref.html#ga1d70927673b42a6d2e42823307280502)
or
[https://talloc.samba.org/talloc/doc/html/group__talloc__ref....](https://talloc.samba.org/talloc/doc/html/group__talloc__ref.html#ga573831a64a47a33cce8c80bde085e158)
(talloc calls this "adding a parent" \- note it does do quite a bit more than
just incrementing an integer.)

You're right that talloc has a memory overhead relative to smart pointers or a
free_foo() function; talloc'ing up an int is very inefficient. Note that you
can add a talloc-integrated "destructor" at zero marginal cost, so e.g. an
array of a million pointers to individually-allocated int's could let talloc
manage the vector and free the int's in the destructor. (That is, you can de-
talloc part of a talloc'ed memory hierarchy.)

Additionally, malloc also has a runtime and memory cost; like talloc's, this
cost is high enough to discourage arrays of pointers to individually allocated
int's and low enough that it doesn't matter for arrays of int's. The one time
I played around with talloc (for a network daemon), I didn't really run into
any allocations where I'd hesitate to use talloc but not to use malloc. (I
forgot the numbers; a quick Google suggests that talloc adds 96 bytes to
malloc's 16 bytes of overhead.)

That said, just being disciplined about memory usage seems to work decently
well, too. (For expert programmers with good tools, usually; it's a hard
problem.)

------
zschoche
It looks like they understand memory as a tree, but in complex structures it's
a graph. I couldn't found how they handle ring topologies. I guess, this
stealing stuff doesn't help.

~~~
JoachimSchipper
IIRC, talloc basically doesn't handle rings. Note, however, that it's fairly
easy to create a "reference to the entire ring" that cleans up some node/the
entire ring when it's deallocated, or to explicitly deallocate some node of
the ring.

For what it's worth, non-tree reference graphs are a lot rarer than you'd
think. (Note, for instance, that Perl doesn't clean them up automatically
except in a few specific cases; this is annoying, but Perl is clearly usable!)

