From: David King Date: 21:34 on 19 Nov 2007 Subject: How NOT to write a shared library On-topic from <http://ubuntuforums.org/showthread.php?t=612606>: I'm interested in audio, so I'm looking over the (frankly horrible) documentation for this new "Pulse Audio" thing, and I see this function in its shared lib: pa_xmalloc() Allocates the specified number of bytes, just like malloc() does. However, in case of OOM, terminate. First of all, why are so many programmers who write a Linux shared library offering a simple wrapper function around malloc? Is it too hard for an app to just call malloc? Is anyone so completely absent- minded that he can't remember malloc is used to allocate memory, so he needs an equivalent wrapper in every shared lib he uses? Second of all, in the case of "out of memory", it just terminates your app right then and there??? Oh great, The enduser spends an hour working on something, does some operation, and BANG! -- all his hard work goes down the drain because some shared library decides it wants to terminate the app. Absolutely ridiculous. And I see that other pulse audio functions call this pa_xmalloc too, so good luck getting around it. I can just see it now. Every programmer who uses pulse audio will need to add a feature to his app called "Auto-save before every Pulse Audio call because this may be your last chance to preserve your work". Yeah, that should really yield "low latency". Sorry, but anyone who puts a call to exit() in a shared library should simply be shot dead. That's really, really crummy, amateurish programming. In a shared lib, if something fails, the professional thing to do is return some error code/signal to the app, indicating that the function failed. Let the app decide what it wants to do. You don't terminate the app on your own. And anyone who makes such a shared library a part of his operating system should be shot dead, and then stabbed through the heart with a stake just to make sure he's really dead. God help us if this pulse audio thing is ever chosen to replace ALSA in the kernel. This will make Linux stability and reliability go to hell if you have operating system calls terminating apps at will.
From: Martin Ebourne Date: 22:00 on 19 Nov 2007 Subject: Re: How NOT to write a shared library On Mon, 2007-11-19 at 13:34 -0800, David King wrote: > On-topic from <http://ubuntuforums.org/showthread.php?t=612606>: > > I'm interested in audio, so I'm looking over the (frankly horrible) > documentation for this new "Pulse Audio" thing, and I see this > function in its shared lib: > > pa_xmalloc() > Allocates the specified number of bytes, just like malloc() does. > However, in case of OOM, terminate. Complete insanity!! Don't they think that malloc returns 0 for a reason? That Kernighan and Ritchie were both plain stupid (the evidence suggests they weren't)? That none of the ANSI & ISO people noticed? > Sorry, Don't apologise - the only sorry thing is this API call. > but anyone who puts a call to exit() in a shared library > should simply be shot dead. That's really, really crummy, amateurish > programming. In a shared lib, if something fails, the professional > thing to do is return some error code/signal to the app, indicating > that the function failed. Let the app decide what it wants to do. You > don't terminate the app on your own. Absolutely. As someone who already uses pulseaudio (and has been lucky enough not to have random app exits yet) I'd be happy if you pointed out the error of their ways to them. Hopefully then they'll fix it before my apps start exiting. Shame really, because the concept is great (and many years overdue) and it does seem to work quite well. Hmm, better go and increase swap size though... > God help us if this pulse audio thing is ever chosen to replace ALSA > in the kernel. This will make Linux stability and reliability go to > hell if you have operating system calls terminating apps at will. Well you're quite safe there at least, pulse is a userland sound server and not a sound driver at all, so it'll never go anywhere near kernel space. (Besides there's quite a few kernel hackers that'd grind the perpetrator's digits off before letting such code near the kernel.) Cheers, Martin.
From: Peter da Silva Date: 22:35 on 19 Nov 2007 Subject: Re: How NOT to write a shared library I've got mixed feelings about this one. On a PDP-11, malloc() fails pretty often, and you've generally got an idea of what to do to handle it. On a modern desktop if malloc() starts failing you're in deep shit. You've filled up RAM, *and* swap, and paged out everything that can be paged out. On a system with a swapfile instead of a partition you've probably also filled up hour hard disk. By the time malloc fails you're out of sump water and you're trying to suck the sweat out of your jockstrap. Or else you've got a totally corrupt heap and you've probably already scribbled all over yourself. Of all the horrible things I've seen in shared libraries, on a scale of 1 - 10, I'd call this about a 2 or a 3.
From: Jarkko Hietaniemi Date: 22:53 on 19 Nov 2007 Subject: Re: How NOT to write a shared library Also, wrapping the malloc is not *always* a bad thing to do: there are mallocs out there that do things you wouldn't expect, e.g. if you try to malloc() zero bytes. (Yes, there are legitimate reasons to do that.) Or you need to write your own memory management abstractions on top of malloc, like slab allocators, or memory pools (per-thread, or for some other reason), or add guard bytes, or ...
From: Nicholas Clark Date: 11:18 on 20 Nov 2007 Subject: Re: How NOT to write a shared library On Mon, Nov 19, 2007 at 05:53:28PM -0500, Jarkko Hietaniemi wrote: > Also, wrapping the malloc is not *always* a bad thing to do: there are > mallocs out there that do things you wouldn't expect, e.g. if you try > to malloc() zero bytes. (Yes, there are legitimate reasons to do that.) C89 - we've heard of it. [And if the world hasn't got C89 right yet, what hope is there for C99?] Nicholas Clark
From: Peter da Silva Date: 23:31 on 19 Nov 2007 Subject: Re: How NOT to write a shared library On 19-Nov-2007, at 16:50, Steff wrote: > Perhaps I'm living in utter innocence here, but my desktop/laptop > systems have a swap partition of fixed size. And they're set WAY too small, too, if it's windows. The old "2 times RAM" logic for swap files was never quite sane to begin with, and it's totally wack now. The whole swapfile/partition voodoo is just pure hate. > Also, would it be impossible to handle the error by emitting some > thing along the lines of "for Jeff's sake delete something from > $disk, I'm dying here" to stdout, stderr and anything else that > springs to mind and periodically retrying the malloc rather than > just dying? From a shared library, the only options are aborting or returning a fatal error that the application should probably respond to by terminating. The application might try something clever, but the results are unlikely to make you happy. What happens when individual programs all try and deal with a system-wide memory shortage using their own clever algorithm is your "computer fletely, mouse and all" (STR).
From: Michael Poole Date: 23:01 on 19 Nov 2007 Subject: Re: How NOT to write a shared library David King writes: > On-topic from <http://ubuntuforums.org/showthread.php?t=612606>: > > I'm interested in audio, so I'm looking over the (frankly horrible) > documentation for this new "Pulse Audio" thing, and I see this > function in its shared lib: > > pa_xmalloc() > Allocates the specified number of bytes, just like malloc() does. > However, in case of OOM, terminate. > > First of all, why are so many programmers who write a Linux shared > library offering a simple wrapper function around malloc? Is it too > hard for an app to just call malloc? Is anyone so completely absent- > minded that he can't remember malloc is used to allocate memory, so > he needs an equivalent wrapper in every shared lib he uses? The system version of malloc() is often Not Elite Enough for your enlightened library authors -- q.v. glib, which apparently wants to use its own allocator for some pretty sketchy reasons. Alternatively, the system allocator(s) is/are Too Damn Broken to work reliably in the various permutations of DLL Hell -- q.v. Windows -- and calling free() on a pointer that another DLL malloc()'ed can crash your application or silently corrupt the heap. (I wish I were joking about this.) That all depends on how portable you want your library API to be, but I agree on the folly of exporting a function that terminates the application on failure. Michael Poole
From: Nicholas Clark Date: 11:20 on 20 Nov 2007 Subject: Re: How NOT to write a shared library On Mon, Nov 19, 2007 at 06:01:17PM -0500, Michael Poole wrote: > Alternatively, the system allocator(s) is/are Too Damn Broken to work > reliably in the various permutations of DLL Hell -- q.v. Windows -- > and calling free() on a pointer that another DLL malloc()'ed can crash > your application or silently corrupt the heap. (I wish I were joking > about this.) Is this documented behaviour? Nicholas Clark
From: Michael Poole Date: 12:51 on 20 Nov 2007 Subject: Re: How NOT to write a shared library Nicholas Clark writes: > On Mon, Nov 19, 2007 at 06:01:17PM -0500, Michael Poole wrote: > >> Alternatively, the system allocator(s) is/are Too Damn Broken to work >> reliably in the various permutations of DLL Hell -- q.v. Windows -- >> and calling free() on a pointer that another DLL malloc()'ed can crash >> your application or silently corrupt the heap. (I wish I were joking >> about this.) > > Is this documented behaviour? The documentation used to warn against doing things that would trigger it; the Visual C++.NET documentation doesn't mention it anywhere I can find. It is a side effect of the multiple heaps you get when you link your DLL against a static version of Microsoft's C runtime (for example, to avoid locking your DLL's user into a certain version of the C runtime DLL), or if your DLL links against a different variant of the C runtime DLL than the application. When you allocate memory from one heap and try to free it in another heap, the debug variant of the runtime library throws an assertion failure. It's been a while since I had to deal with Windows, but I believe the release versions of the runtime continue with silently corrupted heaps. (I suspect this is part of why third-party Windows libraries ship with both debug and non-debug variants: linking against the multi-threaded DLL runtime is an easy workaround, but you're still left choosing between the debug and release options of that, and they use different heaps.) Michael
Generated at 10:27 on 16 Apr 2008 by mariachi