Incremental compilation made easy.

TCC includes libtcc. Check out the API here.

TCC’s release cycle seems a… er, bit slow… but the mailing list and git repo seem active enough to keep tabs on.

I love stumbling across neat shit like this. =)

Fuck it.

After too many attempts at compensating for C’s lack of a usable goto turning into reinventions of trampolines or ucontext.h, I said fuck it. I wasn’t having *fun* anymore. C as a “portable assembly language” simply fails.

NekoVM however has been a fucking delight to work with. I don’t even need to bother with half of the conversion as it supports XML which matches damn near 1:1 with sexps to begin with. I can pretty much just go (display (xml-conv (cps-conv (simplify (parse (read)))))) and be done. Hot.

Hmm.

It occurs to me that just last week I was thinking the implementation of a garbage collector was ‘hard’. And wound up with a basic tri color mark/sweep that’s soon to become generational rather than relying so heavily on free lists. Compared to fighting the stack in C, it’s nothing.

It also occurs to me that by next week, I might find something that makes trying to get TCO and continuations in C seem just as relatively painless.

I cannot decide whether this frightens or excites me.

General Tso’s Chicken for the win, motherfuckers. I’m going to bed now.

I see why VMs are so popular…

At least for functional-flavored languages. C’s stack and the lack of any explicit way to manage it really makes coroutines/fibers a pain in the ass, and turns any attempt at implementing tail calls into a horrific nightmare. At least if you care about performance /and/ portability. It does however excel at implementing state machines, which gets you about one step away from a VM. But to do them (TCO/fibers) directly you’ve really gotta jump through some hoops. The easy solution would be to use goto… but C won’t let you jump to labels across functions (and for good reason, given they didn’t have its use as a /target/ language in mind).

So let’s see some options I’ve come across. We’ll start with fibers/coroutines as anyone bothering to read this will probably find them more useful (and many of the benefits of TCO can be had with them anyway)…

Protothreads. This is probably the single coolest bit of C I’ve ever seen. In a nutshell, it’s a set of macros which turn a function into an instance of Duff’s device where calls to yield form the boundries of the case statements.
Pros:
ANSI C, fast, and ridiculously lightweight with a simple implementation (if you grok duff’s device). Includes a timer mechanism which can be leveraged for some semblance of preemtiveness.
Cons:
Totally loses local variables between yeilds/timeouts. Its idea of “thread” is limited to a function, blocking calls further down the chain need extra work to deal with. Implementation details make use of switch dangerous.
Other:
Employment of static declarations where nessesary were succesful but known problems with volatile makes some other possibilites dangerous. Blocking issue wasn’t particularly investigated, the code generated was treating functions as actors — they were /all/ protothreads with timeouts around the few I/O related functions exposed to the HLL. Performance as such was surprisingly decent. The cons are trivial to deal with in the small code bases of its intend use — embedded devices.

pthreads
Heavyweight OS-backed threads.
Pros:
Has the obvious advantage of being preemptive and able to utilize multiple cores across multiple processors without any additional work.
Cons:
Has the obvious disadvantage of being preemptive in a world where people are sadly often too fucking stupid to use something like STM and/or message passing, preferring instead to fool themselves into believing that they can handle manual locking while walking along the edge of non determinism bolted into a serial language rather than reserving it for cases where profiling actually shows it to be nessesary, which frankly for “typical desktop applications”, it probably isn’t.
Probably a bad idea on typical computers if you intend to spawn half a million threads but obviously useful for distributing fibers across cores.
Not ANSI C, but there are many libraries available that can ease portability in various ways such as Pthreads-w32 or a host of libs that simply abstract the native threading capabilities (if any) of whatever OS they’re running on.

setjmp/longjmp
Not a whole lot to say, essentially just saves/restores the stack pointer and registers for later restoration. ANSI C, fast, but takes more work to build coroutines out of since you can only jump /down/ the /current/ call stack, provided the matching setjmp is still /below/ it. This bites a lot of first attempts at such, as well as a few attempts at using it for exception handling. Chicken Scheme uses this when nessesary (stack is almost full) to jump back down to a trampoline after moving any live stack data to the heap in its implementation of TCO, folowing the ideas of Cheney on the M.T.A., from which it is free to generate the C code in continuation passing style, and thus provide call/cc, which gets you to whatever sort of threading or exception handling you want to build out of it.

ucontext.h
What a lot of people wish setjmp/longjmp were (and many would despise…). Basically gives you one shot continutations by saving the entire stack (rather than just the stack pointer) to heap for later restoration. Unfortunately you need POSIX to get this. But if you have it fibers become nearly trivial. A lot of wrapping libs seem to happily use it where available with or instead of pthreads such as Gnu Pth etc…

Most others attempts I’ve seen at getting coop threading or fibers in C boil down to various wrappers around or reimplementations of setjmp.h or ucontext.h with their own various performance and portability characteristics, many of which were facinating. While not about threads per se, Implementation Strategies for First-Class Continuations is a great overview of differrent methods with more detail than I could ever be bothered with, where even just simpler one shot continuations gets you a stone’s throw from fibers and exceptions and a lot of the problems are the same.

Unfortunately most of this doesn’t really help me to get here, which is where I really want to be (and from which I get the rest practically for free). At least in any intuitive fashion. Surely I can fake it (as mentioned with protothreads) but it just feels… dirty. Of the other possibilities trampolining is common and simple to implement, and Cheney on the M.T.A.’s/Chicken’s way of doing it while also using the stack as part of the GC system I think is quite slick, but they also just don’t seem right. Alas, most the other attempts I’ve seen involve inline assembly, relying on compiler specific extentions (GCC and LLVM will do TCO at least on intel and ppc), or editing the resulting asm to replace calls with jumps before the final assembler/linking passes, which is comical at best, and none of which I’ll attempt. So it seems to me that my only real options are to by whatever means make each function a thread/actor at the C level (as done) or adopt some form of trampolining.

Or… quit worrying about any of it and just implement a VM, which C excells at. Or use any of a number of existing VMs (Neko looks particularly nice for my uses, other than its being under the lgpl which has a tendency to make my brain segfault whenever I attempt to parse it).

Really though, I wish C would just give me a fucking nonlocal goto to use when generating code. Or for someone to throw money at the C– project. And while I’m wishing, being able to depend upon the presence of ucontext.h when I’m writing C by hand would be fucking awesome. I think the C1x people should do that rather than a specific threading implementation, though for many uses the effect would be the same (assuming it retains state as I’d expect, anyway), and just as braindead for embedded devices.

Eh, ok, enough half incoherant rambling. I’ve been awake far too many hours and my stomach is threatening to start digesting my tongue if I don’t provide it with real food soon.

Graph Reduction

Read The Implementation of Functional Programming Languages over the weekend. Didn’t dig it as much as SICP or EOPL or LiSP, but I’ve finally grokked the basics of graph reduction evaluators and have a couple simplistic ones running on LLVM now .

Two days to fustration in #C

The first day I took some flak to be sure. Especially after mentioning getting back into C after time with Scheme. E.g. being told by three different people that I couldn’t do something, and that if I want a language that has feature X then I should use a language that has feature X. All
I was looking at was retaining some scoped state between function calls to write a generator without having to get into coroutines or breaking it into comically short files. In this particular case, adding some peculiarities to rand() rather than calling it multiple times for different ranges. Did it scant moments later with a ‘static’ declaration and hadn’t posed it as a question in the first fucking place. But like I said, I had mentioned being into Scheme. I expected some flak.

On to today, amacleod had actually suggested that iterating over an array (or in this case, a string) simply to copy its contents was “quite fast”, asking “how do you think the C libraries do it?” when I mentioned it’d be piss slow. I guess in amacleod’s world there’s only one implementation of the std C libs, and it’s a clone of the examples in K&R. For the rest of us in
the real world there’s http://bytes.com/topic/c/answers/217848-memcpy-vs-performance or http://cboard.cprogramming.com/c-programming/92339-memcpy-vs-memmove.html or any of a dozen
others all saying the same thing. Or god forbid, actually banging out the code and doing some profiling…

But hey, whatever. I fucked off and made some bacon and eggs. Nom. But when I came back it was to see /this/ bullshit in my buffer.

16:21 < amacleod> However, if you can put it in there you can’t use it as a sentinel because it’s just the number 0.
16:22 -!- para7 [~para@port-83-236-166-89.static.qsc.de] has joined ##C
16:23 -!- zommi [~andreas@nat/ibm/x-lhegmcnivhijoqxu] has quit [Read error: Connection reset by peer]
16:23 -!- arvind_khadri [~arvind@unaffiliated/arvind-khadri/x-2237230] has joined ##C
16:23 < amacleod> cc7gir, it’s really unclear to me what your code is trying to do.
16:23 < amacleod> Helsinkiii, so, tell me what “starting from the beginning” looks like in C code.
16:24 < cc7gir> amacleod: I want to find the mode of the the list of numbers
16:24 < cc7gir> the most frequently occuring number
16:24 < Helsinkiii> amacloed: probably string[i] where i=0
16:25 < Helsinkiii> i’m 2 days into c and im coming from Java, so yeah. i can’t afford our class textbook
16:25 < amacleod> Helsinkiii, yeah.. int i; char string[] = “hello”; for (i = 0; ; ) { } sounds like a good starting point.
16:25 < PoppaVic> Nice.. Well, accept the fail and off to the next class

I won’t expect the guy who lives in a strange monopoly libs world to try just answering the question (which was along the lines of copying until in a loop… which has a one line solution in the K&R book he thinks all libs are based on) but that ray of sunshine from PV was just fucking priceless.

16:26 < Helsinkiii> no
16:26 < Helsinkiii> our teacher said it’s optional
16:26 < Helsinkiii> amacleod: what i did is make 2 do whiles, nested
16:27 < Helsinkiii> the first gets the string
16:27 < Helsinkiii> the nested one copies it into an array
16:27 < Helsinkiii> then at the end, increments, and then the outer loop runs again
16:27 < amacleod> Ok.. the outer loop is because you need to get more than one string from input?
16:27 < Helsinkiii> yes
16:27 < Helsinkiii> as many as the user wants, unitl he presses enter on a blank line
16:27 < Helsinkiii> aka he enters null
16:27 -!- Xorlev [~xorlev@wl-dhcp181-207.Mines.EDU] has joined ##C
16:27 -!- Xorlev [~xorlev@wl-dhcp181-207.Mines.EDU] has quit [Changing host]
16:27 -!- Xorlev [~xorlev@unaffiliated/xorlev] has joined ##C
16:27 < amacleod> so, as nadder pointed out about half an hour ago, you can just use the address of the first element of the array as the target of fscanf
16:28 < amacleod> er.. fgets, I mean.

Which he may have, if he hadn’t been told to use a loop instead simply for the sake of disagreeing with me. But even better, now the alpha asshole steps in, who amacleod immediately gives lead to… and who, to be fair, a concesending prick though he may be, at least has the virtue of actually being familiar with C.

16:28 < Zhivago> You appear to be confused as to what “null” means.
16:28 -!- deadlock [~no_uid@unaffiliated/deadlock] has quit [Ping timeout: 272 seconds]
16:28 < cc7gir> me?
16:28 < Helsinkiii> no me
16:28 < amacleod> Heh.
16:28 < amacleod> Both a yaz, actually.
16:29 < Helsinkiii> so, how would i wrote that. whats the syntax like for doing such a feat
16:29 < amacleod> Helsinkiii was most recently speaking of the empty string (“”) as though it were “null”.
16:29 < Zhivago> I suggest reading the documentation for fgets.
16:29 < amacleod> I concur.
16:30 < Zhivago> If you cannot read then it is time to consider another profession.
16:30 < Zhivago> Perhaps landscape gardening.
16:30 < Prael> Man, I’m glad I didn’t learn C here.

That “empty string” would be a newline which is far from “empty” in C (really amacleod, leave it to Zhivago…) but other than that, ‘nuf said. If other #C’s on other networks are anything like Freenode’s, I’m hardly surprised that C’s popularity is constantly declining.

Still, it’s been serving my purposes quite admirably. One of my ‘toy interpreters’ is now a (mostly) self compiling (to C99) actors language, somewhat like an Objective C with implicit green/lightweight threading and sexps. *grins*.

FreeBSD 8.0 on eee 1000HA

I’ve given Ubuntu 9.04 and 9.10 a spin on my netbook for a good while now, and while I was mostly happy enough with them, I had to manually add a repo just one to many times, and had it stuff shit in /etc/ waaay too many times. So we’re hopping back to FreeBSD now. Which also now has memstick images available for download; handy.

Alas, some snags. Namely audio, X, and wifi.

Far as audio, echo ‘snd_hda_load=”YES”‘ >> /boot/loader.conf and you’re done.

Moving on… X. FreeBSD 8 doesn’t include X in the base install anymore. I can’t decide if I like this choice or not. On the one hand… it really is just an application, and not one which they author. On the other hand… so are Bind and Sendmail. Anyway we just:
portmaster x11/xorg && X -configure && mv xorg.conf.new /etc/X11/

If you like, enable moused in rc.conf beforehand.
And for whatever reason (possibily because I actually used xorg-minimal and installed the rest of the bits nessesary?) I had to tack this to the beginning of my xorg.conf:
Section “ServerFlags”
Option “AllowEmptyInput” “off”
EndSection

Which leaves wifi. The pain in the fucking ass AR2425. It still doesn’t work automagically, but the solution was:
echo ‘wlans_ath0=”wlan0″‘ >> /etc/rc.conf
and then proceed as usual using wlan0.

I’ve not dicked with the SD card reader or webcam yet, but both have worked in NetBSD and Ubuntu… they’re just USB devices… and I installed from a 4GB SD card, so…

That’s pretty much it. If anyone else hits any of these, I hope this helps.

For ports in general, remember to pass -j2 (or -j4, or whatever) to make when building. If you’re using portmaster, you want to use -m’-j2′. This just tells it to run a couple compilers at a time, otherwise it’ll only use one of the silly ass ‘hyperthreads’, effectively only using 50% CPU. Or recompile the kernel without SMP support and not worry about it; the Atom doesn’t really have enough redundency for SMP on it to make much sense anyway.

For those preferring packages vs. compiling, remember you can add -P or -PP to portmaster or use pkg_add -r, but I generally pass -g to portmaster to let it build packages in case a reinstall or rollback is nessesary.