R5RS syntax-rules macros in Gambit Scheme

Stumbled upon Alexey Radul’s list of Scheme Implementation Choices, and noticed that Gambit was listed as no-syntax-rules:

The syntax-rules macro system is off by default in several Schemes. I tried as best I could to find out how to turn it on, but was not uniformly successful.
[...]
RScheme and Gambit produce an error

Well I remember it being a bit tricky to figure out how to do this in Gambit myself not too long ago, so I thought I’d share the method here and hopefully make it easier for others to find.

> (load "~~/syntax-case")
"/usr/home/Martin/.snow-site/v1.1.2/host/gambit/v4.2.0/syntax-case.scm"
> (define-syntax foo (syntax-rules () ((_) 1)))
> (foo)
1
>

or

$ gsi -e '(load "~~/syntax-case")' -
> (define-syntax foo (syntax-rules () ((_) 1)))
> (foo)
1
>

I’m not fond of the ‘magic’ file path to get to it, but it comes with Gambit and it does work. If I recall correctly, it’s a port of psyntax.

Scheme, Forth, and C

I came across an article entitled Forth Versus C, and I couldn’t help but notice how much of it applies directly to Scheme (and mostly Common Lisp) as well, with only a few changes in terminology. Naturally not the parts involving its dual stack nature and such, but that’s far from the majority (and you could bring up a similar topic anyway, just compare the C stack to call/cc instead).

This bit especially caught my attention, especially in light of another recent post:

Q. But how can a non-programmer even read a program, let alone tell whether it is right?
A. Forth syntax can be learnt in an hour or two. If the top level of the program cannot be read and understood by someone who understands the problem domain, it is wrong and you should change it.

I would also say this would apply to *any* Domain Specific Language. Lisps (and apparently Forth) make constructing DSL’s trivially easy, any many of the pros and cons of such are discussed (in relation to C).

Back on topic though, I think this article neatly describes many of the fundamental differences between Lisp and C just as well as Fortran and C. I highly recommend giving it a read.

A new Lisp forum

briancarper.net writes:

Ten days ago I complained that there were no good Lisp equivalents of ruby-forum or perlmonks. It looks like someone went and made one. What good timing.

You can find it here.

And yes, it has a Scheme subforum along with the expected Common Lisp. And Elisp, Clojure, and Arc subforums, as well as a catch-all Other.

A bit odd that it’s a PHP forum for a Lisp community… who knows, maybe it’s running on Roadsend (which is built upon Bigloo Scheme)?

Regardless, it’s about damn time!

How I feel about Scheme’s performance.

I came across this post written earlier today, How fast is Scheme? Well…. which states:

I don’t know much about Scheme [...] but it seems that the Scheme compilers produce quite sluggish code, at least looking through the grainy, distorted lens that is the Computer Language Benchmarks Game.

That seems to make enough sense to me. For one-off heavily numerical tests, Scheme pretty much sucks. Especially considering that to compile the code it often has to go through C.

He goes on to say:

SBCL comes out very much on top; in general the Gambit programs take two or three times as long to do their job (although I haven’t looked at memory usage). But as far as Scheme compilers go, Gambit seems to be improving things.

With the right declarations, SBCL has been known to outperform straight C as well. Its compiler is really quite something. And (IIRC) it doesn’t have the “handicap” of using C as an intermediate language.

Personally however, I tend to consider C generating Scheme compilers more practical for issues of portability and/or easier FFI than performance. Chicken’s a good example of braindead-easy FFI. And I feel where Gambit really shines is when you need many, many threads. It’ll handle hundreds of thousands of threads in a breeze. Termite is a good example of exploiting its threading capabilities. RScheme does real-time GC. SCSH has replaced every use I’ve had for shell/perl/etc scripts. There’s a decent chance that any random Linux box already has Guile installed. SISC and Kawa leverage Java’s JIT machiery and provide trivially easy Java FFI. Bigloo can compile to Java bytecode as well. MzScheme runs its own JIT compiler by default where it can, and is prefered to its C output. And there’s a JIT option for all the C-generating Scheme’s as well, compile with LLVM instead of GCC. I think it’d be especially interesting to compare MzScheme’s JIT to MzScheme’s C + LLVM. And if you’re going the Scheme->C route for performance reasons, I’d think Stalin to be the obvious Scheme to use, whether with GCC or LLVM depending on its expected run time (assuming there’s even noticeable startup overhead with LLVM, it’s embedded VM is really quite minimal). Many of them have really nice OO systems built in as well

But none of these tests run long enough to let any of the JIT options really shine. Basically, I say throw any of the JIT options into the mix, and make the tests long enough to really let them do their magic.

I very strongly suspect the same holds for Java, having run a Freenet node for quite some time it gets noticeably snappier once the JVM has had a chance to see it run for a while, especially for recent versions of Java. I’d also argue that’s exactly (other than its relatively primitive GC) what makes Java so incredibly horrid for client-side stuff such as applets, you’re loading a JIT compiler and GC you’ll hardly get a chance to even use before you’ve closed the window. But its JIT has been making leaps and bounds lately, as mentioned on Good Math, Bad Math:

About a year later, testing a new JIT for Java, the Java time was down to 0.7 seconds to run the code, plus about 1 second for the JVM to start up. (The startup times for C, C++, and Ocaml weren’t really measurable – they were smaller than the margin of error for the measurements.)

This is from the previous measurement of 1 minute and 20 seconds for Java. As I said before, SISC, Kawa, and Bigloo will happily use the Java VM. Straight C scored 0.8 seconds. OCaml kicked all their asses even before compilation, but that’s not the point here. If you really need every last bit of performance you can get though, OCaml seems to be worth looking into.

So yeah, I’m *very* interested in what the performance possibilities for Scheme really are, if nothing else out of shear curiosity. Maybe I’ll wind up running a few benchmarks of my own at some point, reiterating the tests, say, a hundred or a thousand times each… but in the end, even if this does increase their performance relative to SBCL… you don’t need to use an obscure implementation or do JIT tricks with SBCL in the first place. A lot of people havn’t heard of Stalin or LLVM. A lot of people don’t want to load one language (Java) to run another (Scheme). Although again, I’d question whether MzScheme’s performance is really so bad in the long run.

And I’d question whether it was really worth it in the first place. Fluxus and Impromtu are two obvious examples which come to mind, both heavy graphics/audio livecoding systems, which solve many problems the same way you would in, say, Python. Offload much of the heavy work onto libraries. There’s a PDF floating on the net somewhere about MzScheme controlling an array of telescopes, and of course there’s the US Navy’s Metcast project. SchemDoc, Scheme Elucidator, the LAML they’re both based on that you can just feed any XML DTD into and get a Scheme representation of that XML language in. SCWM and Orion window managers. MetaModeler for dealing with many/large databases. For web stuff there’s TeX2page, SISCweb, BRL, WiLiKi, the Hop framework, HtmlPrag, SXML, it goes on. There’s a lot of uses which don’t demand every last bit of performance from the Scheme implementation, and I’m just not really doing anything that does.

And if I were writing something and came across an annoying bottleneck? I’d likely take the NetBSD approach. Instead of trying to tweak things to run faster (LLVM, Java, implementation-specific declarations, etc…), I’d see if I couldn’t find a fundamentally more efficient algorithm first. Which reminds me, I still want those books.

[update] On the Reddit thread where apparently most of the views for this article are coming from (who’d have thought so many people would be interested in some Scheme noob’s opinions of language performance? Well over 500 hits already, scant hours later), there’s a link to a fascinating email thread discussing the floating point speed of Gambit-C. I have Brad Lucier’s paper printing to read later as I type this. There’s also a reply in the thread by Brad himself, my favorite part of which is the end summary, which is generally similar to the “NetBSD approach” mentioned above. Is there some sort of established term for this idea?

Anyway, I’ll certainly be rethinking my opinions of Gambit-C as “that threading/Termite implementation”.

[update 2] It’s morning, getting close to 1500 views for this post now… searching for r5rs performance on Google this is the 3rd result! I still don’t see why this is drawing so much attention, but as long as it is, have you seen Scheme Now!?

Scheme Now!, also known as Snow, is a repository of Scheme packages that are portable to several popular implementations of Scheme.

Snow is a general framework for developing and distributing portable Scheme packages. Snow comes with a set of core packages that provide portable APIs for practical programming features such as networking, cryptography, data compression, file system access, etc. Snow packages can export procedures, macros and records.

[update 3] 1844 views as I write this on July 1st. Also now the first result on google for the aforementioned search. Wow. o_O

Trivial single dispatch OOP with R5RS closures

Just to provide a trivial example should anyone ever be looking for such. It’s easy to find let/lambda examples, but not much with “class methods” or “class variables” (you’ll notice however that it’s basically the same thing, just using two pairs of let/lambda). I figure this would have saved me a bit of time a while ago, so maybe someone else’ll find it useful. It’s far from perfect I’m sure, but it works.

;; Is clobbered later.
(define make-obj #f)

(let ()

  ;; "Class methods" Entirely redundant, pretend these do something interesting.
  (define (setter! obj var val)
    (obj var val))

  (define (other-setter! obj var val)
    (setter! obj var val))

  ;; "Class variables" could also be done here.

  ;; "Export" the instantiator (clobbering the define above the let).
  (set! make-obj

	;; "Initialization variables"
	(lambda (x)

	  ;; "Instance variables"
	  (let ((y 1))			

	    ;; The "object" returned by the instantiator.
	    (define (self msg . arg)

	      ;; "Instance methods"
	      (define (display-vars)
		(display "X: ")
		(display x)
		(newline)
		(display "Y: ")
		(display y)
		(newline))

	      ;; Single dispatch
	      (case msg
		((x)
		 (if (null? arg)
		     x
		     (set! x (car arg))))
		((y)
		 (if (null? arg)
		     y
		     (set! y (car arg))))
		((d)
		 (display-vars))
		((s1)
		 (setter! self 'x (car arg)))
		((s2)
		 (other-setter! self 'y (car arg)))
		(else (display "Unknown message: ")
		      (display msg)
		      (newline))))

            ;; This is the return value of make-obj
	    self)))

  ;; let () ...
  )

And some sample interaction…

> (define object-1 (make-obj 3))
> (define object-2 (make-obj 5))
> (object-1 'd)
X: 3
Y: 1
> (object-2 'd)
X: 5
Y: 1
> (object-1 'x 42)
> (object-1 'd)
X: 42
Y: 1
> (object-2 's2 6)
> (object-2 'y)
6
> (object-2 'd)
X: 5
Y: 6
> (object-1 's2 (+ (object-1 'y) (object-2 'x)))
> (object-1 'd)
X: 42
Y: 6
> (object-2 's1 (/ (object-1 'x) (object-1 'y)))
> (object-2 'd)
X: 7
Y: 6

You have a couple of easy choices for subclassing/inheritance as well. When defining the subclass, make its dispatch for any unknown messages call the parent object(s) with the same message and args. The parent objects(s) can be pre-existing or you might instantiate copies of them from within your subclass, either way has its pros and cons. I’m not really going to get those though as it’s a quick way to wind up playing with call/cc and macros which is far beyond this simple example’s scope. Really if you’re doing anything that complicated, you might consider one of the many OOP systems already written to handle that sort of stuff. There’s MeroonV3, Meroonet, YASOS (Yet Another Scheme Object System), MOS (Macroless Object System), POS (Portable Object System), ScmObj, the list goes on, and all of these are pretty widely portable between implementations. YASOS and MOS both come with SLIB. I might wind up porting POS to Snow. Many implementations already come with one as well.

For the “basic” stuff when you just need a bit of encapsulation though, closures alone are plenty flexible.

Any suggestions to improve or clarify this example is welcome.

Sandvine, Linux, Windows, BSD, and an opinion.

So, Cat in the Red Hat has firewall rulesets for Linux and FreeBSD/Windows for ignoring Sandvine’s forged RST packets.

Now, I’ve always preferred BSD-style firewall configs. But I’ve never seen before such a painfully clear example of *why* I prefer them. If you’re at all familiar with how TCP/IP works, “add drop tcp from any to me 6883 tcpflags rst”, isn’t exactly brain surgery to understand. Now, does “-A INPUT -p tcp –dport 6883 –tcp-flags RST RST -j DROP” really add any relevant information so far as the action you’re trying to perform? Why is RST stated twice?

But really, that’s minor. What’s the real difference here? BSD firewalls use a Domain Specific Language, whereas Linux uses flags to an application. Either way has its pros and cons.

It’s a matter of taste, really. I prefer the one that doesn’t look like line noise, and doesn’t involve an extra process to get to use things like variables (as in the script for Ubuntu/etc…). What if I want to make 6883 a variable, holding a list of subnets? Well, the solution in the linked post was to wrap the rulesets in a shell script. Would iptables support, say, a list of subnets instead of a number as the value of that variable? Well first I’d have to read the documentation for iptables, and either way read up on a separate scripting language (Case 1: It does, and I only need to know how to set a variable in the script language. Case 2: it does not, and I need to also learn how to do a loop in the scripting language). Having a DSL for the rulesets, however, all 2 (or 3) steps above are covered within the same document, within the same section. Indeed, within the very same text. As it explains how to set a variable to multiple values, it can be inferred that rules will accept them, and that explicit looping is unnecessary. Speaking specifically of OpenBSD’s PF here, it’s simply the most elegant firewall system I’ve ever dealt with (and the only one I’d describe as “elegant”, for that matter).

Of course it could be argued that iptables doesn’t make you learn a new language, and that you can just use any scripting language you’re already familiar with (assuming you’re using a *nix because you like *nix, and not just because you hate Windows and can’t afford OSX, you probably know some script language or another). But then I’d say, look at the shear complexity of those flags… you’re still essentially learning a DSL anyway, and if you just really, really want to use your scripting language of choice? The ‘ipfw’ command is available to any OS using IPF as its firewall, and performs the same function as ‘ipfilter’ does for Linux’s firewall. And IPF has been ported to FreeBSD, NetBSD, OpenBSD, SunOS, HP/UX, OpenSolaris, Linux, QNX, OSX, Windows… being the default on at least the first two (maybe Solaris as well, I forget). Who didn’t have to learn what, now?

As an aside, the fact that there’s two configs for what is at its core the same OS, and one script for two entirely different OS’s (and all the others IPF supports, at that) just seems hilariously ironic to me. I would assume that the ipfilter script for Ubuntu/etc… would work on RedHat &co as well, but I’m still laughing.

[update] How odd. The IPTables/IPF postings in Cat in the Red Hat’s blog have disappeared. There’s a story on an IPTables-only version today on Slashdot though.

Why I hate YIM.

A while ago a friend of mine wanted me to access their webcam over Yahoo Instant Messenger (no, not for amateur porn). What the hell ever happened to web-based camera feeds? You know, the “web” in “webcam”? Does anyone here remember JenniCam? Oh well, anyway. I happened to have a Win2K install at the time, Pidgin lacks webcam support, and so I reluctantly installed YIM’s own “official” client.

————————————–
Hate:
————————————–

Forced, entirely unmentioned install of Adobe Flash. Adobe can’t even make a PDF reader not take an agonizing amount of time to load and display even the simplest of PDF’s. Xpdf and gv in Unix start, as far as practical, *instantly*. Foxit does the same for Windows boxen. And the shear size of it! Ugh. You’d think the company that invented the format could produce a decent viewer for it? Apparently not. So why should I want anything else of theirs on my system? Especially given Adobe’s history in recent memory: Adobe Quietly Monitoring Software Use? — Adobe Flash Zero-Day Attack Underway. Yeah. I totally want their shit on my computer. Riiight. To be fair though, at least they’re finally opening up the specs to the Flash format. Not that you couldn’t get them before, but now you’ll be allowed to use them to actually write a player, not just a producer, and projects like Gnash have begun to suck quite a bit less since then. Anyone willing to bet this wasn’t simply in response to Microsoft’s Silverlight? Everyone’s familiar with the phrase Embrace, Extend, and Extinguish. Just sayin’. Regardless, I was trying to install a damn IM client, not a browser plug in. The only plausible excuse I can think of for this is if it’s using Flash for its webcam support… but then why can’t I just point any old flash-capable media player at the stream?

Oh, I know why! Ads in the webcam window, unless it’s fullscreened. Want to multitask? Say, to write to the person who’s webcam you’re viewing to request a different view angle on the problematic component they’re trying to show you? And face it, for many, many users, tits? They know their userbase, and sex sells.

But that’s not the only time you see ads. They’re in your friend’s list. Constantly. Are we ad-ware yet?

During install, must opt out of installing a “toolbar”, and some other browser crap. I want an IM client, not my browser hijacked.

Weather and music crap. A VOIP phone balance telling me I have no minutes either way, which are separately billed, of course. Irrelevant plugins. What the fuck? I even told it during the install that I didn’t give a shit about VOIP. And if I wanted to stream music, or use VOIP, why the hell would it be with an IM client, from Yahoo no less? If they can be turned off, it wasn’t very obvious how to do so. Stop advertising shit I’ll never use. It makes me cringe, it makes me desperate to slam the reset button and get back into BSD, where do one thing, do it well reigns supreme. IM client. I want IM. I have a back yard. I have music. I have a cell phone, kthx?

It starts with windows, without ever having asked to do so. It’s not in the start up folder. Any software which does this is FUCKING EVIL. YIM is not a device driver, the only remotely plausible excuse for this behavior. Again, if this can be corrected, it wasn’t obvious how to do so. IM client, not a device driver.

Yahoo Messenger Insider. What The Fuck? No, I don’t want what you call “news”. I don’t want to see it every goddamn time I log into Yahoo, especially considering how piss fucking slowly it (and YIM itself) loads. I want an IM client, not a goddamn web browser.

Quit checking my fucking email account. I don’t care what’s in it, anyone fucking stupid enough to email me there doesn’t know me well enough for me to give a shit about in the first place. IM client. Not mail client. Why the hell is YIM trying to be the center of my fucking life?

————————————–
Annoyances:
————————————–

GUI. It stands for “Graphical User Interface”, not “Candyland User Interface”. Using it made me feel like I was in preschool.

No tabbed windows. An entire separate window for each person is simply… primitive. Can we get out of the fucking 1990’s, please? Maybe it wouldn’t be so bad if Windows supported tabbed window management, but it doesn’t. Unless you replace the shell, of course.

Webcam only support 1x, 2x, 3x, and fullscreen zoom levels. I can’t remember the last time I saw anything related to the display of media with such limitations. Maybe this is normal in Windows?

————————————–
The aftermath:
————————————–

Bad lightning storm, some bits of kit were fried (along with a tree, a few power line transformers…), the computer I was using is now an NFS server for my laptop, which lacking a (working) HDD is acting as a diskeless client. Because you can do that kind of shit in Unix, without a “pro” or “corporate” version of your OS and yearly Terminal Services licenses. For obvious reasons, I nolonger have a Windows install, and with it no YIM. I’ve also come to find out that Kopete supports the viewing of YIM webcams perfectly well, and have it installed for such cases where it may be nessessary again. I’d just use it exclusively instead of Pidgin, but like I said… I’m a fan of do one thing, do it well, and I don’t appreciate the resources it sucks up loading all of its required support daemons (artsd, dbus, a whole slew of shit under kdeinit…) every time it starts. Granted KDE at least has a nice modular design, I still just don’t want all of that to run a simple IM client, and I’m not interested in the Windows/OSX-cloning desktop-”integration” whoreing behaviors of KDE/Gnome/etc… in general. But that’s for a different post, at a different time.

Data Directed Programming with CLOS

Yeah, it’s been a while. I’ve mostly just been looking for jobs (I hate this town, I really do), working on a couple small projects (the distributed mud, an IRC library, a bot and client which use said library, etc…), lots of yard work type stuff, nothing really to post about.

Having no more luck as usual lately finding a job, I decided to take a break for a couple days and watch the UC Berkeley webcasts for CS61A, their SICP course. After watching the Fri 2/29 video on generic operators it occurred to me that (the general idea of) CLOS seems perhaps to be more like data directed programming than an object orientation system.

While CLOS’s brilliance has never been lost upon me, I was never quite comfortable thinking of it as being OO. I like things such as encapsulation and message passing in an OO system, it just feels more natural to me. Yes, I’ve read Graham’s Why Arc Isn’t Especially Object-Oriented, its response, all the OOP stuff at SchemeWiki, various papers/rants on why it’s a better OO than C++/Java, etc…. CLOS just doesn’t make sense as an OO system to me. I couldn’t quite “get” where CLOS fit in, and so I tended to shy away from its use. But now I have a way of looking at it which makes far more sense to me, and perhaps I’ll wind up grabbing a copy of The Art of the Metaobject Protocol sometime. More importantly, I can think of some *uses* for it now, which I’ll be exploring shortly.

Conversely, I wonder if DDP is a feasible (if, perhaps, naive) method to implement a simplistic CLOS-like system? Oh well, right now most of my interests in OO involve TinyTalk, which sadly is only available for Gambit (and that “other” Scheme-like language). I might try porting it to Snow, at some point.

Anyway, here’s the bit of code that I thought of while watching the video, to give an idea of what the hell I’m on about. It might make more sense if you’re looking at figure 2.2 in section 2.4.3 of SICP as well.

;;; Using a CLOS-ish "OOP" system for Data Directed Programming.
;;; Based on SICP 2.4.3 & a minimal psuedo-CLOS.
;;; matt.a.martin@gmail.com
;;; Released to the public domain.
;;; Fri Jun 20 15:44:54 CDT 2008

;; Meroon or Swindle or ScmObj etc...
(load "clos-ish.scm")

;; The table's colums.
(def-class polar '() '(magnitude angle))
(def-class rectangle '() '(real imag))

;; The table's rows.
(def-generic real-part)
(def-generic imag-part)
(def-generic magnitude)
(def-generic angle)

;; Define real-part-polar &c... here, possibly involving a lexical scope for "encapsulation", or use lambdas instead, etc...

;; Fill in the table.

;; (put 'real-part 'polar real-part-polar)
(def-method (real-part (x polar))
        (real-part-polar x))

;; (put 'real-part 'rectangle real-part-rectanglular)
(def-method (real-part (x rectangular))
        (real-part-rectangular x))

;; (put 'imag-part 'polar imag-part-polar)
(def-method (imag-part (x polar))
        (imag-part-polar x))

;; (put 'imag-part 'rectangular imag-part-rectangular)
(def-method (imag-part (x rectangular))
        (imag-part-rectangular x))

;; (put 'magnitude 'polar magnitude-polar)
(def-method (magnitude (x polar))
        (magnitude-polar x))

;; (put 'magnitude 'rectangle magnitude-rectangle)
(def-method (magnitude (x rectangular))
        (magnitude-rectangular x))

;; (put 'angle 'polar 'angle-polar)
(def-method (angle (x polar))
        (angle-polar x))

;; (put 'angle 'rectangle angle-rectangle)
(def-method (angle (x rectangle))
        (angle-rectangle x))