Extreme Markup 2007: Monday

This is a report on Extreme Markup 2007 for Monday.

Monday was not technically part of Extreme at all; it was a separate workshop called "International Workshop on Markup of Overlapping Structures 2007". Overlap is a special interest of mine: how to deal with marking up things like books, where you may want to preserve both the chapter/paragraph/sentence structure and chapter/page structure (or even just page structure, in books where the chapter boundary doesn't mean the beginning of a page). Bibles can add chapter/verse structure, and indeed one of the presenters was formerly with the Society of Biblical Literature.

I won't give details of the presentations: you will eventually be able to see them on the Extreme Markup site, and googling for LMNL, Trojan Horse markup, TexMecs, Goddag structures, and rabbit/duck grammars should provide more than enough information. I will, however, say a bit about how overlap is typically represented in XML, and then about the two different kinds of (or use cases for) overlap that this workshop identified.

A variety of techniques have been used to represent overlap, but work in the field is generally converging on using XML augmented with milestone elements. A milestone is an empty element that represents the start-tag or end-tag of a virtual element that can overlap with other virtual elements or the actual elements of the document. A milestone is tagged with an attribute to show whether it is meant to be seen as a start-tag or an end-tag, and how the start- and end-tags match up. So <foo sID="xyz"/> would indicate the beginning of a logical element, and <foo eID="xyz"/> would indicate the end. The xyz has no particular significance, but must be present in exactly one start milestone and one end milestone. This technique has the advantage that the virtual element's name is an element name, potentially belonging to a namespace. Clever XSLT processing can be used to convert real elements into virtual ones or (provided no overlap of real elements results) vice versa.

Overlap of the first kind descends from the old SGML CONCUR feature, and is basically about having multiple hierarchies over a single document, like the examples I gave above. Typically you want to be able to pull out each individual hierarchy as a well-formed XML expression without milestones, and then look at it or validate it or process it however you like. Often this is all that's needed for overlapping documents, and it's then a matter of deciding whether all the elements are to be milestones in the base form of the document, or whether some virtual elements (the most important ones) are left as real XML elements rather than milestones.

Overlap of the second kind cares about ranges rather than elements, interval arithmetic (and computational geometry) rather than trees and DAGs, inclusion rather than dominance. It's the kind that most interests me, and LMNL is in my highly biased opinion the state of the art here. If you want dominance information, you can mark it up using annotations (LMNL's broadened version of attributes), or just infer it from what ranges include what, but you aren't assumed to care about it: basic LMNL just has a simple list of ranges over text.

From 2:00 PM onward there was a kind of Quaker meeting, with people going to the mike and talking for 5 minutes. Originally there were speakers and audience members asking questions, but very quickly the discussion became general, as signaled by the audience mike being moved to the front of the room. (When you are in this situation, always use the mike: people with hearing aids can't hear you no matter how loud you yell.) Nothing new emerged here, but participants got to understand one another's viewpoints and systems pretty well. Extreme probably will do this workshop again in a few years, and some other kind of workshop (XML data binding was suggested) next year.

At the end of the day ...

... it's time to wake up and smell the coffee.

Noogler

I am one.

This is a personal blog. Nothing I say here represents the views of anyone else.

Lawrence of Arabia tells a tale

Sunset came down, delightfully red, and after the feast the whole party lay round the outside coffee-hearth lingering under the stars, while Auda and others told us stories. In a pause I remarked casually that I had looked for Mohammed el Dheilan in his tent that afternoon, to thank him for the milch camel he had given me, but had not found him. Auda shouted for joy, till everybody looked at him; and then, in the silence which fell that they might learn the joke, he pointed to Mohammed sitting dismally beside the coffee mortar, and said in his huge voice:--

'Ho! Shall I tell why Mohammed for fifteen days has not slept in his tent?' Everybody chuckled with delight, and conversation stopped; all the crowd stretched out on the ground, chins in hands, prepared to take the good points of the story which they had heard perhaps twenty times. The women, Auda's three wives, Zaal's wife, and some of Mohammed's, who had been cooking, came across, straddling their bellies in the billowy walk which came of carrying burdens on their heads, till they were near the partition-curtain; and there they listened like the rest while Auda told at length how Mohammed had bought publicly in the bazaar at Wejh a costly string of pearls, and had not given it to any of his wives, and so they were all at odds, except in their common rejection of him.

The story was, of course, a pure invention -- Auda's elvish humour heightened by the stimulus of Revolt -- and the luckless Mohammed, who had dragged through the fortnight guesting casually with one or other of the tribesmen, called upon God for mercy, and upon me for witness that Auda lied. I cleared my throat solemnly. Auda asked for silence, and begged me to confirm his words.

I began with the introducing phrase of a formal tale: In the name of God the merciful, the loving-kind. We were six in Wejh. There were Auda, and Mohammed, and Zaal, Gasim el Shimt, Mufaddhi and the poor man (myself); and one night just before dawn, Auda said, 'Let us make a raid against the market'. And we said, 'in the name of God'. And we went; Auda in a white robe and a red head-cloth, and Kasim sandals of pieced leather; Mohammed in a silken tunic of 'seven kings' and barefoot; Zaal ... I forget Zaal. Gasim wore cotton, and Mufaddhi was in silk of blue stripes with an embroidered head-cloth. Your servant was as your servant.'

My pause was still with astonishment. This was a close parody of Auda's epic style; and I mimicked also his wave of the hand, his round voice, and the rising and dropping tone which emphasized the points, or what he thought were points, of his pointless stories. The Howeitat [tribe] sat silent as death, twisting their full bodies inside their sweat-stiffened shirts for joy, and staring hungrily at Auda; for they all recognized the original, and parody was a new art to them and to him. The coffee man, Mufaddhi, a Shammar refugee from the guilt of blood, himself a character, forgot to pile fresh thorns on his fire for fixity of listening to the tale.

I told how we left the tents, with a list of the tents, and how we walked down towards the village, describing every camel and horse we saw, and all the passers-by, and the ridges, 'all bare of grazing, for by God that country was barren. And we marched: and after we had marched the time of a smoked cigarette, we heard something, and Auda stopped and said, 'Lads, I hear something'. And Mohammed stopped and said, 'Lads, I hear something'. And Zaal, 'By God, you are right'. And we stopped to listen, and there was nothing, and the poor man said, 'By God, I hear nothing'. And Zaal said, 'By God, I hear nothing'. And Mohammed said, 'By God, I hear nothing'. And Auda said, 'By God, you are right'.

'And we marched and we marched, and the land was barren, and we heard nothing. And on our right hand came a man, a negro, on a donkey. The donkey was grey, with black ears, and one black foot, and on its shoulder was a brand like this' (a scrabble in the air), 'and its tail moved and its legs: Auda saw it, and said, 'By God, a donkey'. And Mohammed said, 'By the very God, a donkey and a slave'. And we marched. And there was a ridge, not a great ridge, but a ridge as great as from the here to the what-do-you-call-it (hi biliyeh el hok) that is yonder: and we marched to the ridge and it was barren. That land is barren: barren: barren.

'And we marched: and beyond the what-do-you-call-it there was a what-there-is as far as hereby from thence, and thereafter a ridge: and we came to that ridge, and went up that ridge: it was barren, all that land was barren: and as we came up that ridge, and were by the head of that ridge, and came to the end of the head of that ridge, by God, by my God, by very God, the sun rose upon us.'

It ended the session. Everyone had heard that sunrise twenty times, in its immense bathos; an agony piled up of linked phrases, repeated and repeated with breathless excitement by Auda to carry over for hours the thrill of a raiding story in which nothing happened; and the trivial rest of it was exaggerated the degree which made it like one of Auda's tales; and yet, also, the history of the walk to market at Wejh which many of us had taken. The tribe was in waves of laughter on the ground.

Auda laughed the loudest and longest, for he loved a jest upon himself; and the fatuousness of my epic had shown him his own sure mastery of descriptive action. He embraced Mohammed, and confessed the invention of the necklace. In gratitude Mohammed invited the camp to breakfast with him in his regained tent on the morrow, an hour before we started for the swoop on Akaba. We should have a sucking camel-calf boiled in sour milk by his wives: famous cooks, and a legendary dish!

Essentialist Explanations, 13th edition

Thanks to the good folks at Metafilter, I have just updated my Essentialist Explanations page to its 13th edition, including (naturally) a new section entitled "Meta".

Summer haiku

How gracefully she
   inhaling the red roses
      goes down on her knees.

Family felines

(I know, it's Thursday, not Friday. So what.)

Katie sleeping

And what do you want?

Okay, I'm awake now. Can I help you?

It was not my catnip. I'm innocent, innocent I tell you....

A shell script mini-clinic

On one of the mailing lists I subscribe to, someone posted the following Unix shell script as a wrapper for a program which unfortunately littered the current directory with temporary files which it did not remove. (Because this post is not about the faults of that program, I've replaced the reference to it in the fourth line of the script).

#!/bin/bash
pushd . > /dev/null
cd /tmp
some-program $@
popd > /dev/null

The author added, "I'm sure there's a better way to write the script, but this would do the trick." So it does. However, the code exhibits a number of misunderstandings of how shell scripts work that I think are worth clarifying.

The first and fifth lines are used to preserve and restore the current working directory. However, a script (or any Unix process) always has its own working directory; changing the working directory in a script does not affect the caller of the script in any way. This is not true for shell startup scripts like .login, .profile, and .bashrc, or for Windows .bat and .cmd files, all of which should be careful not to permanently change the current working directory.

So the first improvement is to remove the first and fifth lines entirely. Since these are the only parts dependent on the particular shell being run, namely bash, it is now possible to change the first line to read #!/bin/sh. On Linux and Cygwin, this is a distinction without a difference, but on Solaris and BSD Unixes, sh is a different shell from bash, and less of a resource hog; on older operating system versions, bash may not even be present. Therefore, it's always best practice to write simple shell scripts like this one as portably as possible, running them with sh whenever you can. (Of course, there is no reason to avoid bash-specific features in full-fledged bash programs, where you are using bash as a programming language like Perl or Python.)

Lastly, using $@ to mean "all the arguments" is unsafe if any of the arguments might contain a space character. Instead, use "$@". For the same reason, $* should be avoided entirely, unless you want to reparse the arguments according to whatever whitespace they contain. The distinction doesn't happen to matter in this script, but it's a very good habit to get into in general, because some day you will have to process a file (possibly coming from Windows or the Mac) with a space in its name, and then your script will break embarrassingly.

"Who now remembers the annihilation of the Armenians?"

--attr. Adolf Hitler, on the eve of Operation Barbarossa

We do.

.

Ava Cowan Foxy

Here is a condensed version of my web site, found in the Google cache of a server which is apparently down. I've neutered the links and forms, most of which referred to equally nonexistent resources.

Essentialist Explanations, 12th edition

I've finally caught up with the incoming entries, and published the 12th edition of my page Essentialist Explanations, a list of "simplistic and often humorous" (Langmaker) explanations of the form "Language X is essentially language Y under conditions Z." There are now 876 of them; new entries are always solicited, though it takes time for me to post them.

Read and enjoy.

Comments policy for Recycled Knowledge

I reserve the right to remove comments from this blog for any reason or no reason. I will do so primarily in the attempt to maintain a civil tone here. So far this hasn't been necessary, and I hope it never will be. Of course, I will also remove anything that might get me in legal trouble or that I think is spam: so far I've done the latter but not the former.

TagSoup 1.1 released

TagSoup 1.1 is now released at http://tagsoup.info. This release includes JAXP classes for those who want them. HTML comments have been completely revamped (the implementation has been broken for many releases), and a few other small bugs fixed. Do upgrade, if only for the comment handling.

Regularized Inglish: wordlists

Here are some Regularized Inglish wordlists from Wijk.
The following words (from a list of the 1000 most frequent words) have the stressed vowel changed:
abuv, afternoone, agen, agenst, aul, aulmoste, aulreddy, aulso, aultho, aulways, amung, anuther, anser, eny, enything, ask, baul, bair (verb), bayr (noun), beutiful, beuty, becoz, becum, beloe, blud, bloe, branch, breik, bruther, braught, bild, bilding, bilt, bisness, bisy, bye (for "buy"), caul, can't, chance, chainge, Shicago, class, cullour/cullor, cum, cumming, command, cumpany, controel, cood, coodn't, cuntry, corse, cort, cuver, dance, dainger, ded, deth, demand, discuver, doo, duz, duzn't, dun, doen't, dor, dubble, erly, erth, yther/eether, Ingland, Inglish, enuff, example, ie (for "eye"), faul, flor, floe (for "flow"), foar (for "four"), frend, frunt, fooll (for "full"), guvernment, grant, grass, greit, groope, groe, haf, haul, hed, helth, herd (for "heard"), hart (for "heart"), heven, hevy, hight, insted, iorn, jurnal, knoe, knoen, knolledge, laf, lern, Lundon, looze, luv, loe, loer, mashien, meny, mesure, munney, munth, muther, moove(ment), nyther/neether, nun (for "none"), nuthing, wunce, wun, oenly, uther, aught (for "ought"), oen, peeple, plesant, plesure, pritty, proove, pooll (for "pull"), poot (for "put"), quorter, red (for "read"), reddy, receev, remoove, roel, sault, sed, ses, shoo, shood (for "should"), shoelder, shoe (for "show"), smaul, snoe, sum(thing,times), sun (for "son"), soel, spred, strainge, tauk, taul, tair, thare, tharefore, tho, thaught, thru, tuch, tord (for "toward"), trubble, too (for "two"), waul, wont (for "want"), wor (for "war"), worm (for "warm"), wos, wosn't, wosh, Woshington, wotch, wauter, wair, wether, whot(ever), whare, hoo, hoel, hoome (for "whom"), hooze, wooman, wimmen, wunder(ful), woen't, wurd, wurk, wurld, wood (for "would"), woodn't, yoo, yung, yoor(self), yoo'r, yoothe.
And here are the words that have an unstressed vowel changed. Most unstressed vowel spellings are left alone in Regularized Inglish, but these are changed to avoid confusion.
capten, certen, certenly, mounten, forren, felloe, folloe, folloeing, narroe, tomorroe, windoe, yelloe.
From the same list, changes involving adding or removing a final e:
afternoone, childe, wilde, behinde, finde, kinde, minde, winde (verb), sine (for "sign"), moste, poaste, bothe, truthe, coole, foode, foole, moone, roofe, roome, scoole, soone, gon, Europ, ar, wer, figur, promis, purpos, minut, hav, giv, liv, nativ, believ, leav, serv, twelv, themselvs.
And finally, words with changes in consonants:
caracter, dout, gard, onnour/onnor, our (for "hour"), iland, lissen, ov, offen, scoole, shugar, shure, studdy, sugest, sunn, winn.
Here are the analogous lists for the next most frequent 1000 words:
accumpany, ahed, aincient, eny(body,wun,way), arrainge, baught, boe, boel, bred, brekfast, brest, breth, braud, bery, boosh, cam (for "calm"), cassle, chaimber, clark/clerk, curnel, cumfort(able), cupple, currage, cuzin, daingerous, discuvery, disese, duzen, ern, everywhare, exchainge, faulen, faulse, flud, foek, faught, foarth, frend(ly,ship), foolly (for "fully"), glance, gloe, guvernor, greitly, groen, groth, hunney, improove, jurney, kee, lether, luvly, luver, mashienery, magazien, ment, utherwise, oe, oener, poliece, poar, prayr, poosh, quolity, quontity, quorrel, rainge, recaul, recuver, ruff, rute, roe, scaircely, serch, seeze, shoen, sloe (for "slow"), sloely, sum(wum,whot,where), saught, sorce, suthern, steddy, strainger, thare'z, thred, thretten, thruout, throe (for "throw"), throen, tung, tresure, unknoen, wonder (for "wander"), worn (for "warn"), welth, woolf, wunn (for "won"), wurker, wurry, wurse, wurship, wurst, wurthy, woonde, yoo'd, yoo'll, yoo'v.
automobiel, curten, equol, welcum, holloe, shaddoe, sorroe.
blinde, kindely, desine, worne, noone, shoote, smoothe, troope, handsom, determin, engin, examin, imagin, purchase, favourit/favorit, immediat(ly), opposit, privat, senat, separat (adj.), havn't, activ, representativ, observ, ourselvs, preserv, reserv.
ashure, clime, Crist(ian,mas), det, onnest, leag, Lincon, sacrifise, summ, shurely, sord, Tomas, whissle.
These lists probably contain minor errors of transcription.

Regularized Inglish: theory

I see I promised to post on Regularized Inglish (RI) back in 2005, but never got around to it. Here's a brief explanation.

Axel Wijk's Regularized Inglish is a massive multi-decade job (completed in the 1950s, so there's nothing available online about it) of analyzing practically every word in the language, figuring out what the complicated rules behind the spelling system really are, and identifying all the truly irregular words and proposing properly rule-governed spellings for them. English, e.g. is truly irregular in its first vowel only, and so it becomes Inglish.

The underlying principle of RI is that every spelling shall correspond to at most a few sounds, preferably only one; multiple spellings for a single sound, however, are tolerated. Thus -ough is kept for bough, but not for rough, through, plough, hiccough, hough, or borough. Why choose bough? In order to consistently apply the RI rule that says "gh has no effect on the pronunciation of any word".

In my opinion, Wijk goes a bit far in a few places: for example, he introduces dh for the sound of th in father in other than initial positions (the, not dhe) for very little gain; he sorts out long "a" into a as in fat and "aa" as in father; he changes s to z when pronounced that way, except in the plural of nouns (not nounz)and the third-person singular ending of verbs. I wouldn't bother with any of these changes, which have little impact on being able to pronounce words at sight.

But as reformed (not revolutionized) spellings go, RI is a Great Thing.

A haiku

Quantum mechanics:
    Which moves through the springtime pond?
        Frog or a ripple?

This is post 200.

Tagsoup 1.0.4 released!

Just a bug-fix release. See tagsoup.info.

The heart of Celebes Kalossi

I'm finally ready to explain the central ideas of my object-oriented programming model Celebes Kalossi (read the linked post first for the terminology). I've been hanging fire on it for a long time over a terminological issue, which I've decided to just punt on.

In CK, there are two kinds of relationships between classes: subtyping and incorporation. Subtyping is like the relationship between a Java interface and its superinterface; incorporation is something like C++ private inheritance. These two concepts are intertwingled in various ways in various programming languages, but CK completely separates them. Every class subtypes one or more classes except for the root of the class hierarchy (if there is one); classes can incorporate zero or more other classes.

When you declare that class A subtypes class B, you mean that A (the subclass) has all the methods that are declared or defined public in B (the superclass) and all of B's superclasses, plus those declared or defined public in A itself. You are not saying that any of the definitions provided in B or its superclasses are or are not available in A, so there is no problem if a given method is public in more than one superclass. You are also suggesting that an instance of A is Liskov substitutable for an instance of B, although it is impossible to check this property mechanically. We call the public methods of A and its superclasses the interface of A.

When you declare that class C incorporates class D, on the other hand, you mean that the non-private methods defined in class D (the incorporated class) are effectively given the same definition in class C (the incorporating class). Provided that the methods in D have been properly declared in class C, they can be invoked on objects of class C just as if they had been defined as standard or public methods of class C. It does not matter if a method is defined in class D and declared in class C (which is C++ private inheritance) or vice versa. However, the fact that a method is public in class D does not make it public in class C unless it is declared public in class C or one of its superclasses: incorporation affects behavior but not interface.

For convenience, we say that a class subtypes itself and incorporates itself. Both incorporation and subtyping are transitive: class A subtypes class B's superclasses as well as class B itself, and classes incorporated by class D are implicitly incorporated by class C as well. All the methods in the incorporated classes of C are placed on an equal footing: it does not matter how they were incorporated. A loop in the subtype hierarchy specifies that all the types in the loop have the same interface; a loop in the incorporation hierarchy is ignored, so even if C incorporates D, D can also incorporate C. Note that an implementation may provided types which are outside the model: typical examples would be numeric types, strings, and exception objects.

If a method is declared in a class X or in any of the classes incorporated (directly or indirectly) by class X, it must also be defined exactly once in one of those classes. (If there are no definitions of some method, the class is abstract and must be declared as such.) If incorporating class Z would cause conflicts, CK provides for renaming or hiding unwanted methods when incorporating a class: class L can incorporate class M including specified methods (in which case all others are hidden), excluding specified methods, or renaming a method as a new name. Classes that incorporate L see the changed view.

There are no inheritance rules in CK, because there is no inheritance as such: if you want X to use the same implementation of method m as its superclass Y, then incorporate whichever class Z from which Y gets its implementation of method m. You can do the same thing for classes that are not related to you in the superclass hierarchy, or even for your subclasses if you really want to -- the model is nothing if not flexible.

I'll have another post, hopefully soon, about how CK might be implemented on the JVM or CLR.

TagSoup 1.0.3 released!

For TagSoup users among my readers, there's a new release at http://tagsoup.info, providing control of the output encoding and fixing a few bugs.

If you aren't a user but think you might like to be, TagSoup is a SAX-compliant parser written in Java that, instead of parsing well-formed or valid XML, parses HTML as it is found in the wild: poor, nasty and brutish, though quite often far from short. TagSoup is designed for people who have to process this stuff using some semblance of a rational application design. By providing a SAX interface, it allows standard XML tools to be applied to even the worst HTML. TagSoup also includes a command-line processor that reads HTML files and can generate either clean HTML or well-formed XML that is a close approximation to XHTML.

Update: TagSoup 1.0.2 had some brown-paper-bag bugs, so I've released 1.0.3 as a replacement.

The old farts have it

Age and treachery, it is said, beat youth and X every time. But what is X?

Googling suggests that this part of the proverb isn't very stable. I find: enthusiasm, skill, idealism, enthusiasm, innocence, inexperience and experience, endurance, exuberance, ability, virility, skills, reflexes, speed, talent, baggy pants, and a bad haircut.