Commercial programmers have overwhelmingly developed software using imperative languages, with a strong shift from procedural languages to object oriented languages over time. While declarative style programming has had some successes (most notably SQL), functional programming (FP) has been traditionally seen as a play-thing for academics.

FP is defined in Wikipedia as:

A programming paradigm that treats computation as the evaluation of mathematical functions and avoids state and mutable data.

Whereas an imperative language allows you to specify a sequence of actions ('do this, do that'), a functional language is written in terms of functions that transform data from one form to another. There is no explicit flow of control in a functional language.

In an imperative language variables generally refer to an address in memory, the contents of which can change (i.e. is 'mutable'). For example the rather unmathematical looking "x=x+1" is a valid expression. In FP there are no mutable variables and no state.

In an imperative language a function can return different values for the same input, either because of stored state (e.g. global or static variables) or because it is interfacing with an external device (e.g. a file, database, network or system clock). But a pure functional language always returns the same value from a function given the same input. This 'referential integrity' means an FP function call has no 'side-effects' and consequently can't interface with external devices. In other words it can't actually do anything useful - it can't even display the result of a computation to your VDU. The standard joke is that you only know a pure functional program is running because your CPU gets warmer.

The functional language Haskell works around the side-effects issue by allowing some functions to access external devices in a controlled way through 'monads'. These 'impure' functions can call 'pure' functions, but can never be called by them. This clearly separates out the pure parts of the program (without side-effects) from the impure ones (with side-effects). This means that it is possible to get many of the advantages of FP and still perform useful tasks.

FP is much closer to mathematics than imperative programming. This means that some types of problems (particularly algorithmic ones) can be expressed much more elegantly and easily as functional programs. The fact that a function has no side effects also means that it's structure is much easier to analyse automatically. Consequently there is greater potential for a computer to optimise a functional program than an imperative program. For example in FP:

y = f(x) + f(x);

Can always be rewritten as:

z = f(x);

y = 2 * z;

Saving a function call. This is more difficult to do in an imperative language, because you need to show that second call to f(x) won't return a different value to the first.

Functional programs are also inherently much easier to parallelise, due to the lack of side-effects. We can let the FP interpreter/compiler take care of parallelism. No need to worry about threads, locks, critical sections, mutexes and deadlocks. This could be very useful as processors get ever more cores. However imperative languages, with their flow of control and mutable variables, map more easily than functional languages onto the machine instruction of current (von Neumann architecture) computer. Consequently writing efficient FP interpreters and compilers is hard and still a work in progress.

Elements of FP are steadily making their way into mainstream commercial software:

• Erlang is being used in commercial systems, including telecoms switching systems.
• Microsoft Research has implemented F#, a .Net language that includes FP elements based on ML.
• Work is underway to add elements of FP to version 2.0 of the D programming language.
• Google's MapReduce is based on ideas from FP.
• The Mathematica programming language has support for FP.
• The K programming language is used in financial applications.
• The Perl 6 compiler is being written in Haskell. <insert your own sarcastic comment here>.

I recently attended ACCU 2008 which had a whole stream of talks on FP. All the FP talks I attended were packed out. That is quite something given that the audience is primarily hardcore C++ programmers. There seemed to be quite a consensus in these talks that:

• FP is starting to move out of academia and into commercial use.
• FP is more suitable than imperative style programming for some classes of problem.
• FP is not going to replace imperative programming. The bulk of commercial development will still be done in an imperative style, but with FP mixed in where appropriate.
• Hybrid languages that mix OO and FP will become more common.

I don't see Haskell replacing C++ any time soon. But I can definitely see the benefits of using FP to tackle some types of problems.

Further reading:

The Functional programming reference in Wikipedia

This article is based loosely on notes I made at ACCU 2008 from attending the following talks:

• "Caging the Effects Monster: the next decade's big challenge", Simon Peyton-Jones
• "Functional Programming Matters", Russel Winder
• "Grafting Functional Support on Top of an Imperative Language", Andrei Alexandrescu

Any mistakes are almost certainly mine.

All writings you see on "Anointed Vessel's Weblog" are the intellectual property of Yours Truly and are licensed under a Creative Commons Attribution-Noncommercial-No derivative Works 3.0 United States License

I believe in giving credit where credit is due, the term "Church Mafia" originally(as far as I can tell), can be traced to a June 22, 2003 NY Times article written by Daniel J. Wakin. In this article, Wakin explains the comparison, to the mob as a byproduct of numerous sexual abuse scandals in the Roman Catholic church. The term underwent a metamorphosis of a sort, when Pastor Jay Cameron of Urban Change Ministries produced a stage play by the name of "Church Mafia" which exposed the near occult behavior of control that is being found in a lot of churches today. This particular blog "Fitting in with a shoe horn", paints "Church Mafia's" in somewhat of a different light, by discussing the "Church Mafia" that sits in the pews. These "Pew Mafia's" exist as rogue groups that seek to destroy the fabric of well established churches through "divide and conquer tactics". Thanks again Alan Higgins for the Blog on "Church Mafia's!"

via http://jaortega.wordpress.com/2007/04/11/erlang-now/

and now for something completely different:

g( . )( . )gle is watching you...

But these URIs are not atomic, they are merely treated as such in their syntactic expression. They are what is popularly known as a microformat.

So what is a microformat anyway?

A microformat is a relative term in that thing that has a format is not a microformat in and of itself, but once that format becomes injected into an atom of another format, it becomes a microformat. That is, once a format uses other formats inside itself, we can begin to talk about microformats.

The canonical example for this is XML and URIs.

<Person email="mailto:bent@example.com"/>

As we see here, the email attribute contains structured information, but the structure is not expressed using structured XML, instead it is simply concatenated inside an attribute.

This means essentially that what we are doing is to inject a domain-specific language, in its own syntax, within a domain-neutral container format - XML. It would be possible to actually abandon the concrete URI syntax and encode the abstract URI syntax into the XML Information Set datamodel directly!

This would have the consequence that the email property in our above example wil have to be reformulated as an element instead; something like

<Person>
<email>
<URI>
<scheme>mailto</scheme>
.....
</URI>
</email>
</Person>

It is worth noticing how the compact concrete URI syntax makes the markup shorter and perhaps easier to read, but it also means we loose one of the benefits of XML - its ability to encode information genercally using structured markup so we can more easily parse the content.

It is clear to me that the reason XML is not used in a more structured way and we have all the microformat mess is simply because XML is a human-readable verbose format. If it were to become more verbose, it would become less readable - to humans.

The upcomming Infoset encoding, the Efficient XML Interchange format (EXI) may change this game and move us away from the microformat game because it will then be able to create a compact and uniformly structured encoding of URIs inside the other data.

Now the URI may not be the ideal example of this, but as a very used identifier string (which is useful of course) and as a structured value as well, it does have some usefulness as an example in this regard.

The SVG path attribute is a better example. As machines are supposed to generate SVG, we should not care about human-readability - if we want readability we create interactive visualizations of the data (text or a drawing).

The Resource Description Framework (RDF) from the W3C has a concept which is somewhat related to this concept of unification: Reification.

In RDF one can create statements about statements. Normally a statement is the quintessential "primitive" of RDF, but what if you want to say something about a statement; you can choose to express the statement via other statements, thereby reifying it.

I'm not saying reification and unification as shown here is always the right thing to do, but I think the world should move more away from microformats and towards unified abstract data models. If the URI is parsed into XML Information Items, then the job of the URI constructor is trivial.

As an identfier a URI may still be valuable in its canonical concrete form, but software could also create this concrete form from the structured form if necessary. It's more cumbersome the other way around.

Should you want to create an XML or RDFS/OWL datamodel for URIs, here's the concrete syntax for them, expressed in Backus-Naur form.

Appendix A. Collected ABNF for URI

URI = scheme ":" hier-part [ "?" query ] [ "#" fragment ]

hier-part = "//" authority path-abempty
/ path-absolute
/ path-rootless
/ path-empty

URI-reference = URI / relative-ref

absolute-URI = scheme ":" hier-part [ "?" query ]

relative-ref = relative-part [ "?" query ] [ "#" fragment ]

relative-part = "//" authority path-abempty
/ path-absolute
/ path-noscheme
/ path-empty

scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )

authority = [ userinfo "@" ] host [ ":" port ]
userinfo = *( unreserved / pct-encoded / sub-delims / ":" )
host = IP-literal / IPv4address / reg-name
port = *DIGIT

IP-literal = "[" ( IPv6address / IPvFuture ) "]"

IPvFuture = "v" 1*HEXDIG "." 1*( unreserved / sub-delims / ":" )

IPv6address = 6( h16 ":" ) ls32
/ "::" 5( h16 ":" ) ls32
/ [ h16 ] "::" 4( h16 ":" ) ls32
/ [ *1( h16 ":" ) h16 ] "::" 3( h16 ":" ) ls32
/ [ *2( h16 ":" ) h16 ] "::" 2( h16 ":" ) ls32
/ [ *3( h16 ":" ) h16 ] "::" h16 ":" ls32
/ [ *4( h16 ":" ) h16 ] "::" ls32
/ [ *5( h16 ":" ) h16 ] "::" h16
/ [ *6( h16 ":" ) h16 ] "::"

h16 = 1*4HEXDIG
ls32 = ( h16 ":" h16 ) / IPv4address
IPv4address = dec-octet "." dec-octet "." dec-octet "." dec-octet

dec-octet = DIGIT ; 0-9
/ %x31-39 DIGIT ; 10-99
/ "1" 2DIGIT ; 100-199
/ "2" %x30-34 DIGIT ; 200-249
/ "25" %x30-35 ; 250-255

reg-name = *( unreserved / pct-encoded / sub-delims )

path = path-abempty ; begins with "/" or is empty
/ path-absolute ; begins with "/" but not "//"
/ path-noscheme ; begins with a non-colon segment
/ path-rootless ; begins with a segment
/ path-empty ; zero characters

path-abempty = *( "/" segment )
path-absolute = "/" [ segment-nz *( "/" segment ) ]
path-noscheme = segment-nz-nc *( "/" segment )
path-rootless = segment-nz *( "/" segment )
path-empty = 0

segment = *pchar
segment-nz = 1*pchar
segment-nz-nc = 1*( unreserved / pct-encoded / sub-delims / "@" )
; non-zero-length segment without any colon ":"

pchar = unreserved / pct-encoded / sub-delims / ":" / "@"

query = *( pchar / "/" / "?" )

fragment = *( pchar / "/" / "?" )

pct-encoded = "%" HEXDIG HEXDIG

unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~"
reserved = gen-delims / sub-delims
gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@"
sub-delims = "!" / "$" / "&" / "'" / "(" / ")"
/ "*" / "+" / "," / ";" / "="

See also

• Towards Useful Names
• Knowledge Representation Series - 1

It allows one to create new items of certain types: links, books, documents, videos, contacts, etc. But it is also more than that. In the extreme it can subsume blogs, homepages, wikis and to some extent search engines.

As opposed to a regular bookmark a twine item has a type and semantics associated with it.

I have some suggestions for improving Twine further, though

1. A twine item type; thus twines can become sets of elements from other twines (unions)
2. A twine item type designer; establish an item type ecosystem; with subtype reuse - create exponential growth (but constrain and sanity-check using rules)
3. A twine item type visual designer; create a visual template for an item type
4. A distinction between "my twines", "my collaborative twines" and "other twines"; menus or submenus for each, making navigation easier; and the twinelet must take this distinction into account
5. A set of twine maps; visualizations of twine; heat maps (activity), geo maps (earth), topic maps (hot topics), taxonomies, etc; create mapping dimensions that slice twine in various ways of interest
6. A set of twine layouts; twine is a bit boring in its layout; create new views that allow one to display twines and items in grid layout
7. A blog item type
8. A mass bookmark import option and twine distribution based on analysis (see 11)
9. A set of dynamic layouts - use RIA technology (see 6,10)
10. A thin desktop application (Adobe AIR) - drag n drop, etc; intertwine twine with the personal computer/desktop; RIA applications respond faster and look better (drive in users in the millions)
11. A natural language parsing and indexing of items (Powerset secret sauce)
12. A broader focus on user-interaction; how might users interact and communicate
13. A statistics view; let users track how their items are growing and spreading
14. A roadmap or at least some public information about where twine is going for us beta chasers
15. A twine connection item type (do we have that?)
16. A geographical and temporal map of items (with type selection) and time slicing and motion (requires "RIA" technology); users as well (see 15)

Invest heavily in at least some of these goals; and do not forget the user-experience - make the users time to find and browse information a key metric for success.

There. Some of my thoughts on what Twine lacks to be truly compelling. There are more ideas, but they'll come...

There are many protocols for finding and identifying things in the world. HTTP, FTP, P2P, etc. The principle way of identifying things on the Internet is by using Uniform Resource Identifiers (URI) or Internationalized Resource Identifiers (IRI). These have schemes defined, typically for various representation access protocols such as HTTP, FTP and such. There are also representation independent schemes such as the URN subspace of URIs.

The URN subspace can be used to express things detached from a particular access protocol. For example one might say

urn:isbn:<my book isbn number>
urn:ssn:<my social security number>

The two above URI/URN schemes can be used to find information regardless of protocol.

A more precise way to find information is to uniquely name the information representation. This is what hash functions do. They map between some sequence of bits and some hash code of lesser length but with sufficient length to reasonably ensure uniqueness in the real world.

Peer to Peer (P2P) networks use hash codes to uniquely name representations as it would otherwise be near impossible to create a coherent representation by downloading from multiple sources at once, simply because each file (and each part of the file) could be named arbitrarily. What data does "my book name" refer to? What if two books are called the same? Sure, we could use ISBN's, but they are bound to books and so not generic.

By coupling URIs and hash codes, we can uniquely identify data.

urn:sha1:<my hash code for my file>

Hash codes can also be used to conceil information by obscuring the real identity of it to anyone other than the owner. For example

urn:[field:ssn],[sha1:<hash of ssn>]
urn:[field:password],[sha1:<hash of password>]

The benefit?

Apart of obscurement, it also has the natural benefit of relative uniqueness and compression, as the name is shorter than the bits it represents. It may be seen as lossy compression for the purpose of naming - we can compress any data to the point where we mostly can't reverse engineer the data, but we can reasonably trust the uniqueness of the name with respect to the data it identifies.

This is a crucial benefit for the Semantic Web. It  enables one to say things about other data, without binding oneself to a particular access protocol or mechanism and yet future search engines can use hash codes to find any data on any namespace it knows of, including ones invented in the future.

In other words: it's future proof. - To a point.

The precise hashing mechanism may change over time, but the benefit of hashing is that one can still identify the hashing mechanism via name.

One might even use a hash code to refer to the implementation used to hash

urn:data:{class:sha1},{implementation:hash:<hash>},{identification:hash:a<hash>}

The hash code of the implementation would be hashed using the implementation itself of course! The only problem here is that the exact identity of this implementation is dependent on the implementation itself - to some extent.

I'm somewhat disguisted by names such as

http://www.w3.org/1999/02/22-rdf-syntax-ns#type

The idea of this HTTP URI is that it can be used to fetch the definition of this term via the protocol used to express the name of the term. That's beneficial of course. But it also ties the name to the current HTTP protocol. It's only disguisting in retrospect and not completely so.

urn:sha1:<hash>

Then for each protocol, a search engine should be used to look up the protocol specific address of that data - or indeed even the latest cached representation.

A potential problem, is of course that the expression of the meaning of the term itself, is dependent on some representation language such as RDF/XML. But this is to be expected and hopefully RDF/XML (or better: RDF/EXI (or better still: A triple variant of RDF expressed in EXI)) will long outlive RDF/XML.

Of course old names can be mapped to new names via semantics. So one can say

<urn:data:sha1:<hash>> <same-as/hash-of> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>

Search engines of the future may also be used to find and transcode information - even to transparently map the old web to the new web so it's possible to switch from HTTP to some other protocol more or less seamlessly (ahemn).

So for future proofing data names, there can be only one: Use hash codes, but there are ways to deal with oldschool names.

Lang.Net Conference 2008
3-07 - Modeling and Languages - Don Box

The talk is about representation independence (oxymoron?), declarative forms and abstractions. It has a historic point of view, showing how abstraction and modelling has been applied over time.

But he is careful not to really talk about anything concrete with respect to what Microsoft is doing now. The talk is done in a slideshow driven by a domain-specific XAML for slideshows which is then transformed into a WPF model that is easy on the eyes.

That in and of itself is nothing special. It is common to encode data in XML, transform it with XSLT into HTML plus CSS etc.

To me the talk has a certain "Intentional" smell to it. I'm guessing we may finally see Microsoft move towards a model where we have domain-specific representations, domain-specific presentations and domain-specific input models.

The "two beers" test (as Don calls it) is not passed because the slideshow is encoded in a CLR specific XML dialect (XAML). If there was a DSL editor for it, it would be, not to say idiotproof, but domain-proof, and with all noise absent.

I wonder if "Avalon" (WPF) could play a role as a code presentation model and interaction model in the future. But it wouldn't be the first time we'd see a new framework being invented.

Don also talks about search. That leads more towards a type-aware filesystem which again leads me to think of WinFS. It's time may still come. Maybe in an even more sophisticated semantic form based on semantic web principles and technologies.

The paradigm for the presentation.

models
presentations
representations
interactions
filesystems
searches

concept: [model]; domain-specific expressions.

concept: [representation]; text, markup, schemas, etc.

concept: [presentation]; "sensual" representations (see, hear, etc)

concept: [interaction]; I/O - put information into the model and get it out of the model - "via" the presentation layer

concept: [filesystem]; A type-aware filesystem; files are not atomic, files are type instances.

concept: [search]; A type-aware filesystem is very searchable

In simple terms I believe Microsoft is working on Intentional programming things with domain-specific models and domain-specific code representation, visualization and editing.

That's what I hope, at least... It'll also tie in very nicely with Microsofts framework and tool investments.

What does it do?

It takes a set of template elements, contained in the DOM tree of the current document decorated with marker classes, and expands it with some data.

The data is given as a combination of Javascript objects and arrays. An object's key selects the DOM Element for which the corresponding value supplies the data.

An array value means the selected elements should be repeated and expanded using the elements of the array.

How does it work?

The data object is processed recursively. If an associative array is found, the first element of the template with the marker class given as the key of the association is selected and both the template and the data are processed deeper from there.

If the value of the association is an array, the elements selected by the key is repeated and filled with the elements of the array recursively.

Whenever a primitive value (ie, a number, or a string) is found, the currently selected element's text is set accordingly, and recursion stops.

Examine the following example.

[sourcecode language="html"]

[/sourcecode]

Here we have a HTML template described under the div 'template'. We intend to place an expanded version of it into 'testTarget'. See how it is done.

[sourcecode language='jscript']
var data = {
'columnName': ['Name', 'Price', 'Qty'],

'data': [
{'item': ['1. Apple', 125, 0.5]},
{'item': ['2. Banana', 210, 0.4]},
{'item': ['3. Cat', 300, 0.2]},
{'item': ['4. Dog', 200, 3]}
]
};

expandTemplate('testTemplate', 'testTarget', data);[/sourcecode]

Let's se what we have here.

• the 'columnName' array is there to instruct the engine to repeat the <th class="columnName" /> three times and set them up so they say 'Name', 'Price' and 'Qty'.
• the 'data' array says we need four copies of the <tr class="data">...</tr> and everything contained within
• each of the 'item' arrays says we need 3 instances of <td class="item" /> in each <tr class="data"> -s, and they should be filled with whatever is in the arrays.

The resulting HTML looks like this:

[sourcecode language="html"]

[/sourcecode]

Note that there are multiple trs having the marker class data. These are repeated in alternating succession, and this is the preferred way to expand table templates with that cool zebra look. ;)

AJAX

Yeah, Ajax is supported in a rather trivial manner. We have the shortcut method tmpl, which performs an Ajax request, interprets the resulting text as the Json representation of a data object, and expands the given template using it. It even fires the onComplete event afterwards!

E.g. the template

[sourcecode language="html"]

[/sourcecode]

And the js:

[sourcecode language="jscript"]
tmpl('template', 'target', 'primes.php', {
data: {
from:2,
to:20
}
});[/sourcecode]

Imagine we have a primes.php that returns the primes between 'from' and 'to' as a Json object that looks like this:

[sourcecode language="jscript"]
{
title:"List of some primes",
item:[2,3,5,7,11,13,17,19]
}[/sourcecode]

Then we have the following result:

[sourcecode language="html"]

[/sourcecode]

There you have it, a zebra colored list of primes. Just as advertised.

Known limitations

Plain old data is processed using Element.setText, and that means no attributes can be set currently. Take the following example.

[sourcecode language="html"]

[/sourcecode]

It would be nice if the src, alt and title attributes of the img -s could be expanded like this:

[sourcecode language="html"]

[/sourcecode]

But there is no way for such a thing right now. I have yet to figure out a solution. Comments are indeed welcome :)

Summary

• as far as I know, the engine described above takes a rather novel approach, and one that meets my needs better than anything I have seen in this field to date
• first of all, there is no custom language with ifs and fors and whatnot. This also implies there is need for a costly parser for a tiny sublanguage. Besides, in my opinion a template is no place for any control logic. Or any logic for that matter. It is far too late to process the data when it finally comes to template expansion!
• the 'language' describing the template is highly declarative. Again, I don't think a template language is for them imperative language constructs.
• I have yet to come up with a way to set attributes in a template. Something like having data objects like {'src':'image.jpg', 'alt':'An image...', 'styles': {'height':'150px', 'border':'1px solid black'}} and calling Element.setProperties on them might do the trick, dunno... :/
• There should be events before and after a leaf of the template is expanded, providing some means to prevent further expansion and there should be an event after expansion is complete.
• Stay tuned for a PHP and an ASP.NET version ;)

Obtaining

Download from here, change extension to js: tmpl.js