Kodenya dibuat masih belum dioptimasi, struktur proses yang direalisasi dibuat menjadi dua kelas yaitu LISPLexer dan LISPParser sedangkan struktur yang menampung representasi list-nya sendiri menjadi satu kelas LISPNode. OK, let's jump to the code. kita mulai dari LISPNode.

TLISPNode
List di LISP sebetulnya dinotasikan sebagai S-Expression. jadi List seperti ini

(a b)
(a b c)
(a (b))

sebetulnya dalam notasi S-expression akan menjadi seperti ini

(a.(b.NIL))
(a.(b.(c.NIL)))
(a.((b.NIL).NIL))

setiap pasangan (a.t) merupakan sebuah simpul dengan a adalah head dan t adalah tail. Setiap simpul dapat merupakan atom/nilai atomik (number, symbol, string) atau list (pointer ke s-expression lainnya, seperti pada baris ke-tiga). Berikut ini deklarasi kelas TLISPNode.

type
  TLISPNodeType = (ntAtom, ntList);
  TLISPAtomType = (atSymbol, atString, atNumber);
  TLISPNode = class
  protected
    vtype     : TLISPNodeType;
    Fatomtype  : TLISPAtomType;
    atomval   : string;
    Fchild : TLISPNode;
    Fnext : TLISPNode;
  public
    constructor Create;
    destructor Free;
    function ToString:string;
    function Clone : TLISPNode;
    function GetHead : TLISPNode;
    function GetTail : TLISPNode;
    property Child : TLispNode read FChild write FChild;
    property Next : TLispNode read FNext write FNext;
    property Value : string read atomval write atomval;
    property NodeType : TLISPNodeType read vtype write vtype;
    property AtomType : TLISPAtomType read FAtomType write FAtomType;
  end;

Keterkaitan antar simpul dibuat sebagai list satu arah yang menunjuk ke sibling dan child (jika simpul tersebut bukan atom).

TLISPLexer
kelas LISPLexer berfungsi untuk melakukan pemisahan string menjadi bagian-bagian yang diberi label (token). Pemisahan dan pemberian label ini disebut juga sebagai analisis leksikal. Hal yang dilakukan sebetulnya dapat dibilang parsing, namun tujuannya lebih sederhana (tanpa pohon sintaks yang rumit). Kode interface bagian Lexer ditunjukkan dalam potongan kode berikut.

const
  HTAB = #9;
  CRLF = #10#13;
  LBRA = '(';
  RBRA = ')';
  DQUO = '"';
  S_ALPHA = ['a'..'z','A'..'Z'];
  S_DIGIT = ['0'..'9'];
  S_ALNUM = S_ALPHA+S_DIGIT;
  S_WHITE = [' ',HTAB,#10,#13];

type
  TTokenType = (ttString, ttSymbol, ttNumber, ttLBRACKET, ttRBRACKET);
  TToken = record
    tokentype : TTokenType;
    sym : string;
  end;

  TLispLexer = class
  private
    function GetCount: integer;
  protected
    FToken : array of TToken;
    FPita : string;
    FIdx : integer;
    function GetToken(tid : integer):TToken;
    procedure Proses;
    procedure skipwhite;
    function iswhite(c:char):boolean;
    function isalpha(c:char):boolean;
    function isalnum(c:char):boolean;
    function isdigit(c:char):boolean;
    procedure SetPita(const Value: string);
    procedure add_token(toktype: TTokenType; tokval:string);
  public
    constructor Create;overload;
    constructor Create(inputstring: string);overload;
    destructor Free;
    procedure Clear;
    function dumptoken(t:ttoken):string;
    function dump:string;
    property Token[tid : integer]: TToken read GetToken;
    property Pita: string read FPita write SetPita;
    property Count : integer read GetCount;
  end;

Tipe token untuk LISP untungnya cukup sederhana, karena pemisah antar 'kata'-nya hanya whitespace dan tanda kurung '(' dan ')'. Bagian yang perlu diperlihatkan di sini adalah di prosedur proses.

procedure TLispLexer.Proses;
var
  cur_kata : string;
  cc : char;
  cur_token : TTokenType;
begin
  if length(FPita)=0 then exit;
  FIdx := 1;
  cur_kata := '';
  cur_token := ttLBRACKET;
  skipwhite;
  while (FIdx <= length(FPita)) do begin
    cc := FPita[FIdx];
    case cc of
      LBRA:add_token(ttLBRACKET, cc);
      RBRA:add_token(ttRBRACKET, cc);
      DQUO:begin
        inc(FIdx);
        cur_kata := '';
        while (FIdx <= length(FPita)) and (FPita[FIdx] <> DQUO) do begin
          cur_kata := cur_kata + FPita[FIdx];
          inc(FIdx);
        end;
        add_token(ttString, cur_kata);
      end;
      'a'..'z','A'..'Z':begin
        cur_kata := '';
        while (FIdx <= length(FPita)) and (FPita[FIdx] in S_ALNUM+['-']) do begin
          cur_kata := cur_kata + FPita[FIdx];
          inc(FIdx);
        end;
        dec(FIdx);
        add_token(ttSymbol, cur_kata);
      end;
      '0'..'9','-':begin
        cur_kata := '';
        while (FIdx <= length(FPita)) and (FPita[FIdx] in S_DIGIT+['-','e','E','.']) do begin
          cur_kata := cur_kata + FPita[FIdx];
          inc(FIdx);
        end;
        dec(FIdx);
        add_token(ttNumber, cur_kata);
      end;
    end;
    inc(FIdx);
    skipwhite;
  end;
end;

TLISPParser
Setelah string karakter dipisahkan dan diberi label menjadi list token, maka proses selanjutnya adalah menterjemahkan deretan token ini menjadi struktur list yang sesuai. Berikut ini deklarasi kelas LISPParser. Method utama yang diperlukan adalah fungsi ParseNode yang diantar oleh fungsi umum Parse.

type
  TLISPParser = class
  protected
    FLexer : TLISPLexer;
    FTokenPtr : integer;
    function GetToken:TToken;
    function ParseNode:TLISPNode;
    procedure NextToken;
  public
    function Parse(LISPString:string):TLISPNode;
  end;

Fungsi Parse merupakan antarmuka yang melakukan persiapan dan memanggil proses analisis leksikal sebelum diserahkan ke fungsi ParseNode yang berjalan secara rekursif.

function TLISPParser.Parse(LISPString:string): TLISPNode;
begin
  result := nil;
  FLexer := LISP_Lexer;
  FLexer.Clear;
  FLexer.SetPita(LISPString);
  FTokenPtr := 0;
  if (FLexer.Count <> 0) and (GetToken.tokentype=ttLBRACKET) then begin
    NextToken;
    result := ParseNode;
  end;
end;

function TLISPParser.ParseNode: TLISPNode;
var
  tmp : TLISPNode;
  t : TToken;
begin
  result := nil;
  { proses car }
  t := GetToken;
  case t.tokentype of
    { this is an empty list }
    ttRBRACKET:exit;

    { car is an atom }
    ttSYMBOL, ttSTRING, ttNumber:begin
      { check if nil }
      if t.sym = '' then exit;
      { normal atom value }
      result := TLISPNode.Create;
      result.NodeType := ntAtom;
      case t.tokentype of
        ttSYMBOL:result.atomtype := atSymbol;
        ttString:result.atomtype := atString;
        ttNumber:result.atomtype := atNumber;
      end;
      result.Value := t.sym;
    end;

    { car is a list }
    ttLBRACKET:begin
      result := TLISPNode.Create;
      result.NodeType := ntList;
      NextToken;
      { parse child }
      tmp := ParseNode;
      { configure node }
      result.Child := tmp;
    end;

  end; { case..of }

  { proses next }
  NextToken;
  if FTokenPtr < FLexer.Count then begin
    t := GetToken;
    tmp := ParseNode;
    result.Next := tmp;
  end;
end;

Untuk yang penasaran dengan keseluruhan kodenya bisa diunduh di sini.

Если есть BNF-подобное описание грамматики без левой рекурсии, то по её нетерминалам легко пишутся функции их разбора. Каждая такая функция является парсером и имеет тип (здесь и далее используется язык Python) Sequence(a) -> (b, Sequence(a)), где a — тип токенов (например, банальных str), b — тип результата разбора (в общем случае object), а Sequence — тип любой последовательности (вот один из вариантов того, что можно считать последовательностью). Для таких функций-парсеров можно ввести очень удобные комбинаторы типа альтернативы, следования, множественного применения. Вообще, это очень показательный пример преимуществ функционального программирования.

Используя этот метод, я написал парсер JSON на Python на основе грамматики из RFC 4627. Известны проблемы парсеров JSON на Python, но я строго следовал грамматике RFC, так что в лексике и синтаксисе ошибок быть не должно. Приведу исходный код. Вначале часть, которую можно было бы сделать библиотекой: несколько удобных комбинаторов, взятых из книги Харрисона, а также объектная обёртка комбинаторов для перегрузки операторов Python ради красоты:

class NoParse(Exception):
    def __init__(self, tokens, msg=None):
        self.tokens = tokens
        self.msg = msg

    def __str__(self):
        s = 'cannot parse input'
        e = 'unparsed data remains:\n%s' % self.tokens
        msg = ' %s' % self.msg if self.msg is not None else ''
        return '%s:%s\n%s' % (s, msg, e)

class Parser(object):
    def __init__(self, p):
        self.wrapped = p

    def __call__(self, tokens):
        return self.wrapped(tokens)

    def __add__(self, other):
        @Parser
        def f(tokens):
            v1, r1 = self(tokens)
            v2, r2 = other(r1)
            return (v1, v2), r2
        return f

    def __or__(self, other):
        @Parser
        def f(tokens):
            try:
                return self(tokens)
            except NoParse:
                return other(tokens)
        return f

    def __rshift__(self, treatment):
        @Parser
        def f(tokens):
            v, r = self(tokens)
            return treatment(v), r
        return f

@Parser
def finished(tokens):
    if len(tokens) == 0:
        return None, tokens
    else:
        raise NoParse(tokens, 'should have reached eof')

def many(p):
    @Parser
    def f(tokens):
        try:
            v, next = p(tokens)
            vs, rest = many(p)(next)
            return [v] + vs, rest
        except NoParse:
            return [], tokens
    return f

def some(predicate):
    @Parser
    def f(tokens):
        if len(tokens) == 0:
            raise NoParse(tokens, 'no tokens left in the stream')
        else:
            t, ts = tokens[0], tokens[1:]
            if predicate(t):
                return t, ts
            else:
                raise NoParse(tokens, 'token "%s" doesn\'t match against the predicate' % t)
    return f

def a(value):
    return some(lambda t: t == value)

def several(predicate):
    return many(some(predicate))

А теперь код самого парсера JSON. Я не хитрил и реализовал разбор «в лоб», просто следуя RFC. Как я уже писал, код всё равно получается прозрачный и краткий:

def json_loads(s):
    'str -> object'
    head = lambda (x, xs): x
    from operator import add
    word = lambda w: reduce(add, [a(c) for c in w])

    # RFC 4627 productions, nearly 1-to-1
    ws = several(lambda c: c in ' \t\n\r')
    begin_array = ws + a('[') + ws
    end_array = ws + a(']') + ws
    begin_object = ws + a('{') + ws
    end_object = ws + a('}') + ws
    name_separator = ws + a(':') + ws
    value_separator = ws + a(',') + ws
    false = (word('false')
        >> (lambda _: False))
    true = (word('true')
        >> (lambda _: True))
    null = (word('null')
        >> (lambda _: None))
    unescaped = some(lambda c: any(min <= ord(c) <= max
        for min, max in [(0x20, 0x21), (0x23, 0x5b), (0x5d, 0x10ffff)]))
    escape = a('\\')
    quotation_mark = a('"')
    hexdig = some(lambda c: c in '0123456789ABCDEF')
    char = (
          unescaped
        | ((escape + (
              a('"')
            | a('\\')
            | a('/')
            | a(unichr(0x0008)) # backspace
            | a(unichr(0x000c)) # form feed
            | a('\n')
            | a('\r')
            | a('\t')
            | (a('u') + hexdig + hexdig + hexdig + hexdig # uXXXX
                >> (lambda ((((_, d1), d2), d3), d4):
                   (unichr(int('%s%s%s%s' % (d1, d2, d3, d4), 16)))))
        )) >> (lambda (_, v): v))
    )
    string = (quotation_mark + many(char) + quotation_mark
        >> (lambda ((_1, cs), _2): ''.join(cs)))
    minus = a('-')
    plus = a('+')
    e = a('e') | a('E')
    decimal_point = a('.')
    digit1_9 = some(lambda c: c in '123456789')
    zero = a('0')
    digit = digit1_9 | zero
    integer = (
          zero
        | (digit1_9 + many(digit)
            >> (lambda (x, xs): ''.join([x] + xs))))
    frac = (decimal_point + digit + many(digit)
        >> (lambda ((_, x), xs): ''.join([x] + xs)))
    exp = (
          (e + minus + digit + many(digit)
            >> (lambda (((_1, _2), x), xs): '-%s' % ''.join([x] + xs)))
        | (e + plus + digit + many(digit)
            >> (lambda (((_1, _2), x), xs): ''.join([x] + xs)))
    )
    number = (
          (minus + integer + frac + exp
            >> (lambda (((_, x), f), e): float('-%s.%se%s' % (x, f, e))))
        | (minus + integer + frac
            >> (lambda ((_, x), f): float('-%s.%s' % (x, f))))
        | (minus + integer + exp
            >> (lambda ((_, x), e): float('-%se%s' % (x, e))))
        | (minus + integer
            >> (lambda (_, x): int('-%s' % x)))
        | (integer + frac + exp
            >> (lambda ((x, f), e): float('%s.%se%s' % (x, f, e))))
        | (integer + frac
            >> (lambda (x, f): float('%s.%s' % (x, f))))
        | integer
            >> (lambda x: int(x))
    )
    @Parser
    def value(tokens):
        return (
            false | null | true | object | array | number | string
        )(tokens)
    array = (
          (begin_array + value + many(value_separator + value) + end_array
            >> (lambda (((_1, x), vs), _2):
               ([x] + [v for _, v in vs])))
        | (begin_array + end_array
            >> (lambda _: []))
    )
    member = (string + name_separator + value
        >> (lambda ((k, _), v): (k, v)))
    object = (
          (begin_object + member + many(value_separator + member) + end_object
            >> (lambda (((_1, x), ms), _2):
               (dict([x] + [m for _, m in ms]))))
        | (begin_object + end_object
            >> (lambda _: {}))
    )
    json_text = (object | array) + finished >> head
    return head(json_text(s))

Имея готовую библиотечную часть парсера и грамматику языка BNF, можно довольно легко писать парсеры для разных языков.

Несколько замечаний об использовании:

• В грамматике RFC 4627 лексемы отдельно не описываются, так что я лексический анализ как стадию не выделил. Но вообще парсить поток токенов с их типом, значением и позицией в потоке удобнее
• Если в грамматике есть левая рекурсия, то её нужно устранить, т. к. это LL-парсер
• За счёт возврата при переборе парсер может заглядывать на сколько угодно токенов вперёд, то есть это LL(*)
• При использовании в качестве входа функций разбора обычных строк без состояния нельзя установить место, в котором произошла ошибка. К тому же весь разбираемый текст надо держать в памяти. Введение состояния портит красоту и немного усложняет код. Мне кажется, таких проблем можно избежать в монадических парсерах

Segway Tours, die im Olympiapark durchgeführt werden. Ich denk, das sollte ich auch mal machen.

Übrigens, die 9 Mio Webseiten sind doch noch nicht online. Der Aufbau der Datenbank hat mehr als 24 Stunden gedauert, so dass der Job zum Aufbau der Datenbank, den ich von daheim gestartet habe, terminiert wurde als die 24h-Zwangstrennung durchgeführt wurde. Also nochmal mit nohup...

Und jetzt sitze ich wieder an der Doktorarbeit und brauche zur Umsetzung einer Idee möglichst sofort (und eigentlich ja auch schon gestern - um den Spruch mal wieder zu bringen) einen C++ und Java-Parser. Und wenn möglich auch noch einen Parser für Ruby und Python. Eine kurze Suche mit Google (war eigentlich nur der Presse das Wort "googeln" verboten wurden?) nach einem Parser für C++ hat nur Müll hervorgebracht. Und der Code vom GCC würde ebenfalls keine mir bekannten Coding Guides/Styles erfüllen. Eigene Parser schreiben für jede der vier Programmiersprachen ist sicherlich zu aufwendig. Also irgendwas muss es doch geben. Wo sind denn nur die Studenten wenn man sie braucht, die sowas sonst in Form von Diplomarbeiten, Bachelor- oder Masterarbeiten oder irgendwelchen Studentenprojekten erledigen.

Nachtrag: Aha! Man muss wohl nur g'scheit googeln. Anstatt nach einen Parser für C++ zu suchen, hätte ich wohl gleich nach "parse java code" suchen sollen. Anscheinend bietet Antlr die Funktionalität, die ich benötige. Mit einer ewig langen Liste von Grammatiken z.B. auch für C++, Java und Pyhton. Ma gucken, ob's was taugt...

Now I just have to finish the Server-side version of the BBCode parser. I'm trying out a few ideas and hopefully we'll have a working model here soon

Hopefully I'll be posting again/editing this post here soon with updates.

Cheers,

-Jhecht

2. I haven't seen a lot of innovation in user interfaces for help desk applications, I wonder if creating a riff on the basic Getting Thing Done UIs I have seen would work and be helpful.

3. With all the unstructured data out there I wonder if a simple graphical tool designed to turn this data into actionable information would be helpful?  

What I am thinking is a visual environment with drag-and-drop parsing workflow creation where one could use predefined rules (or create their own rules) that parses a document extracting the information needed in a structured format 

So for instance you could create a "parser" for a series of emails that always have the essential same structure and get that data into a dB.

Hmm, maybe I need to think more about this, it may actually be quite useful...

4. Another interesting blog post today on /Message.  Stowe is right, at every turn and with every interaction we are exposing ourselves to loosely coupled, information rich data flows and we need to be able to mine these flows to extract actionable information otherwise we are stuck with static silos. Information is out there, we just need better means of extracting it from the crushing weight of all the data we see every day.  

5. Another OpenID mechanism.  From the site "Emailtoid is a simple mapping service that enables the use of email addresses as OpenID identifiers."

Update: Sorry, repository cut off by the administrator. I'll inform you when the sources are back.

Note rapide pour vous informer que le code source d'Extrapol est maintenant disponible au public. Il ne s'agit pas encore d'une version officielle -- en particulier, le code n'a pas encore de licence et il manque des bibliothèques (disponibles ailleurs). Plus de détails dès qu'une version officielle est disponible.

Additif: Désolé, je viens d'apprendre que le dépôt de source a été isolé par l'administrateur. Je vous tiendrai au courant dès que le code source est de nouveau public.

Veja que beleza:
[sourcecode language='python']# coding: utf-8
# 2008-06-11 AF
# crawler.py

import urllib, httplib

class crawler:
data = ''
host = None
method = 'GET'
params = {}
headers = {}
request = '/'

def __call__(self, fetch_anyway=False):
if not self.host:
raise ValueError('You must provide at least the host')

if not self.data:
# make the connection
fd = httplib.HTTPConnection(self.host)

# set the params and send the request
params = urllib.urlencode(self.params)
fd.request(self.method, self.request, params, self.headers)

# get the response
response = fd.getresponse()
self.status, self.reason = response.status, response.reason

# read the data
if self.status == 200 or fetch_anyway is True:
self.data = response.read()

fd.close()

return self.data[/sourcecode]
Basicamente, se eu criar uma classe que herda dela e especificar apenas o host ela, já funciona. Depois de todo o trabalho, resolvi fazer um programa relativamente simples que extrai o desempenho dos clubes do Campeonato Brasileiro 2008, usando o site da Gazeta Esportiva.

Pra interpretar o conteúdo de maneira bem fácil e rápida, usei o Beautiful Soup. Apesar do meu Palmeiras não estar bem, com esse papo furado ai do Luxa, é legal ver a lista de desempenhos - especialmente por não ter o Corinthians nela, ainda mais depois da derrota de hoje contra o Sport. :)

Aí vai o programa (cheio de comentários):
[sourcecode language='python']#!/usr/bin/env python
# coding: utf-8

import re, sys
from crawler import crawler
from BeautifulSoup import BeautifulSoup

class fetch(crawler):
host = 'www.gazetaesportiva.net'
request = '/campeonatos/futebol/nacional/2008/brasileirao/conteudo/desempenho.php'

if __name__ == '__main__':
# cria o crawler
doc = fetch()

# busca o documento
buffer = doc()
if not buffer:
print doc.status, doc.reason
sys.exit(1)

# armazena as listas de times e desempenho nas rodadas
name_list = []
data_list = []

# cria o soup, já convertendo as entidades HTML pra
# caracteres unicode: aacute vira "á"
soup = BeautifulSoup(buffer, convertEntities='html')

# encontra os nomes dos times
name = soup.findAll('strong')

# depois de encontrar os nomes, compilamos essa
# expressão regular que irá remover as tags
# NOTA: talvez já tenha isso no BS e eu não sei :p
junk = re.compile('', re.IGNORECASE)

# encontra os dados dos times (pelo caracter º)
data = soup.findAll(text=re.compile('\d{1,2}[\xc2\xba]'))

# cria a lista de nomes removendo as tags
# e qualquer outro lixo
for n in name:
text = junk.sub('', str(n)).strip()
if text: name_list.append(text)

# cria a lista de desempenho por time
if len(data) % len(name_list):
print 'Oops! Problemas com os dados.'
sys.exit(1)

div = len(data) / len(name_list)
for n in range(0, len(data)+1, div):
data_list.append(data[n:n+div])

# imprime tudo :)
for k, v in zip(name_list, data_list):
print k, ', '.join(v)[/sourcecode]
Além de simples e razoavelmente rápido, o Beautil Soup tem muitos recursos que permitem interpretar o conteúdo de forma eficiente, evitando o trabalho demorado de escrever um mega parser pra cada HTML que vai interpretar.

Se rodar o programa, ele mostra isso:

$ python desempenho-futebol.py
Atlético/MG 11º, 13º, 15º, 7º, 12º
Atlético/PR 6º, 5º, 5º, 9º, 5º
Botafogo 2º, 10º, 9º, 15º, 9º
Cruzeiro 2º, 2º, 1º, 1º, 2º
Coritiba 2º, 8º, 7º, 8º, 13º
Figueirense 9º, 4º, 12º, 11º, 16º
Flamengo 1º, 3º, 2º, 2º, 1º
Fluminense 11º, 17º, 19º, 20º,  20º
Goiás 13º, 14º, 16º, 17º, 19º
Grêmio 6º, 6º, 3º, 5º, 4º
Internacional 6º, 11º, 13º, 13º, 17º
Ipatinga 14º, 20º, 20º, 16º, 15º
Náutico 5º, 1º, 4º, 3º, 3º
Palmeiras 18º, 12º, 10º, 6º, 10º
Portuguesa 9º, 16º, 18º, 19º, 14º
Santos 17º, 7º, 14º, 14º, 17º
São Paulo 14º, 15º, 17º, 18º, 11º
Sport 18º, 17º, 11º, 10º, 6º
Vasco 14º, 8º, 7º, 4º, 7º
Vitória 18º, 17º, 6º, 12º, 8º

Se você for copiar e colar o código, não esqueça que esse maldito CSS do wordpress fica mexendo nas aspas. Pra resolver isso, tem os esquemas que publiquei aqui pra resolver.

Membros do grupo presentes: Carlos, Rodrigo, Tadeu

A reunião se iniciou às 8:15, abordando a elicitação de requisitos. Os clientes informaram que o trabalho deve seguir o padrão estabelecido pelo software legado, mas que não deve ser realizada mera "colagem". Também foi especificado que o sistema deve controlar o usuário, mas deve desprezar a existência de mais de um.

Fora perguntado pelo grupo se a possibilidade de utilização de LEX e YACC na implementação do parser. Os clientes disseram que essa seria uma solução possível, mas que provavelmente haveriam formas mais adeqüadas de tratar o problema da análise sintática, haja visto que trata-se de uma plataforma web.

Os clientes também definiram que a aquisição do vocabulário mínimo -- conjunto mínimo de palavras da língua portuguesa que conseguem definir uma vastidão de conceitos de forma não-ambígua -- é de responsabilidade do grupo. Especificaram também, que, caso não encontrado, o próprio grupo deveria gerar tal vocabulário.

Fora pedido também, ao encerramento da reunião, que o grupo já apresetasse parte do trabalho na terça-feira, 10 de junho.

sections can be inherited, you can define attributes and have special attributes to handle the parsing.. some features are not implemented yet..

teh pwnage code:

class Gizmo_Config_Xml {
    public $data;
    private $protectedAttributes = array( "_extends"=>"string","_private"=>"boolean" );

    public function __construct($filename,$section = null){
        if (empty($filename)) {
            throw new Exception('Filename is not set');
        }
        $config = simplexml_load_file($filename);
        if (null === $section)
            foreach ($config as $sectionName => $sectionData){
                $this->$sectionName = $sectionData;
            }
    }

    public function __set($var,$val){
        foreach ($val->attributes() as $k=>$v)
            $this->data[$var]['attributes'][$k]= (string) $v;
        $this->data[$var] = $this->_processSection($val,$var);

    }

    private function _processSection($xmlObj){
        $config = array();
        if(count($xmlObj->children())>0){
            foreach($xmlObj->children() as $k=>$v)
                $config['elements'][$k] = $this->_processSection($v);
        }
        foreach($xmlObj->attributes() as $k=>$v)
            $config['attributes'][$k] = (string) $v;
        if(count($xmlObj->children())==0)
            $config['value'] = (string) $xmlObj;

        return $config;
    }

    private function _toArray($data,$sn){
        if(is_array($data['attributes'])){
            foreach($data['attributes'] as $k=>$v)
                if(array_key_exists($k,$this->protectedAttributes)){
                    try{
                        settype($v, $this->protectedAttributes[$k]);
                    }catch (Exception $e){
                        echo $e->getMessage();
                    }
                    if($k=="_extends") {
                        $ret_data = $this->_arrayMergeRecursive($this->data[$v],$data);
                        unset($ret_data['attributes']['_extends']);
                    }
                    if($k=="_private"){
                        $flag_private = true;
                    }
                }else
                    $ret_data['attributes'][$k] = $v;
        }

        if(is_array($data['elements']))
            foreach($data['elements'] as $k=>$v)
                $ret_data['elements'][$k] = $this->_toArray($v,$k);
        elseif(!isset($ret_data))
            $ret_data = $data;

        if(count($ret_data['elements']))
            unset($ret_data['value']);
        else
            $ret_data['value'] = $data['value'];

        return ($flag_private)?FALSE:$ret_data;
    }

    private function _arrayMergeRecursive($array1, $array2){
        if (is_array($array1) && is_array($array2))
            foreach ($array2 as $key => $value)
                if (isset($array1[$key]))
                    $array1[$key] = $this->_arrayMergeRecursive($array1[$key], $value);
                else
                    $array1[$key] = $value;
        else
            $array1 = $array2;

        return $array1;
    }

    public function toArray(){
        $ret = array();
        foreach($this->data as $k=>$v){
            $a = $this->_toArray($v,$k);
            $a!==false ? $ret[$k] = $a : null;
        }
        return $ret;
    }

}

teh xml example:

<?xml version="1.0" encoding="UTF-8"?>
<config>
    <_topo _private="true" type="X1:topo">
        <catId type="long" optional="true"></catId>
        <topoId type="long" optional="true" />
        <netId type="long">0</netId>
    </_topo>

    <_pagi _private="true" type="X2:Pagi" namespace="http://x.com/ws/schema">
        <cnt type="boolean" />
        <pageNum type="positiveinteger">1</pageNum>
        <ipp type="positiveInteger">10</ipp>
    </_pagi>

    <_sort _private="true" type="X3:Sort" namespace="http://x.com/ws/schema">
         <sortBy type="string" enum="viewCount,price,averageRating">enum[0]</sortBy>
         <sortOrder type="string" enum="asc,desc">enum[0]</sortOrder>
    </_sort>

    <getSomeList>
        <xId type="long" optional="true" />
        <topology _extends="_topo" />
        <pagination _extends="_pagi" />
        <sort _extends="_sort" />
        <testParam _extends="_sort">
            <testSubParam type="Qx:test" namespace="y.com">
                <testSubSubParam type="long" />
            </testSubParam>
        </testParam>
    </getSomeList>
</config>

tip: it can be used in some nifty soap actions

But nevermind that, because it's not part of the issue at the moment. The issue is, that I have written so far in my life written 2 parser algoritms for xml files: one in Delphi/Object-Pascal, and today in C++.

And you seriously wouldn't ask me "Why?" if you had actually seen those horrible DOM interfaces where implementations come with a size ranging from 10-50MB.

The reason I wrote them is to "just" directly convert XML formatted strings or files, to the internal tree-node-leaf-whatever system I may be using at the time.

The point is, I write these things in about - let's see - this one in C++ is about 170 lines of code. When I start writing it, and it happened both times, I don't actually know what I'm doing... I just start writing, I know there's some if structures, state booleans and position integers - but I'm not actually 100% aware of what I'm writing. It's like typing on a keyboard like a monkey, deleting and retyping, deleting and retyping, and suddenly the code is there and it works.

I seriously don't think any kind of fileformat should be able to put you through that state of not knowing. A fileformat should be *almost* as easy as writing the actual output functions, which everyone know in case of XML takes you about 60 seconds to write.

Anyway, enough ranting. Time to upload it to subversion.

Feat Calculators

Feat calculator at conanarmory.com

Has a lower browser requirement, which makes it usable for more people.

Feat calculator at feats.goonheim.com

Most commonly referenced feat calculator in the AoC community.

Feat calculator at tentonhammer.com

Spell and Combo references

Detailed list of all spells and combos at www.hybes.org

Maps

Zone maps at conanarmory.com

Very slick and useful maps, with user added information.

WIKI

The Age of Conan Wiki at aoc.wikia.com

Leveling Flowchart and Overview of the zones

Flowchart at got3n.com

User Interface mods

Mirage UI

The most popular UI mod at the moment, it is well done and offers some customization.  I have switched to using this UI mod and I'm happy with how it works.

Werik UI

A minimalist approach. It removes all the decorative frills and makes all the buttons smaller.

Curse Gaming Age of Conan downloads

Popular gaming site for MMO mods.

AoC Custom UI Patcher

Every time Funcom releases a patch, it breaks the custom UI's.  This external program updates the files with the latest version number, so it works properly with the newest patch.

Combat Log Parser

AoCombaTotals

An external program that parses your combat log in real time.  It also has a small window that displays your damage/sec and xp/sec.  If you are running AoC in windowed mode, it allows you to see real time data.  Very useful.

In addition, I am currently using this module to write a parser combinator library for OCaml. This library has reached early testing stage and will hopefully be added to Batteries Included soon.

Expressing the program logic in a diagram give it the advantage of making it more understandable. This comes from the fact that the human brain is thinking in graphical way. Most of developers document their code with a flowchart. It will be better it we create a tool doing this job. We use a recursive decent parser parses a C# code and draw the corresponding flowchart.

C# flow control:

In C# language there are a lot of flow-control statements:

1. IF-Else.
   
2. While.
   
3. Do-while.
   
4. For.
   
5. Foreach.
   
6. Switch.
   

Each of these statements affects the flow of events in C# code. Some of these statements are considered decision statement (If-Else and switch). The others are considered loop statements (while, do-while, for and foreach).

Decision statements put a condition to go through a block of code. If the condition occurred the block of code will be executed. Otherwise, the block of code will be skipped. If-else statement exactly does this job CFG:

If-statement := if
(condition)
{ statements1 } [else { statements2 }]

The if part is necessary put the else part is optional. So the flowchart of the if-else statement takes two forms:

The switch statement guides the program to a path from multiple paths. The CFG of switch statement is:

Switch_statement := switch
(expression) { { case value: statements break; } }

As the switch statement has multi-paths the flowchart will be in the form:

Loop statements repeat a block of statements according to a condition. Only the expression and the expression place changes from a statement to another. The do-while loop which have the following CFG:

Do-while := do
{ statements } while
(condition );

Takes the form:

The other statements have the same form in flowchart. There CFG's are:

While-statement:= while
(condition )
{statements}

For-statement:= for
(for-Exp)
{ statements }

Foreach := foreach(foreach-exp)
{statements }

The flowchart form of these statements is:

Process

The process starts when a new code entered. This code enters a scanner that output tokens. The tokens enter a recursive decent parser gives a parse tree represents the entered code. The parse tree then enters the drawing module to be displayed as a model.

Scanner

The scanner takes the code and tries to split the code into a set of tokens each token belongs to one of these types:

1. Flow-control keyword: keywords represent one of the six flow-control statements discussed in C# flow control section.
   
2. Block start: like "{" symbol.
   
3. Block end: like "}" symbol.
   
4. While spaces.
   
5. Separators: set of symbols separate two tokens.
   

According to these types tokens will take there places in parse tree discussed in next section.

Parse Tree

The parse tree structure used in this tool must have two criteria. First it can represent the sequential statements. Second, it must represent the nested statements. So we use a tree structure represented by these classes:

Stake current=new TreeNode("main block");

Class TreeNode

{

Block current

Public Relation[] relation;

}

Class Relation

{

String Name;

treeNode from;

treeNode to;

}

TreeNode is a class represents a node in the parse tree. Each node have object from Block class represents a specified block in flowchart may be decision, loop start, loop end, process start state or end state.
In TreeNode
there is array of relations represents the relations from this TreeNode to its child nodes.

Parser

The function of parser is to take the tokens and place them in their places in a parse tree. The parser specifies the token place in parse tree according to token types. If the token is new flow control token it will be added as a new node in the node existing in the top of stack and this new node will be pushed in the current stack. If token is not a flow control token, the token will be added in the "current" block in the current node. Finally if the token is block termination token (like: } ) pop the first node in current stack.

Drawer

After the parser produced the parse tree, the Drawer takes the parse tree and draws a flowchart representing it. The Algorithm used in the Drawer as follow:

An algorithm is needed to determine the place where the drawer should place the shapes it draws. The y-position is the same for nodes have the same parent. While the x-position should be incremented for a child than the previous one.

Conclusion

To Convert a C# code to flowchart you need to parse the code searching for the flow control statements and code blocks then its very simple to draw the flowchart if you have a well structured parse tree.

• ANTLR은 무엇입니까?

ANTLR은 다양한 타겟 언어에서 액션을 포함하는 문법적 설명으로 부터 분석자(Recognizers), 통역자(Interpreters), 컴파일러(Compilers)와 번역기를 제공하는 언어 도구이다. ANTLR은 트리 구성, 트리 순회, 번역, 오류 회복, 오류 알림을 위한 뛰어난 지원을 제공한다. 현재 한달에 약 5,000건의 ANTLR 소스가 다운로드 되고 있다.

공식 사이트 : http://www.antlr.org/

번역 문서 : 저작권이 있는 문서이므로 공개를 중지합니다. 번역 후 가이드를 만들어 올리겠습니다.

Please find the below URL for more details :http://mercury.ccil.org/~cowan/XML/tagsoup/

Download: parser-combinators.scm

The code will run in Chicken Scheme.

We can define a parser as a procedure which takes an input string and an index (i.e. the index of the current character on the input). If the parse succeeds, it returns a value and an index ≥ the original index to the unconsumed input. On failure it returns #f for the both the value and index (the index is checked for failure and the value is ignored).

The two simplest parsers are fail, which always fails, and (return v), which always succeeds and returns v without consuming any input.

(define fail (lambda (s i) (values #f #f)))
(define (return v) (lambda (s i) (values v i)))

The parser any-char removes one character of the input or fails if the end of the input is reached.

(define any-char
  (lambda (s i)
    (if (< i (string-length s))
        (values (string-ref s i) (+ i 1))
        (values #f #f))))

On their own, parser procedures are cumbersome. Haskell provides a neat solution: do notation. Now for some Macro Magic. Let's write a macro called parser which allows us to write Haskell-style code.

(define-for-syntax *v-name* (gensym 'v))
(define-for-syntax *s-name* (gensym 's))
(define-for-syntax *i-name* (gensym 'i))

(define-for-syntax (expand-parser-body forms)
  (match forms
    [(_ '<- _)
       (error "parser must end with non-binding form")]
    [(p)
       `(,p ,*s-name* ,*i-name*)]
    [(v '<- p . xs)
       `(let-values ([(,v ,*i-name*) (,p ,*s-name* ,*i-name*)])
          (if ,*i-name*
              ,(expand-parser-body xs)
              (values #f #f)))]
    [(p . xs)
       `(let-values ([(,*v-name* ,*i-name*) (,p ,*s-name* ,*i-name*)])
          (if ,*i-name*
              ,(expand-parser-body xs)
              (values #f #f)))]))

(define-macro (parser . body)
  `(lambda (,*s-name* ,*i-name*)
     ,(expand-parser-body body)))

Now we can write parsers which look and work just like Haskell code!

(define two-chars-swap
  (parser
    a <- any-char
    b <- any-char
    (return (string b a))))

This example shows how we can use the any-char parser to write a parser to read two characters returning the string of the characters in reverse order. The results of any-char are bound to local variables which can be used immediately after definition.

How does this work? Let's look at what two-chars-swap expands into.

1:  (lambda (s3 i4)
2:    (let-values ([(a i4) (any-char s3 i4)])
3:      (if i4
4:          (let-values ([(b i4) (any-char s3 i4)])
5:            (if i4
6:                ((return (string b a)) s3 i4)
7:                (values #f #f)))
8:          (values #f #f))))

In line 1 we see the parser is actually a function, which takes a string (s3) and an index (i4), as expected (these are gensym'd variable names). Line 2 binds the result of calling any-char with the string and index to the a variable defined by the user. Notice how the call is implicit in the unexpanded form. The new index is re-bound to i4, shadowing the original index. Line 3 tests the index to see if the parse failed. Failure here causes failure for the whole parser (line 8). Otherwise we continue and call any-char again with the new index, binding the result to b and shadowing i4 just like in line 2. Line 5 checks for failure (line 7). Finally we call the (return (string b a)) parser, whose result is also the result of the whole parser.

No parser demo is complete without an implementation of an infix calculator language. One disadvantage of parser combinators is that left-recursion is not allowed (it will cause an infinite loop). This can be overcome by using a loop to slurp up sequences of operators of the same precedence, e.g. 1 + 2 + 3. See this page for a good explanation. Curiously, most examples I found on the web were wrong (they would parse "1 + 2" ignoring "+ 3"). I guess nobody actually tests their grammars.

Before we can write interesting parsers we need a few low-level utilities. The choice procedure takes a list of parsers and returns a parser which tries each parser in sequence until one of them succeeds. This gives us basic backtracking.

(define (choice . ps)
  (lambda (s i)
    (let loop ([p ps])
      (if (pair? p)
          (let-values ([(v i) ((car p) s i)])
            (if i
                (values v i)
                (loop (cdr p))))
          (values #f #f)))))

The matches procedure returns a parser which matches a particular string or fails. This is useful for matching symbols or keywords.

(define (matches m)
  (lambda (s i)
    (let ([n (string-length m)])
      (if (and (<= (+ i n) (string-length s))
               (string=? m (substring s i (+ i n))))
          (values (substring s i (+ i n)) (+ i n))
          (values #f #f)))))

The while-char procedure returns a parser which accepts characters while the character predicate holds. while1-char works similarly but requires at least one character.

(define (while-char pred)
  (lambda (s i)
    (let ([len (string-length s)])
      (let loop ([j i])
        (if (and (< j len) (pred (string-ref s j)))
            (loop (+ j 1))
            (values (substring s i j) j))))))

(define (while1-char pred)
  (parser
    s <- (while-char pred)
    (if (> (string-length s) 0) (return s) fail)))

In the calculator language we need to match decimal integers. Thanks to while1-char this is very easy!

(define decimal
  (parser
    s <- (while1-char digit?)
    (return (string->number s))))

To accept spaces around numbers and operators we use token which slurps up spaces before calling the given parser.

(define (token p)
  (parser
    (while-char space?)
    x <- p
    (return x)))

Finally we have enough to define the classical term/factor/expr parser. This version returns the s-expression instead of calculating the result.

(define expr
  (parser
    lhs <- term
    (let loop ([lhs lhs])
      (choice
        (parser
          opr <- (token (choice (matches "+") (matches "-")))
          rhs <- term
          (loop (list (string->symbol opr) lhs rhs)))
        (return lhs)))))

(define term
  (parser
    lhs <- factor
    (let loop ([lhs lhs])
      (choice
        (parser
          opr <- (token (choice (matches "*") (matches "/")))
          rhs <- factor
          (loop (list (string->symbol opr) lhs rhs)))
        (return lhs)))))

(define factor
  (choice
    (token decimal)
    (parser
      (token (matches "("))
      e <- expr
      (token (matches ")"))
      (return e))
    (parser
      (token (matches "-"))
      e <- factor
      (return (list '- e)))))

Now we can use test-parser (see the code file for details) to test the expression parser.

#;1> (test-parser expr)
>> 1 + 5/3 * (8 + (9 - -4)) / (7*7 + 6) + 2
Parsed    : "1 + 5/3 * (8 + (9 - -4)) / (7*7 + 6) + 2" (40 characters)
Returned  : (+ (+ 1 (/ (* (/ 5 3) (+ 8 (- 9 (- 4)))) (+ (* 7 7) 6))) 2)

Veremos como construir un parser genérico de XML y como lo podemos adaptar a nuestras necesidades.

Entre las clases más importantes a tener en cuenta tenemos:

Interface org.xml.sax.XMLReader:
Está interfaz que debe implementar un lector de XML.
Cada vez que el XMLReader encuentra el principio del archivo, el final, un elemento, un caracter especial, un espacio, etc. notifica esto al ContentHandler asociado.
Cada vez que encuentra un error se lo notifica al ErrorHandler asociado.

Interface org.xml.sax.ContentHandler:
La clase que la implementa obtiene la capacidad de recibir todas las notificaciones de contenido de un XMLReader.
La forma de asociar este a un XMLReader es mediante el método setContentHandler.

Interface org.xml.sax.ErrorHandler:
La clase que la implementa obtiene la capacidad de recibir todas las notificaciones de error producidas por un XMLReader
La forma de asociar este a un XMLReader es mediante el método setErrorHandler.
En caso de que no exista ningun ErrorHandler asociado al XMLReader, este ultimo no reportara ningun error, salvo las excepciones SAXParseException.

Clase org.xml.sax.helpers.DefaultHandler:
Clase que implementa tanto a ContentHandler como a ErrorHandler (además de DTDHandler y EntityResolver que no veremos por ahora), proveiendo implementaciones por defectos para todos sus métodos.

Esta clase es de la que extenderemos para poder crear nuestro propio parser de XML.

Clase org.xml.sax.helpers.XMLReaderFactory:
Clase que provee métodos estáticos para la creación de XMLReaders.
Utilizaremos el método estático XMLReaderFactory.createXMLReader() que nos devuelve el XMLReader un por defecto para nuestro sistema.

Creando el Lector
Para ello haremos de la siguiente forma:

[sourcecode language="java"]
class LectorXML extends DefaultHandler {

private final XMLReader xr;

public LectorXML() throws SAXException {
xr = XMLReaderFactory.createXMLReader();
xr.setContentHandler(this);
xr.setErrorHandler(this);
}
}[/sourcecode]
Hasta aquí hemos creado un clase LectorXML que hereda de DefaultHandler, donde en el constructor creamos el XMLReader mediante XMLReaderFactory.
Al XMLReader le asociamos el ContentHandler y ErrorHandler, que es el propio LectorXML.Agregamos un método para leer archivos XML:
[sourcecode language="java"]
class LectorXML extends DefaultHandler {

private final XMLReader xr;

public LectorXML() throws SAXException {
xr = XMLReaderFactory.createXMLReader();
xr.setContentHandler(this);
xr.setErrorHandler(this);
}

public void leer(final String archivoXML)
throws FileNotFoundException, IOException,
SAXException {
FileReader fr = new FileReader(archivoXML);
xr.parse(new InputSource(fr));
}
}[/sourcecode]
En este punto estamos preparados para probar nuestro Lector:
[sourcecode language="java"]
public class PruebaSAX {
public static void main(String[] args)
throws FileNotFoundException, IOException,
SAXException {
LectorXML lector = new LectorXML();
lector.leer("test.xml");
}
}[/sourcecode]
Si ejecutamos veremos que se produce nada (salvo que la ruta al XML sea incorrecta). Esto es porque los métodos de DefaultHandler tienen implementaciones vacías de los métodos.Sobrescribimos entonces algunos de los métodos para ver la salida:
[sourcecode language="java"]
class LectorXML extends DefaultHandler {

private final XMLReader xr;

public LectorXML() throws SAXException {
xr = XMLReaderFactory.createXMLReader();
xr.setContentHandler(this);
xr.setErrorHandler(this);
}

public void leer(final String archivoXML)
throws FileNotFoundException, IOException,
SAXException {
FileReader fr = new FileReader(archivoXML);
xr.parse(new InputSource(fr));
}

@Override
public void startDocument() {
System.out.println("Comienzo del Documento XML");
}

@Override
public void endDocument() {
System.out.println("Final del Documento XML");
}

@Override
public void startElement(String uri, String name,
String qName, Attributes atts) {
System.out.println("tElemento: " + name);

for (int i = 0; i < atts.getLength(); i++) {
System.out.println("ttAtributo: " +
atts.getLocalName(i) + " = "+ atts.getValue(i));
}
}

@Override
public void endElement(String uri, String name,
String qName) {
System.out.println("tFin Elemento: " + name);
}
}[/sourcecode]
Si ejecutamos ahora veremos como nos muestra el comienzo y fin del documento, así como todos los elementos y sus atributos de nuestro documento.

De la misma forma podemos sobrescribir los métodos para ver los caracteres especiales que aparecen en el XML (espacios, saltos de lineas, etc), mostrar errores y excepciones generadas a la hora de realizar la lectura y parseo, etc.

Espero que les sirva.
Hasta la Próxima.

Más Info