W3C网站上的有点小bug,而且他们写的正则是FLEX一类的,和自己手写时不太一样。瞎改了一下,贴这,谁用得着谁拿去,有问题请通知作者更新。虽然是python的,其实现在这些RegEx都是perl风格的,应该能通用。
除了做词法,这些定义也可以这么用:如果是找找特定字串,从这里选取一部分改吧改吧就好了;或者在这堆天书里看看有啥用法是咱不知道的?
(?:..)的意思是不建立capturing group,不知道别的运行时是不是这么定义的;字串前面的r意思是不用转义字符,.NET里好像是@?已然忘了。另外,W3C的词法分析配置是忽略大小写,不忽略空白:别忘了后代选择器是基于空白的。另外,w3c的没有.class的定义,这是我自己加的,会让微软的filter变成:IDENT CLASS CLASS LPARENTHESE..这样(本来w3c版本也解析不了这个),其它副作用不明。
(版本:CSS 2.1 + CSS 3 Selector,问我为啥不80字符换行?不乐意。)
A = r"(?:a|\\0{0,4}(?:41|61)(?:\r\n|[ \t\r\n\f])?)"
C = r"(?:c|\\0{0,4}(?:43|63)(?:\r\n|[ \t\r\n\f])?)"
D = r"(?:d|\\0{0,4}(?:44|64)(?:\r\n|[ \t\r\n\f])?)"
E = r"(?:e|\\0{0,4}(?:45|65)(?:\r\n|[ \t\r\n\f])?)"
G = r"(?:g|\\0{0,4}(?:47|67)(?:\r\n|[ \t\r\n\f])?|\\g)"
H = r"(?:h|\\0{0,4}(?:48|68)(?:\r\n|[ \t\r\n\f])?|\\h)"
I = r"(?:i|\\0{0,4}(?:49|69)(?:\r\n|[ \t\r\n\f])?|\\i)"
K = r"(?:k|\\0{0,4}(?:4b|6b)(?:\r\n|[ \t\r\n\f])?|\\k)"
L = r"(?:l|\\0{0,4}(?:4c|6c)(?:\r\n|[ \t\r\n\f])?|\\l)"
M = r"(?:m|\\0{0,4}(?:4d|6d)(?:\r\n|[ \t\r\n\f])?|\\m)"
N = r"(?:n|\\0{0,4}(?:4e|6e)(?:\r\n|[ \t\r\n\f])?|\\n)"
O = r"(?:o|\\0{0,4}(?:4f|6f)(?:\r\n|[ \t\r\n\f])?|\\o)"
P = r"(?:p|\\0{0,4}(?:50|70)(?:\r\n|[ \t\r\n\f])?|\\p)"
R = r"(?:r|\\0{0,4}(?:52|72)(?:\r\n|[ \t\r\n\f])?|\\r)"
S = r"(?:s|\\0{0,4}(?:53|73)(?:\r\n|[ \t\r\n\f])?|\\s)"
T = r"(?:t|\\0{0,4}(?:54|74)(?:\r\n|[ \t\r\n\f])?|\\t)"
U = r"(?:u|\\0{0,4}(?:55|75)(?:\r\n|[ \t\r\n\f])?|\\u)"
X = r"(?:x|\\0{0,4}(?:58|78)(?:\r\n|[ \t\r\n\f])?|\\x)"
Z = r"(?:z|\\0{0,4}(?:5a|7a)(?:\r\n|[ \t\r\n\f])?|\\z)"
# from css 3 selector
#D = r"d|\\0{0,4}(44|64)(\r\n|[ \t\r\n\f])?"
#E = r"e|\\0{0,4}(45|65)(\r\n|[ \t\r\n\f])?"
#N = r"n|\\0{0,4}(4e|6e)(\r\n|[ \t\r\n\f])?|\\n"
#O = r"o|\\0{0,4}(4f|6f)(\r\n|[ \t\r\n\f])?|\\o"
#T = r"t|\\0{0,4}(54|74)(\r\n|[ \t\r\n\f])?|\\t"
#V = r"v|\\0{0,4}(58|78)(\r\n|[ \t\r\n\f])?|\\v"
s = r"[ \t\r\n\f]+"
w = r"[ \t\r\n\f]*"
nl = r"(?:\n|\r\n|\r|\f)"
num = "\-?(?:[0-9]+|[0-9]*\.[0-9]+)"
nonascii = r"[^\0-\237]" # 2.1: [\200-\4177777]
unicode_ = r"\\[0-9a-f]{1,6}(?:\r\n|[ \n\r\t\f])?"
escape = "(?:" + unicode_ + r"|\\[^\n\r\f0-9a-f])"
string1 = r'\"(?:[^\n\r\f\\"]|\\' + nl + r'|' + escape + ')*\"'
string2 = r"\'(?:[^\n\r\f\\']|\\" + nl + r"|" + escape + r")*\'"
string = "(?:" + string1 + "|" + string2 + ")"
invalid1 = r'\"(?:[^\n\r\f\\"]|\\' + nl +'|' + nonascii + '|' + escape + ')*'
invalid2 = r"\'(?:[^\n\r\f\\']|\\" + nl + "|" + nonascii + "|" + escape + ")*"
invalid = invalid1 + "|" + invalid2
comment = r"\/\*[^*]*\*+(?:[^/*][^*]*\*+)*\/"
badcomment1 = r"\/\*[^*]*\*+(?:[^/*][^*]*\*+)*"
badcomment2 = r"\/\*[^*]*(?:\*+[^/*][^*]*)*"
badcomment = r"(?:" + badcomment1 + "|" + badcomment2 + ")"
nmchar = r"(?:[_a-z0-9\-]|" + nonascii + "|" + escape + ")"
nmstart = r"(?:[_a-z]|" + nonascii + "|" + escape + ")"
ident = r"[-]?" + nmstart + "(?:" + nmchar + ")*"
name = nmchar + "+"
unicode_range = r"u\+[0-9a-f?]{1,6}(?:-[0-9a-f]{1,6})?"
openbrace = r"\{"
closingbrace = r"\}"
openparenthese = r"\("
closingparenthese = r"\)"
openbracket = r"\["
closingbracket = r"\]"
url = r"(?:[\!\#\$\%\&\*-\~]|" + nonascii + "|" + escape + ")*" # from http://www.w3.org/TR/CSS21/grammar.html
uri = r"url\(" + w + string + w + r"\)" + r"|url\(" + w + r"(?:[\!\#\$\%\&\*-\[\]-\~]|" + nonascii + r"|" + escape + r")*" + w + r"\)"
baduri1 = r"url\(" + w + "(?:[\!\#\$\%\&\*-\[\]-\~]|" + nonascii + "|" + escape + ")*" + w
baduri2 = r"url\(" + w + string + w
baduri3 = r"url\(" + w + invalid
baduri = "(?:" +baduri1 + "|" + baduri2 + "|" + baduri3 + ")"
NOT = N+O+T
IMPORT = I+M+P+O+R+T
PAGE = P+A+G+E
MEDIA = M+E+D+I+A
IMPORTANT = I+M+P+O+R+T+A+N+T
URL = U+R+L
words = (
("S", s),
("COMMENT", comment),
("BADCOMMENT", badcomment),
("CDO", "<!--"),
("CDC", "-->"),
("INCLUDES", r"\~\="),
("DASHMATCH", r"\|\="),
("PREFIXMATCH", r"\^\="),
("SUFFIXMATCH", r"\$\="),
("SUBSTRINGMATCH", r"\*\="),
("STRING", string),
("BADSTRING", invalid),
("URI", "(?:" + URL + r"\(" + w + string + w + r"\)|" + URL + r"\(" + w + url + w + r"\))"),
("BAD_URI", baduri),
("FUNCTION", ident + r"\("),
("IDENT", ident),
("HASH", r"\#" + name),
("CLASS", r"\." + ident),
("IMPORT_SYM", r"\@" + IMPORT),
("PAGE_SYM", r"\@" + PAGE),
("MEDIA_SYM", r"\@" + MEDIA),
("CHARSET_SYM", r"\@charset "),
("IMPORTANT_SYM", r"\!(" + w + "|" + comment + ")*" + IMPORTANT),
("EMS", num + E + M),
("EXS", num + E + X),
("LENGTH", "(?:" + num + P + X + "|" + num + C + M + "|" + num + M + M + "|" + num + I + N + "|" + num + P + T + "|" + num + P + C + ")"),
("ANGLE", "(?:" + num + D + E + G + "|" + num + R + A + D + "|" + num + G + R + A + D + ")"),
("TIME", "(?:" + num + M + S + "|" + num + S + ")"),
("FREQ", "(?:" + num + H + Z + "|" + num + K + H + Z + ")"),
("DIMENSION", num + ident),
("PERCENTAGE", num + r"\%"),
("NUMER", num),
#("FUNCTION", ident + r"\("),
("PLUS", w + r"\+"),
("GREATER", w + r"\>"),
("COMMA", w + r"\,"),
("TILDE", w + r"\~"),
("NOT", ":" + NOT + r"\("),
("ATKEYWORD", r"\@" + ident),
("SEMICOLON", r"\;"),
("COLON", r"\:"),
("LBRACE", openbrace),
("RBRACE", closingbrace),
("RPARENTHESE", closingparenthese),
("LPARENTHESE", openparenthese),
("LBRACKET", openbracket),
("RBRACKET", closingbracket),
("EQUALSIGN", r"\="),
("PERIOD", r"\."),
("STAR", r"\*"),
("ANY", ".")
)
简单说明一下,如果用RegEx库做词法分析,我是把这些用“|”连起来,然后不停的match。现在RegEx都是回溯引擎,至少对于Python来说,放前面的会被优先test,所以具有相同“First集”的,越特殊越要放前面。比如num,LENGTH以它开头,NUMBER就是num,那么LENGTH的放前头。FUNCTION、URL之类同理。对于NFA/DFA引擎或等价(或严格的超集)的标准正则表达式库(!=RegEx),会leftmost longest,所以应该无需关心顺序。
最后那个ANY是干嘛?没法match的全都ANY。解析CSS很少需要严格要求用户的文件,不认识的不理就算了。词法上是这样,语法的时候更容易处理。
最后骂一句,我怎么干上程序员这行当了,弄这些繁琐的旁枝末节真TM无聊的紧。还有,早知道要处理整套页面应用相关问题,绝B用C/C++写了,从webkit里直接copy,还能顺便学点其它不相干的经验,日了。