• Ogre源代码浅析——脚本及其解析(三)


          在完成了对脚本源文件的“词法分析”后,Ogre将进入到parse也就是“语义分析”阶段。这两个阶段的作用,从各自定义的数据结构中就可以一窥端倪,看下相应的结点定义:

     1     struct ScriptToken
     2     {
     3         /// This is the lexeme for this token
     4         String lexeme, file;
     5         /// This is the id associated with the lexeme, which comes from a lexeme-token id mapping
     6         uint32 type;
     7         /// This holds the line number of the input stream where the token was found.
     8         uint32 line;
     9     };
    10     typedef SharedPtr<ScriptToken> ScriptTokenPtr;
    11     typedef vector<ScriptTokenPtr>::type ScriptTokenList;
    12     typedef SharedPtr<ScriptTokenList> ScriptTokenListPtr;
    13 
    14 --------------------------------------------------------------------
    15 
    16     struct ConcreteNode;
    17     typedef SharedPtr<ConcreteNode> ConcreteNodePtr;
    18     typedef list<ConcreteNodePtr>::type ConcreteNodeList;
    19     typedef SharedPtr<ConcreteNodeList> ConcreteNodeListPtr;
    20     struct ConcreteNode : public ScriptCompilerAlloc
    21     {
    22         String token, file;
    23         unsigned int line;
    24         ConcreteNodeType type;
    25         ConcreteNodeList children;
    26         ConcreteNode *parent;
    27     };

         以上代码中,上半部分(1-12行)是“词法分析”的输出数据的数据结构;下半部分(16-27行)是“语义分析”阶段输出数据的数据结构。从定义可以看出“词法分析”的结果,要以ScriptToken为单位来保存脚本中的各“词素(lexeme)”信息,然后将所有ScriptToken对象存放到vector中。vector中各ScriptToken之间是一种并列的关系,它只一一列举了相应源脚本中所有的“词素(lexeme)”的数据,而并未反应出这些lexeme之间的逻辑关系。那么,Ogre的脚本文件在编写时是否都要遵循相同的规则?它们有统一的结构吗?为解决此问题,先列出“.material”、“.program”、“.particle”和“.compositor”四种脚本的片段,来探究一下它们在结构上的共同规律:

      1 SdkTrays.overlay
      2 
      3 template container Panel(SdkTrays/Cursor)
      4 {
      5     metrics_mode pixels
      6     transparent true
      7     
      8     // You can offset the image to change the cursor "hotspot"
      9     element Panel(CursorImage)
     10     {
     11         metrics_mode pixels
     12         material SdkTrays/Cursor
     13         width 32
     14         height 32
     15     }
     16 }
     17 ...
     18 
     19 ---------------------------------------------------------
     20 
     21 SdyTrays.material
     22 
     23 material SdkTrays/Base
     24 {
     25     technique
     26     {
     27         pass
     28         {
     29             lighting off
     30             scene_blend alpha_blend
     31             depth_check off
     32             
     33             texture_unit
     34             {
     35                 tex_address_mode clamp
     36                 filtering linear linear none
     37             }
     38         }
     39     }
     40 }
     41 ...
     42 ----------------------------------------------------------------
     43 
     44 smoke.particle
     45 
     46 particle_system Examples/Smoke
     47 {
     48     material            Examples/Smoke
     49     particle_width      35
     50     particle_height     35
     51     cull_each           true
     52     quota               500
     53     billboard_type      point
     54     sorted                true
     55     
     56     // Area emitter
     57     emitter Point
     58     {
     59         position 0 15 -15
     60         angle 35
     61         emission_rate 15
     62         time_to_live 4
     63         direction 0 1 0
     64         velocity_min 50
     65         velocity_max 80        
     66     }
     67 
     68     affector ColourImage
     69     {
     70         image smokecolors.png
     71     }
     72 
     73        affector Rotator
     74        {
     75         rotation_range_start 0
     76         rotation_range_end 360
     77         rotation_speed_range_start -60
     78         rotation_speed_range_end 200
     79        }
     80 
     81        affector Scaler
     82        {
     83            rate 50
     84        }
     85 
     86 }
     87 
     88 --------------------------------------------------------
     89 
     90 Example.compositor
     91 
     92 compositor Glass
     93 {
     94     technique
     95     {
     96         texture rt0 target_width target_height PF_R8G8B8
     97 
     98         target rt0 { input previous }
     99 
    100         target_output
    101         {
    102             // Start with clear output
    103             input none
    104 
    105             pass render_quad
    106             {
    107                 material Ogre/Compositor/GlassPass
    108                 input 0 rt0
    109             }
    110         }
    111     }
    112 }

          可以看到,在这些脚本中,真正的数据单位是以左右大括号为限定符的数据单元。如果把一个数据单元看作一个结点,那么其中一些结点可以嵌套在另一些结点中,且所有的结点将形成一个树形结构;而对于每一个结点来说,“词素”又是构建它的最小单位。因此,如果把“词法分析”的结果vector中的数据作为输入的话,那么“语义分析”的输出结果就应该是——以数据单元为结点构建成的“结点树”。ScriptParser::parse()函数就是用来完成语义分析任务的函数,看一下相关的代码:

      1     ConcreteNodeListPtr ScriptParser::parse(const ScriptTokenListPtr &tokens)
      2     {
      3         // MEMCATEGORY_GENERAL because SharedPtr can only free using that category
      4         ConcreteNodeListPtr nodes(OGRE_NEW_T(ConcreteNodeList, MEMCATEGORY_GENERAL)(), SPFM_DELETE_T);
      5 
      6         enum{READY, OBJECT};
      7         uint32 state = READY;
      8 
      9         ConcreteNode *parent = 0;
     10         ConcreteNodePtr node;
     11         ScriptToken *token = 0;
     12         ScriptTokenList::iterator i = tokens->begin(), end = tokens->end();
     13         while(i != end)
     14         {
     15             token = (*i).get();
     16 
     17             switch(state)
     18             {
     19             case READY:
     20                 if(token->type == TID_WORD)
     21                 {
     22                     if(token->lexeme == "import")
     23                     {
     24                         node = ConcreteNodePtr(OGRE_NEW ConcreteNode());
     25                         node->token = token->lexeme;
     26                         node->file = token->file;
     27                         node->line = token->line;
     28                         node->type = CNT_IMPORT;
     29 
     30                         // The next token is the target
     31                         ++i;
     32                         if(i == end || ((*i)->type != TID_WORD && (*i)->type != TID_QUOTE))
     33                             OGRE_EXCEPT(Exception::ERR_INVALID_STATE, 
     34                                 Ogre::String("expected import target at line ") + 
     35                                     Ogre::StringConverter::toString(node->line),
     36                                 "ScriptParser::parse");
     37                         ConcreteNodePtr temp(OGRE_NEW ConcreteNode());
     38                         temp->parent = node.get();
     39                         temp->file = (*i)->file;
     40                         temp->line = (*i)->line;
     41                         temp->type = (*i)->type == TID_WORD ? CNT_WORD : CNT_QUOTE;
     42                         if(temp->type == CNT_QUOTE)
     43                             temp->token = (*i)->lexeme.substr(1, token->lexeme.size() - 2);
     44                         else
     45                             temp->token = (*i)->lexeme;
     46                         node->children.push_back(temp);
     47 
     48                         // The second-next token is the source
     49                         ++i;
     50                         ++i;
     51                         if(i == end || ((*i)->type != TID_WORD && (*i)->type != TID_QUOTE))
     52                             OGRE_EXCEPT(Exception::ERR_INVALID_STATE, 
     53                                 Ogre::String("expected import source at line ") + 
     54                                     Ogre::StringConverter::toString(node->line),
     55                                 "ScriptParser::parse");
     56                         temp = ConcreteNodePtr(OGRE_NEW ConcreteNode());
     57                         temp->parent = node.get();
     58                         temp->file = (*i)->file;
     59                         temp->line = (*i)->line;
     60                         temp->type = (*i)->type == TID_WORD ? CNT_WORD : CNT_QUOTE;
     61                         if(temp->type == CNT_QUOTE)
     62                             temp->token = (*i)->lexeme.substr(1, (*i)->lexeme.size() - 2);
     63                         else
     64                             temp->token = (*i)->lexeme;
     65                         node->children.push_back(temp);
     66 
     67                         // Consume all the newlines
     68                         i = skipNewlines(i, end);
     69 
     70                         // Insert the node
     71                         if(parent)
     72                         {
     73                             node->parent = parent;
     74                             parent->children.push_back(node);
     75                         }
     76                         else
     77                         {
     78                             node->parent = 0;
     79                             nodes->push_back(node);
     80                         }
     81                         node = ConcreteNodePtr();
     82                     }
     83                     else if(token->lexeme == "set")
     84                     {
     85                         node = ConcreteNodePtr(OGRE_NEW ConcreteNode());
     86                         node->token = token->lexeme;
     87                         node->file = token->file;
     88                         node->line = token->line;
     89                         node->type = CNT_VARIABLE_ASSIGN;
     90 
     91                         // The next token is the variable
     92                         ++i;
     93                         if(i == end || (*i)->type != TID_VARIABLE)
     94                             OGRE_EXCEPT(Exception::ERR_INVALID_STATE, 
     95                                 Ogre::String("expected variable name at line ") + 
     96                                     Ogre::StringConverter::toString(node->line),
     97                                 "ScriptParser::parse");
     98                         ConcreteNodePtr temp(OGRE_NEW ConcreteNode());
     99                         temp->parent = node.get();
    100                         temp->file = (*i)->file;
    101                         temp->line = (*i)->line;
    102                         temp->type = CNT_VARIABLE;
    103                         temp->token = (*i)->lexeme;
    104                         node->children.push_back(temp);
    105 
    106                         // The next token is the assignment
    107                         ++i;
    108                         if(i == end || ((*i)->type != TID_WORD && (*i)->type != TID_QUOTE))
    109                             OGRE_EXCEPT(Exception::ERR_INVALID_STATE, 
    110                                 Ogre::String("expected variable value at line ") + 
    111                                     Ogre::StringConverter::toString(node->line),
    112                                 "ScriptParser::parse");
    113                         temp = ConcreteNodePtr(OGRE_NEW ConcreteNode());
    114                         temp->parent = node.get();
    115                         temp->file = (*i)->file;
    116                         temp->line = (*i)->line;
    117                         temp->type = (*i)->type == TID_WORD ? CNT_WORD : CNT_QUOTE;
    118                         if(temp->type == CNT_QUOTE)
    119                             temp->token = (*i)->lexeme.substr(1, (*i)->lexeme.size() - 2);
    120                         else
    121                             temp->token = (*i)->lexeme;
    122                         node->children.push_back(temp);
    123 
    124                         // Consume all the newlines
    125                         i = skipNewlines(i, end);
    126 
    127                         // Insert the node
    128                         if(parent)
    129                         {
    130                             node->parent = parent;
    131                             parent->children.push_back(node);
    132                         }
    133                         else
    134                         {
    135                             node->parent = 0;
    136                             nodes->push_back(node);
    137                         }
    138                         node = ConcreteNodePtr();
    139                     }
    140                     else
    141                     {
    142                         node = ConcreteNodePtr(OGRE_NEW ConcreteNode());
    143                         node->file = token->file;
    144                         node->line = token->line;
    145                         node->type = token->type == TID_WORD ? CNT_WORD : CNT_QUOTE;
    146                         if(node->type == CNT_QUOTE)
    147                             node->token = token->lexeme.substr(1, token->lexeme.size() - 2);
    148                         else
    149                             node->token = token->lexeme;
    150 
    151                         // Insert the node
    152                         if(parent)
    153                         {
    154                             node->parent = parent;
    155                             parent->children.push_back(node);
    156                         }
    157                         else
    158                         {
    159                             node->parent = 0;
    160                             nodes->push_back(node);
    161                         }
    162 
    163                         // Set the parent
    164                         parent = node.get();
    165 
    166                         // Switch states
    167                         state = OBJECT;
    168 
    169                         node = ConcreteNodePtr();
    170                     }
    171                 }
    172                 else if(token->type == TID_RBRACKET)
    173                 {
    174                     // Go up one level if we can
    175                     if(parent)
    176                         parent = parent->parent;
    177 
    178                     node = ConcreteNodePtr(OGRE_NEW ConcreteNode());
    179                     node->token = token->lexeme;
    180                     node->file = token->file;
    181                     node->line = token->line;
    182                     node->type = CNT_RBRACE;
    183 
    184                     // Consume all the newlines
    185                     i = skipNewlines(i, end);
    186 
    187                     // Insert the node
    188                     if(parent)
    189                     {
    190                         node->parent = parent;
    191                         parent->children.push_back(node);
    192                     }
    193                     else
    194                     {
    195                         node->parent = 0;
    196                         nodes->push_back(node);
    197                     }
    198 
    199                     // Move up another level
    200                     if(parent)
    201                         parent = parent->parent;
    202 
    203                     node = ConcreteNodePtr();
    204                 }
    205                 break;
    206             case OBJECT:
    207                 if(token->type == TID_NEWLINE)
    208                 {
    209                     // Look ahead to the next non-newline token and if it isn't an {, this was a property
    210                     ScriptTokenList::iterator next = skipNewlines(i, end);
    211                     if(next == end || (*next)->type != TID_LBRACKET)
    212                     {
    213                         // Ended a property here
    214                         if(parent)
    215                             parent = parent->parent;
    216                         state = READY;
    217                     }
    218                 }
    219                 else if(token->type == TID_COLON)
    220                 {
    221                     node = ConcreteNodePtr(OGRE_NEW ConcreteNode());
    222                     node->token = token->lexeme;
    223                     node->file = token->file;
    224                     node->line = token->line;
    225                     node->type = CNT_COLON;
    226 
    227                     // The following token are the parent objects (base classes).
    228                     // Require at least one of them.
    229 
    230                     ScriptTokenList::iterator j = i + 1;
    231                     j = skipNewlines(j, end);
    232                     if(j == end || ((*j)->type != TID_WORD && (*j)->type != TID_QUOTE)) {
    233                         OGRE_EXCEPT(Exception::ERR_INVALID_STATE, 
    234                             Ogre::String("expected object identifier at line ") + 
    235                                     Ogre::StringConverter::toString(node->line),
    236                             "ScriptParser::parse");
    237                     }
    238 
    239                     while(j != end && ((*j)->type == TID_WORD || (*j)->type == TID_QUOTE))
    240                     {
    241                         ConcreteNodePtr tempNode = ConcreteNodePtr(OGRE_NEW ConcreteNode());
    242                         tempNode->token = (*j)->lexeme;
    243                         tempNode->file = (*j)->file;
    244                         tempNode->line = (*j)->line;
    245                         tempNode->type = (*j)->type == TID_WORD ? CNT_WORD : CNT_QUOTE;
    246                         tempNode->parent = node.get();
    247                         node->children.push_back(tempNode);
    248                         ++j;
    249                     }
    250 
    251                     // Move it backwards once, since the end of the loop moves it forwards again anyway
    252                     j--;
    253                     i = j;
    254 
    255                     // Insert the node
    256                     if(parent)
    257                     {
    258                         node->parent = parent;
    259                         parent->children.push_back(node);
    260                     }
    261                     else
    262                     {
    263                         node->parent = 0;
    264                         nodes->push_back(node);
    265                     }
    266                     node = ConcreteNodePtr();
    267                 }
    268                 else if(token->type == TID_LBRACKET)
    269                 {
    270                     node = ConcreteNodePtr(OGRE_NEW ConcreteNode());
    271                     node->token = token->lexeme;
    272                     node->file = token->file;
    273                     node->line = token->line;
    274                     node->type = CNT_LBRACE;
    275 
    276                     // Consume all the newlines
    277                     i = skipNewlines(i, end);
    278 
    279                     // Insert the node
    280                     if(parent)
    281                     {
    282                         node->parent = parent;
    283                         parent->children.push_back(node);
    284                     }
    285                     else
    286                     {
    287                         node->parent = 0;
    288                         nodes->push_back(node);
    289                     }
    290 
    291                     // Set the parent
    292                     parent = node.get();
    293 
    294                     // Change the state
    295                     state = READY;
    296 
    297                     node = ConcreteNodePtr();
    298                 }
    299                 else if(token->type == TID_RBRACKET)
    300                 {
    301                     // Go up one level if we can
    302                     if(parent)
    303                         parent = parent->parent;
    304 
    305                     // If the parent is currently a { then go up again
    306                     if(parent && parent->type == CNT_LBRACE && parent->parent)
    307                         parent = parent->parent;
    308 
    309                     node = ConcreteNodePtr(OGRE_NEW ConcreteNode());
    310                     node->token = token->lexeme;
    311                     node->file = token->file;
    312                     node->line = token->line;
    313                     node->type = CNT_RBRACE;
    314 
    315                     // Consume all the newlines
    316                     i = skipNewlines(i, end);
    317 
    318                     // Insert the node
    319                     if(parent)
    320                     {
    321                         node->parent = parent;
    322                         parent->children.push_back(node);
    323                     }
    324                     else
    325                     {
    326                         node->parent = 0;
    327                         nodes->push_back(node);
    328                     }
    329 
    330                     // Move up another level
    331                     if(parent)
    332                         parent = parent->parent;
    333 
    334                     node = ConcreteNodePtr();
    335                     state = READY;
    336                 }
    337                 else if(token->type == TID_VARIABLE)
    338                 {
    339                     node = ConcreteNodePtr(OGRE_NEW ConcreteNode());
    340                     node->token = token->lexeme;
    341                     node->file = token->file;
    342                     node->line = token->line;
    343                     node->type = CNT_VARIABLE;
    344 
    345                     // Insert the node
    346                     if(parent)
    347                     {
    348                         node->parent = parent;
    349                         parent->children.push_back(node);
    350                     }
    351                     else
    352                     {
    353                         node->parent = 0;
    354                         nodes->push_back(node);
    355                     }
    356                     node = ConcreteNodePtr();
    357                 }
    358                 else if(token->type == TID_QUOTE)
    359                 {
    360                     node = ConcreteNodePtr(OGRE_NEW ConcreteNode());
    361                     node->token = token->lexeme.substr(1, token->lexeme.size() - 2);
    362                     node->file = token->file;
    363                     node->line = token->line;
    364                     node->type = CNT_QUOTE;
    365 
    366                     // Insert the node
    367                     if(parent)
    368                     {
    369                         node->parent = parent;
    370                         parent->children.push_back(node);
    371                     }
    372                     else
    373                     {
    374                         node->parent = 0;
    375                         nodes->push_back(node);
    376                     }
    377                     node = ConcreteNodePtr();
    378                 }
    379                 else if(token->type == TID_WORD)
    380                 {
    381                     node = ConcreteNodePtr(OGRE_NEW ConcreteNode());
    382                     node->token = token->lexeme;
    383                     node->file = token->file;
    384                     node->line = token->line;
    385                     node->type = CNT_WORD;
    386 
    387                     // Insert the node
    388                     if(parent)
    389                     {
    390                         node->parent = parent;
    391                         parent->children.push_back(node);
    392                     }
    393                     else
    394                     {
    395                         node->parent = 0;
    396                         nodes->push_back(node);
    397                     }
    398                     node = ConcreteNodePtr();
    399                 }
    400                 break;
    401             }
    402 
    403             ++i;
    404         }
    405 
    406         return nodes;
    407     }

          可以看出,“语义分析”的输入数据是ScriptTokenList,其输出是ConcreteNodeList;整个分析过程用的是,“逐词素分析+状态机”的实现机制,这与“词法分析”机制类似(参见之前的讨论:http://www.cnblogs.com/yzwalkman/archive/2013/01/02/2841607.html)。需要说明的是,按照脚本文件的固有结构,我们一般会自然地想到:如果把左右大括号为限定符的数据作为一个数据单元的话,那么,上面定义中出现的ConcreteNode结构就应该直接对应此数据单元;而ConcreteNodeList就应该是用来保存同一文件中所有数据单元的列表。但从代码中我们可以看到,情况并不是这样的。

         实际上,每一个ConcreteNode对象对应的仍旧是一个词素结点(ScriptToken)对象。parse函数在生成一个ConcreteNode对象后,会直接把相应的ScriptToken中的数据赋值给它(参见24-28行、85-89行等)。然后parse()函数会在此基础上再做两步工作:1.确定此ConcreteNode的子结点;2.确定此ConcreteNode对象的父结点。而父结点与子结点的判定依据就是上面说的——确定了脚本文件固有结构的数据单元间的关系。以材质脚本为例,假设一个脚本文件中定义且只定义了10个材质,那么此脚本在进过语义分析后,相应的ConcreteNodeList就会保存这10个ConcreteNode对象的指针,此容器中每个ConcreteNode对象实际上是相应材质的第一个词素;同时,每个材质内部的数据单元也会按此方法保存。为更清楚说明,可以上面的脚本SdkTrays.material中的材质material SdkTrays/Base为例,SdkTrays.material脚本被解析后,material SdkTrays/Base将被作为第一个数据单元被保存在ConcreteNodeList中(因为它是脚本中第一个被定义的数据单元),而作为此单元根结点的ConcreteNode对应着词素——material,接下来的三个词素SdkTrays/Base和左、右大括号所对应的ConcreteNode对象,将作为三个并列的子结点保存在material结点的子结点列表(结点定义的第25行)中,相应的,这三个子结点的父结点指针(见结点定义的第26行)将指向material结点对象。而紧随其后的technique词素所对应生成的ConcreteNode结点对象,将被作为左括号的第一个子结点被保存,technique结点的父结点指针将指向此左括号对象,其余结点的操作以此类推。

         从代码中可以看到,整个分析过程只在两种READY和OBJECT两种状态中转换(第6行)。在处理完左括号(268-298行)或右括号(299-336行)后,分析会从OBJECT状态转换到READY状态,也就是说,OBJECT状态是用来处理包含在左右括号内的数据单元内部信息的,而READY状态是用来处理,分析完一个数据单元或开始一个新的数据单元之后要做的相关工作的。

         最后比较一下ScriptToken和ConcreteNode的各自可取的type的定义:

     1     enum ConcreteNodeType
     2     {
     3         CNT_VARIABLE,
     4         CNT_VARIABLE_ASSIGN,
     5         CNT_WORD,
     6         CNT_IMPORT,
     7         CNT_QUOTE,
     8         CNT_LBRACE,
     9         CNT_RBRACE,
    10         CNT_COLON
    11     };
    12 
    13     enum{
    14         TID_LBRACKET = 0, // {
    15         TID_RBRACKET, // }
    16         TID_COLON, // :
    17         TID_VARIABLE, // $...
    18         TID_WORD, // *
    19         TID_QUOTE, // "*"
    20         TID_NEWLINE, // \n
    21         TID_UNKNOWN,
    22         TID_END
    23     };

          可以看到它们有对应的项,也有各自独立的项。这是由这两个结点自身所承担的任务来决定的。

    作者:yzwalkman
    转载请注明出处。
  • 相关阅读:
    正则表达式速查表
    Python第三方库管理Anaconda
    Python3.x和Python2.x的区别
    python 学习 “笨办法学python”(随书补充)
    python 中文输入的注意事项
    mongodb update 字符 操作(补充)
    mongodb update 字符 操作
    04.视频播放器通用架构实践
    05.视频播放器内核切换封装
    03.视频播放器Api说明
  • 原文地址:https://www.cnblogs.com/yzwalkman/p/2842904.html
Copyright © 2020-2023  润新知