mardi 21 juin 2016

tokenizing list of ply

# List of token names.   This is always required
tokens = (
   'INTEGER',
   'PLUS',
   'MINUS',
   'TIMES',
   'DIVIDE',
   'LPAREN',
   'RPAREN',
   'REAL',
   'STRING',
   'LIST'
)

# Regular expression rules for simple tokens
t_PLUS    = r'+'
t_MINUS   = r'-'
t_TIMES   = r'*'
t_DIVIDE  = r'/'
t_LPAREN  = r'('
t_RPAREN  = r')'
t_STRING  = r'"[^"]*"'
t_LIST    = r'([s*("[^"]*")?(s*,s*"[^"]*"s*)*]) | ([s*d*s*(s*,s*d*s*)*]) | ([s*(d*(.d | d.)d*)?(s*,s*d*(.d | d.)d*s*)*])'

# A regular expression rule with some action code
#def t_LIST(t):
def t_REAL(t):
    r'd*(.d | d.)d*'
    try:
         t.value = float(t.value)    
    except ValueError:
         print ("Line %d: Real %s is too large!" % (t.lineno,t.value))
         t.value = 0
    return t

def t_INTEGER(t):
    r'd+'
    try:
         t.value = int(t.value)    
    except ValueError:
         print ("Line %d: Integer %s is too large!" % (t.lineno,t.value))
         t.value = 0
    return t

For types of list, I consider there are 3 types:

  1. int [s*d*s*(s*,s*d*s*)*]

  2. real [s*(d*(.d | d.)d*)?(s*,s*d*(.d | d.)d*s*)*]

  3. string [s*("[^"]")?(s,s*"[^"]"s)*]

For the int and real numbers list:

How can I convert each element(string) to int or float in the list?

Just like the function real and integer.

Something like

def t_List(t):
  if string list:
     return t
  elif int list:
     for i in t:
         convert each i to int(i)
     return t
  elif real list:
     for i in t:
         convert each i to float(i)
     return t

Also, it this the correct to tokenize the list?

Here are the examples of list:

[ "a" , "bcd" , "ef" ]

[ 1111 , 2333 , 33 ]

[ 1.23, 3.14 , 5.01 ]

Aucun commentaire:

Enregistrer un commentaire