You are not logged in or registered. Please login or register to use the full functionality of this board...
SIGN IN Join Our Community For FREE


scannerless parsing
06-04-2017, 04:09 AM (This post was last modified: 06-04-2017 04:38 AM by figosdev.)
Post: #1
 (Print Post)
scannerless parsing
wikipedia has an entire article on it.

im not against language complexity when its justified, im against arguments that imply complexity is a necessary ingredient as a rule. these arguments always remind me of the (narrow) views against basic itself; they are subjective and often taken out of context to put things down (like basic itself) without any good reason other than perceived-- and inaccurate-- correlation.

i will never be and could never be donald knuth, however i admire him. just as knuth defended the idea that goto could be used in structured programming (he wrote an entire book on it, while admitting that he wasnt really championing the goto command itself!) i still defend the idea that simple and beginner implementations have as legitimate a place-- in learning, if not everyday use-- as simple and beginner-friendly languages. if the basic community cant accept that, then in my opinion its the basic communitys loss (of direction or context or whatever.)

another pioneer, niklaus wirth proposed a cleaned-up, simple version of algol for algol-x, but it was deemed not complex enough and it left him to create algol-w instead, which lead to pascal. i like a guy who is not only expert, but whose language proposal is rejected for its simplicity.

generally speaking, a programming language implementation will have some kind of parser.

people will then tell you about parsers vs. lexers and scanners, and tokens and passes and optimisation. which is all really great stuff! heres a quote from the wikipedia article with essentially what i have to say about that:

Quote:
In computer science, scannerless parsing (also called lexerless parsing) refers to performing both tokenization (breaking a stream of characters into words) and parsing (arranging the words into phrases) in a single step, rather than breaking it up into a pipeline of a lexer followed by a parser, executing concurrently. It also refers to the associated grammar: using a single formalism to express both the lexical (word level) grammar and phrase level grammar used to parse a language.

https://en.wikipedia.org/wiki/Scannerless_parsing


so you can actually break a stream of characters (a string or a text file) into words and phrases in a single step, or using whatever steps you choose. it continues:

Quote:
Dividing processing up into a lexer followed by a parser is generally viewed as better design because it is more modular, and scannerless parsing is primarily used when a clear lexer–parser distinction is unneeded or unwanted. Examples of when this is appropriate include TeX, most wiki grammars, makefiles, simple per application scripting languages, and Perl 6.

wow! so there are REAL LANGUAGES that have scannerless parsers. very cool. (im a little surprised about perl 6.)

the way most hobbyists and first-year computer science majors seem to view programming language design, you have to do things "the best" to do them at all.


the funny thing is, they think there is one way thats "the best." even wikipedia (which we know is always right, naturally) seems to know this isnt true, as it states very clearly that sometimes, you can get away with this sort of alternative even in perl! (perhaps because most wiki grammars do it too? obvious bias there.)

why do i like this so much?





i like it because id rather there were a lot more people with a basic understanding of language design, rather than every language designer being a (self-professed) expert. id like for language design (like programming in basic once was) to be something that can be made simple enough for everyone. while other beginners are using tedious things like javascript (an increasingly powerful option for sure) to do everyday tasks, id like for someone out there to design the BASIC of programming language design (a concept ive been writing about for years.) almost no one, if anyone, is trying to make language design itself as easy as basic made learning to program. i believe this is doable.

but to do it, it is necessary to reduce language design to BASIC concepts, rather than stay with the (superficial) philosophy that language design is (like programming was, before basic) something reserved for experts, math majors and cs majors. basic put programming in the hands of everyone-- we have yet to put language design in the hands of everyone, but we are still getting closer. one thing thats helping is free/libre/open-source programming languages like python, freebasic and qb64 (and my favourite basic dialects, smallbasic, basic256 and sdlbasic.)





and the reason for this is many-fold, including that id like more language design ideas explored, more user-friendly ideas tried out, and a wider variety of designs. in the pioneer days, people were not afraid to try things-- they learned a lot, and id rather not see that learning turned into some kind of religious movement. instead, id like people to feel free to do things in the same spirit that the pioneers did.

of course this means some experiments will get repeated. but thats alright too-- a great way to learn is to repeat experiments. and what if new findings result? well that would be very cool, too.

in my experience with self-identified experts, they start with parsing and eventually get to functions. my approach to making python easier for beginners (if possible) was to start with functions-- i never even worried about putting a language on top of it, then months or years later i started doing that for a lark.

part of the reason i did it this way was because **its a lot easier.** but it turns out, this also lets you design a language with multiple parsers for different types of language-- you could create a web or block interface to the language, or you could create a whitespace or brace-based syntax on top of it. and before i learned that "trick" from watching a real cs class on language design, i figured it out for myself.

if you dont spend so much time worrying about writing the perfect parser (a component which can be be built a variety of ways, including reusing one from a free software language like fb, sb or qb64-- or using a parser generator, or simply writing your own) and making the parser modular, then you can make the ENTIRE LANGUAGE modular to the point where it can have various parsers.

and best of all, thats the easy way! you may even find that instead of making a full language with a parser, your happy to make a library of functions and just call them from your language (using includes or imports) and use your favourite language as the "parser."

if the language is modular, the parser can be simple and replaced with something more sophisticated later. so you can write your own little parser first, and then go and buy (or build) yourself a big fancy "best" parser for your language.

its easier and best of all-- it will stay simple until you do actually NEED it to be more complex. (unlike algol, perhaps.)

i also strongly believe that if we are going to have people trying out very new ideas with their designs, that this pseudo-orthodoxy thats been established is simply a way of ensuring that designs will repeat themselves in a relatively narrow way.

superficially, you will have differences, but people will be afraid to try the newest or weirdest ideas, because "it isnt done that way." well so what? there are probably enough people doing it "the best way" to try out most or all of your own ideas, if you have some. you can always make your second language "the right way" if you need to.

perhaps after youve made that toy language you never thought youd do for fun, youll go take a university course for language design. or-- since youve always coded for fun, perhaps you will just keep making toy languages for fun and helping other people do the same. i mean, we werent all planning to go work for microsoft or a big video game publisher-- or were we? is that the only reason to learn basic now? to become a professional coder?
Find all posts by this user
Like Post
The following 2 users Like figosdev's post:
Aurel, Waltersmind (Admin)
06-04-2017, 02:47 PM
Post: #2
 (Print Post)
RE: scannerless parsing

.7z  tascript.7z (Size: 24.12 KB / Downloads: 11)
My hobby language may fall into this category, I dust it off now and then and play with it but I set out to make something without all the "look ahead" buffers  and using or putting back characters so it processes the entire line and acts on it.  Also not sure I added the attachment where it belongs.  Not unusual.

phred
Find all posts by this user
Like Post
The following 1 user Likes phred's post:
figosdev
06-04-2017, 03:22 PM
Post: #3
 (Print Post)
RE: scannerless parsing
phred,

Welcome to the community.

Please help support The Joyful Programmer and The QB64 Edition by visiting our online store provided by Amazon.com. We hand-picked books related to computer programming from Amazon.com and added them to our store. When you make a purchase from our store, we do receive a small commission from the sale. Visit our store at: http://www.thejoyfulprogrammer.com/qb64/...azon-store
Find all posts by this user
Like Post
06-04-2017, 03:33 PM (This post was last modified: 06-04-2017 03:35 PM by figosdev.)
Post: #4
 (Print Post)
RE: scannerless parsing
i have edited your program so it shows what i think is the basic structure of the main features. for those who are reading this post first, the full version is here: http://www.thejoyfulprogrammer.com/qb64/...p?aid=1469

of interest to me is that it starts more or less like i would expect a hobby language to, with what appears to be a very simple parser, and then at the bottom you get (what i consider) fancy with recursive parsing. theres nothing wrong with that at all, if we could all do it that would be cool. i think thats the most difficult part of writing a language like this, and i managed to avoid it in my own language. yours looks easy enough to take apart and repurpose, which is nice.

Code Snippet: [Select]
OPEN filename$ FOR INPUT AS #ff
FOR j = 1 TO linecount 'do a little error checking and label table building
  LINE INPUT #ff, cmdarray$(j) 'read in each script line and pre-process
  cmdarray$(j) = LTRIM$(cmdarray$(j)) 'in the event there are leading spaces
  IF cmdarray$(j) = "" THEN cmdarray$(j) = "$rem " 'handle blank lines by nmaking them remarks rather than errors
  IF LEFT$(cmdarray$(j), 1) <> "$" THEN Hmc 5, j, temp$ 'make sure there's a command

  cmdend = INSTR(cmdarray$(j), " ")
  IF cmdend = 0 THEN 'if there's no space after a single word command tack one on for parsing
    cmdarray$(j) = cmdarray$(j) + " "
    cmdend = INSTR(cmdarray$(j), " ")
  END IF
  cmd$ = LCASE$(LEFT$(cmdarray$(j), cmdend - 1)) 'pick off the '$' and the command

  IF cmd$ = "$end" THEN endfound = TRUE
  IF cmd$ = "$label" THEN 'build a table of label locations
    name$ = trim$(LCASE$(MID$(cmdarray$(j), cmdend + 1)))
    IF name$ = "" THEN Hmc 3, j, "unnamed label" 'no name
    IF LEN(name$) > namelen THEN Hmc 15, j, ""
    FOR k = 1 TO maxlabels
      IF name$ = lblarray$(k, 1) THEN Hmc 8, j, " label already in use on script line " + lblarray$(k, 2) 'name has already been used
    NEXT k
    IF lblcount + 1 > maxlabels THEN
      Hmc 10, j, ""
    ELSE 'if there's space add the label name and location to the list
      lblcount = lblcount + 1
      lblarray$(lblcount, 1) = name$
      lblarray$(lblcount, 2) = LTRIM$(STR$(j))
    END IF
  END IF
NEXT j
CLOSE #ff






sl = 1 'counter for script lines
DO
  temp$ = cmdarray$(sl) 'each line is read into a temporary variable for execution with leading spaces trimmed
  IF trace THEN
    PRINT "    - "; temp$ 'if the line trace feature is enabled print the script line
    IF tt > 0 THEN SLEEP tt
  END IF

  cmdend = INSTR(temp$, " ")
  cmd$ = LCASE$(LEFT$(temp$, cmdend - 1)) 'save the command
  temp$ = LTRIM$(MID$(temp$, cmdend + 1)) 'peel off the command and any spaces for the parameters, etc.

  SELECT CASE cmd$

    CASE "$add", "$div", "$mul", "$sub"
      IF temp$ = "" THEN Hmc 3, sl, ""
      temp$ = LCASE$(temp$)
      cf = INSTR(temp$, ",")
      IF cf = 0 THEN Hmc 3, sl, ""
      p1$ = trim$(MID$(temp$, 1, cf - 1))

      temp$ = trim$(MID$(temp$, cf + 1))
      IF temp$ = "" THEN Hmc 3, sl, ""
      cf = INSTR(temp$, ",")
      IF cf = 0 THEN Hmc 3, sl, ""
      p2$ = trim$(MID$(temp$, 1, cf - 1))

      p3$ = trim$(MID$(temp$, cf + 1))
      IF p3$ = "" THEN Hmc 3, sl, ""

      ' variable parameter assignments are handled in solve%
      IF cmd$ = "$add" THEN temp$ = p2$ + "+" + p3$
      IF cmd$ = "$div" THEN temp$ = p2$ + "\" + p3$
      IF cmd$ = "$mul" THEN temp$ = p2$ + "*" + p3$
      IF cmd$ = "$sub" THEN temp$ = p2$ + "-" + p3$
      x = solve%(temp$, sl)
      putvar p1$, LTRIM$(STR$(x)), sl


    CASE "$additem"
      IF temp$ = "" THEN Hmc 3, sl, ""
      cf = INSTR(temp$, ",")
      IF cf = 0 THEN Hmc 3, sl, ""
      p1$ = MID$(temp$, 1, cf - 1) '1st parameter, name of list to add to
      p1$ = stripref$(p1$, sl)
      IF p1$ = "" THEN Hmc 3, sl, ""
      readvar p1$, type$, value$, element, sl 'check if the name is valid
      IF type$ <> "list" THEN Hmc 7, sl, p1$ + " is not a list type"

      p2$ = trim$(MID$(temp$, cf + 1)) '2nd parameter, value to add
      p2$ = vlookup$(p2$, sl)
      addlistitem p1$, p2$, sl


    CASE "$blj", "$bsr" 'branch to subroutine, equivalent to BASIC on..gosub, branch line jump works as on..goto
      IF temp$ = "" THEN Hmc 3, sl, ""
      temp$ = LCASE$(temp$)
      j = 0 'this could possibly be 'short circuited' to jump as soon as the input matches
      done = FALSE
      DO
        j = j + 1
        cf = INSTR(temp$, ",") 'look for a comma
        IF cf = 0 THEN Hmc 3, sl, "" ' added 11/20
        sf = INSTR(temp$, ";") 'semicolon separates jump locations
        IF sf > 0 THEN 'while there are more combinations
          IF cf THEN
            params$(j, 1) = trim$(MID$(temp$, 1, cf - 1)) 'take the expected input
            params$(j, 2) = trim$(MID$(temp$, cf + 1, sf - cf - 1)) 'take the jump location label
            gll$ = getlblloc$(params$(j, 2), sl) 'swap the label for a line number
            IF gll$ > "" THEN 'the name is valid
              params$(j, 2) = gll$ 'replace the label location as the reference
            ELSE 'it wasn't found
              Hmc 9, sl, params$(j, 2)
            END IF
            IF isintstr(params$(j, 2)) = FALSE THEN Hmc 7, sl, params$(j, 2) + " is not an integer" 'it should be an integer
            temp$ = LTRIM$(MID$(temp$, sf + 1))
          END IF
        ELSE 'last one
          params$(j, 1) = RTRIM$(MID$(temp$, 1, cf - 1))
          params$(j, 2) = trim$(MID$(temp$, cf + 1))
          gll$ = getlblloc$(params$(j, 2), sl)
          IF gll$ > "" THEN
            params$(j, 2) = gll$
          ELSE
            Hmc 9, sl, params$(j, 2)
          END IF
          IF isintstr(params$(j, 2)) = FALSE THEN Hmc 7, sl, params$(j, 2) + " is not an integer" 'should never be called
          params$(j, 2) = gll$
          done = TRUE
        END IF
      LOOP WHILE NOT (done)

      IF cmd$ = "$bsr" THEN Push sl, sl

      FOR k = 1 TO j
        IF char$ = params$(k, 1) THEN 'if the input is something expected process it
          sl = VAL(params$(k, 2))
          EXIT FOR
        ELSEIF params$(k, 1) = "*" THEN 'if not default to this location, typically a loop
          sl = VAL(params$(k, 2))
        END IF
      NEXT k


    CASE "$clear" 'clear screen
    CASE "$cmp", "$cmr" 'compare two integer values
      cf = INSTR(temp$, ",") 'first parameter
      cf = INSTR(temp$, ",") 'second parameter
      cf = INSTR(temp$, ",") 'third parameter, label for less than comparison
      cf = INSTR(temp$, ",") 'fourth parameter, label for equal to comparison
      p5$ = trim$(temp$) 'fifth parameter, label for greater than comparison is the last bit of the string
    CASE "$color" 'set text color
      cf = INSTR(temp$, ",")
      IF cf AND LEN(RTRIM$(MID$(temp$, cf + 1))) > 0 THEN 'fore and back colors
        COLOR fg, bg
      ELSE 'just a foreground color
      END IF


    CASE "$concat" 'concatenate two strings, !! may eventually change to multiple string possibility, ie: concat a$, b$, c$, d$
      p2$ = MID$(temp$, cf + 1) 'trim the 1st parameter off of the string
    CASE "$decr", "$incr" 'subtract from or add to a variable

    CASE "$def" 'define a new variable or list
      p1$ = trim$(LCASE$(MID$(temp$, 1, cf - 1))) 'variable name
        p2$ = trim$(LCASE$(MID$(temp$, 1))) 'variable type
        p3$ = "" 'variable value
      addvar p1$, p2$, p3$, sl


    CASE "$edititem" 'edit the contents of a list item
    CASE "$end" 'end script
    CASE "$equ" 'solve an equation
    CASE "$erase" 'move a file to the recycle bin
    CASE "$fget" 'read information from a file
      p1$ = trim$(LCASE$(LEFT$(temp$, cf - 1))) '1st parameter is the file name
        p2$ = trim$(temp$) 'take care of single input or the last variable of the input string
        IF LEN(p2$) = 0 THEN Hmc 3, sl, ""

    CASE "$fput" 'write information to a file
      p1$ = trim$(LEFT$(temp$, cf - 1)) '1st parameter is the file name
      p2$ = trim$(LEFT$(temp$, cf - 1)) '2nd parameter is mode

    CASE "$getchar" 'wait for user input, input is captured as an ascii code and returned in 'kee'
    CASE "$getcursorx"
    CASE "$getcursory"
    CASE "$getline" 'input a line of text
      DO 'if expecting integer input loop until it is supplied
          readvar p1$, p1type$, value$, element, sl 'make sure it's a variable and check the type
          readvar p1$, p1type$, value$, element, sl 'make sure it's a variable and check the type

    CASE "$gosub", "$jmp" 'sub or jump to a label
    CASE "$goto" 'mov to any script line, not just a label
    CASE "$insert" 'insert a string into another string in the designated position
    CASE "$label" 'branch label
    CASE "$lcase", "$ucase" 'returns a string in all lowercase
    CASE "$len" 'copy the length of a string value to an integer variable
    CASE "$lpos" 'return the last position of a search string within a string
    CASE "$ltrim", "$rtrim", "$trim" 'trims beginning and ending spaces from a string
    CASE "$mid" 'returns a portion of a string
    CASE "$mov" 'replaces a variable value
    CASE "$ord" 'the ascii code (ordinal) of a character
    CASE "$pos" 'position of one string in another
    CASE "$random" 'generate a random number
    CASE "$readitem"
    CASE "$rem" 'remark, no operation occurs
    CASE "$repeat" 'create a string of chars, n chars long, !! may change to include a repeated string of more than one character
    CASE "$replace" 'replace a substring within a string with options for starting point, number of replacements and case sensitivty
      p1$ = trim$(LEFT$(temp$, cf - 1)) 'first param, expression contaiming substring to replace.
      p2$ = trim$(LEFT$(temp$, cf - 1)) 'second param, substring being searched for.
        p3$ = trim$(LEFT$(temp$, cf - 1)) 'third param, Replacement substring.
      'optional parameters
        p4$ = trim$(LEFT$(temp$, cf - 1)) 'fourth param, Position within expression where substring search is to begin. If omitted, 1 is assumed.
    CASE "$return" 'return from subroutine
    CASE "$rev" 'returns a reversed string
    CASE "$say" 'print to the screen
    CASE "$scmp", "$scmr" 'compare two string values
    CASE "$sco" 'show character output after ask commands
    CASE "$setcursor" 'place cursor at a particluar row and column
    CASE "$sound" 'make a sound
    CASE "$trc" 'set line trace on or off
    CASE "$ucount"
    CASE "$val" 'returns the integer value of a string
    CASE "$wait"
    CASE "$wrap" 'need text$, max width, indent & color
      IF cf = 0 THEN 'just one parameter, only width was given
      ELSE 'width and indent
      END IF
      wordwrap value$, p2, p3, sl
    CASE ELSE 'command wasn't found
      Hmc 4, sl, ""
  END SELECT

  '  exitselect:
  sl = sl + 1 'increment to the next script line
LOOP UNTIL sl > linecount
END SUB


SUB addlistitem (llist$, newitem$, curline)
END SUB

SUB addvar (name$, type$, value$, curline) 'create a new variable
END SUB

SUB editlistitem (llist$, entry$, index, curline)
END SUB

FUNCTION getlblloc$ (name$, curline) 'returns the line number of a label
END FUNCTION

SUB Hmc (errno, curline, extra$) 'error handler
END SUB

FUNCTION isdigit (char$)
END FUNCTION

FUNCTION isintstr (charstr$) 'checks for a valid integer, 0-9 & "-" for negatives numbers
END FUNCTION

FUNCTION isvar (name$, curline) 'determine if the variable name is valid
END FUNCTION

FUNCTION keycode
END FUNCTION

FUNCTION listmax (llist$, curline)
END FUNCTION

FUNCTION Pop (curline)
END FUNCTION

SUB Push (num, curline) 'places a number on the stack, typically the return address from a sub call
END SUB

SUB putvar (name$, value$, curline) 'replace a variable value
END SUB

FUNCTION readlistitem$ (llist$, index, curline)
END FUNCTION

SUB readvar (name$, type$, value$, element, curline) 'get the type and/or value of a variable
END SUB

FUNCTION replacesub$ (expression$, find$, replacewith$, start$, count$, compare$)
END FUNCTION

FUNCTION reverse$ (char$)
END FUNCTION

FUNCTION revinstr (char$, srch$) 'returns the last position of a substring in a string
END FUNCTION

FUNCTION SendToRecycleBin& (filename$)
END FUNCTION

FUNCTION stripref$ (char$, curline) ' removes value reference braces - []
END FUNCTION

FUNCTION trim$ (txt$)
END FUNCTION

FUNCTION vlookup$ (char$, curline) 'check for variables in a string and replace with the value
END FUNCTION

SUB wordwrap (test$, maxcol, indent, curline)
END SUB

'*********************************
'a simple recursive descent parser
'based on Jack Crenshaws tutorial
'*********************************

FUNCTION solve% (eval$, curline)
'replace variables with values
eval$ = vlookup$(eval$, curline)
'test validity of the equation
eval$ = isequation$(eval$, curline)
'start the process
getchar eval$, char$
'begin evaluation
isexpr eval$, char$, value%, curline
solve% = value%
END FUNCTION

SUB getchar (eval$, char$)
eval$ = LTRIM$(eval$)
IF eval$ <> "" THEN
  'if there's a string grab the first character
  char$ = LEFT$(eval$, 1)
  'and remove it from the sequence
  eval$ = MID$(eval$, 2)
ELSE
  char$ = " "
END IF
END SUB

SUB getdigits (eval$, char$, value%)
IF eval$ <> "" THEN
  'while the characters are numbers collect them
  WHILE isdigit(char$) 'char$ >= "0" AND char$ <= "9"
    number$ = number$ + char$
    getchar eval$, char$
  WEND
ELSE
  number$ = char$
END IF
value% = VAL(number$)
END SUB

SUB isexpr (eval$, char$, value%, curline)
'is it a valid term?
isterm eval$, char$, value%, curline
WHILE INSTR("+-", char$)
  op$ = char$
  getchar eval$, char$
  isterm eval$, char$, value2%, curline
  IF op$ = "+" THEN
    value% = value% + value2%
  ELSE
    value% = value% - value2%
  END IF
WEND
END SUB

SUB isterm (eval$, char$, value%, curline)
'is it a valid factor?
isfactor eval$, char$, value%
WHILE INSTR("*\/", char$)
  op$ = char$
  getchar eval$, char$
  isfactor eval$, char$, value2%
  IF op$ = "*" THEN
    value% = value% * value2%
  ELSE
    IF value2% = 0 THEN Hmc 17, curline, value2$
    value% = value% \ value2% 'decimal division is changed to integer
  END IF
WEND
END SUB

SUB isfactor (eval$, char$, value%)
'is it a partial formula in braces?
IF char$ = "(" THEN
  getchar eval$, char$
  isexpr eval$, char$, value%, curline
  getchar eval$, char$
ELSE
  getdigits eval$, char$, value%
END IF
END SUB

'*********************************
Find all posts by this user
Like Post
06-04-2017, 04:35 PM (This post was last modified: 06-04-2017 04:35 PM by Anthony.R.Brown.)
Post: #5
 (Print Post)
RE: scannerless parsing
i have edited your program so it shows what i think is the basic structure of the main features. for those who are reading this post first, the full version is here: http://www.thejoyfulprogrammer.com/qb64/...p?aid=1469

figosdev 

What right do you have to Edit his Post ?

Are you trying to make him look like you! ?
Find all posts by this user
Like Post
06-04-2017, 04:40 PM (This post was last modified: 06-04-2017 04:42 PM by figosdev.)
Post: #6
 (Print Post)
RE: scannerless parsing
as i said in plain english, ive edited a program-- a second COPY of a program-- not a post or attachment.

ive also explained why.

if anyone whos *the author of that program* is offended i hope they will let me know so i can do something to fix it. sometimes its difficult to post on this forum without getting a bunch of crap for no reason, but if my edit actually does any harm (the version i posted wont likely work and that isnt why it was posted) i would like to know.
Find all posts by this user
Like Post
06-04-2017, 05:10 PM
Post: #7
 (Print Post)
RE: scannerless parsing
Anthony,

FigOsDev did not modify any posts at this point. However, you asked him what right he had to modify a post. Well, I gave him the right a few weeks back when I made him a moderator.

Let me make this clear to all that FigOsDev now has the right, per my permission, to modify and/or unapprove posts in accordance with the Terms & conditions of this forum. If anyone has any issues with this, please send me a PM (Private Message) where we can discuss it in private.

Please help support The Joyful Programmer and The QB64 Edition by visiting our online store provided by Amazon.com. We hand-picked books related to computer programming from Amazon.com and added them to our store. When you make a purchase from our store, we do receive a small commission from the sale. Visit our store at: http://www.thejoyfulprogrammer.com/qb64/...azon-store
Find all posts by this user
Like Post
06-04-2017, 06:11 PM
Post: #8
 (Print Post)
RE: scannerless parsing
FIG
do you know that Aurel Basic is scannerless dialect of basic?
maybe you know but probably you forget...
Even is written on such a primitive way it is full completed language.
But this way have one big problem which is speed of execution.
simply because each operation require constant parsing.
Ruben is let say semi-tokenizer and is little bit faster.
In my next PL i would like to write completely tokeized interpreter,no bytecode
toy is a bytecode and is really fast but because is not my work and because
conflict with one ultra expert about toy -internals i abandoned that project.

yes i like phred scripting ..is this written in free basic or qb64?
it uses siilar method i have in Aurel Basic + some methods from Ruben
all in all nice Wink

about post modifications ..please let it as is in original !
you know troubles with overmoderated post on former bp.org
by our friend bplus....we almost fight about that .

basicPro forum:
http://basicpro.mipropia.com/smf/index.php
EU Radioboard forum:
http://euradioboard.createmybb3.com/index.php
AurelSoft main site:
http://aurelsoft.ucoz.com
Find all posts by this user
Like Post
06-04-2017, 06:21 PM
Post: #9
 (Print Post)
RE: scannerless parsing
Quote:
do you know that Aurel Basic is scannerless dialect of basic?

i did not know that.


Quote:
But this way have one big problem which is speed of execution.
simply because each operation require constant parsing.

not a problem for a source compiler, but as aurel basic is an interpreter, this is a good point.

note with compiler vs. interpreter i think both are alright-- fig is a compiler, python is an interpreter--

both have good and less good points.


Quote:
Ruben is let say semi-tokenizer and is little bit faster.

cool.


Quote:
yes i like phred scripting ..is this written in free basic or qb64?

i found underscores, so im pretty sure its qb64.


Quote:
about post modifications ..please let it as is in original !

i have edited 0 posts so far. the powers ive been granted ive had for 4 or 5 days, thereabouts.


Quote:
you know troubles with overmoderated post on former bp.org

walt is much more likely to edit a post than i am-- and faster.
Find all posts by this user
Like Post
06-05-2017, 11:00 AM
Post: #10
 (Print Post)
RE: scannerless parsing
It doesn't bother me at all for an edited version of my toy language to be posted, especially when it's done for the purpose of clarity.  Also, I made it clear at the top of the code it could be used in any way.  Actually I'm sort of flattered people are discussing it.
It was (is) being written in QB64 but as I mentioned in the documentation  I've tried to stick to QB syntax as much as possible in the event I ever wanted to compile it in another flavor of BASIC. 
Thanks to everyone interested!
I've attached a game written in the language, I haven't done a run through lately but I don't think I've broken anything in it.  Compile the language and drag and drop the game on it.

phred.


.7z  nightfall13.7z (Size: 5.23 KB / Downloads: 7)
Find all posts by this user
Like Post
The following 2 users Like phred's post:
Aurel, figosdev



Forum Jump:


User(s) browsing this thread: 1 Guest(s)




QB64 Member Project - OpenGL Triangles
QB64 Member Project - Score 4
QB64 Member Project - Rotating Background
QB64 Member Project - Splatter
QB64 Member Project - Point Blank
QB64 Member Project - STxAxTIC 3D World
QB64 Member Project - Martin Fractals version two
QB64 Member Project - Exit
QB64 Member Project - Martin Fractals version four
QB64 Member Project - MAPTRIANGLE
QB64 Member Project - Othello
QB64 Member Project - Dakapo
QB64 Member Project - Pivet version one
QB64 Member Project - Kobolts Monopoly
QB64 Member Project - Touche
QB64 Member Project - Overboard
QB64 Member Project - Spiro Roses
QB64 Member Project - Kings Valley verion one
QB64 Member Project - Amazon
QB64 Member Project - Connect Four
QB64 Member Project - Martin Fractals version one
QB64 Member Project - ARB Checkers
QB64 Member Project - Inside Moves
QB64 Member Project - Sabotage
QB64 Member Project - Dreamy Clock
QB64 Member Project - Input
QB64 Member Project - Kings Court
QB64 Member Project - RGB Color Wheel
QB64 Member Project - Quarto
QB64 Member Project - Qubic
QB64 Member Project - Foursight
QB64 Member Project - Martin Fractals version three
QB64 Member Project - Bowditch curve
QB64 Member Project - Red Scrolling LED Sign
QB64 Member Project - Kings Vallery version two
QB64 Member Project - Spinning Color Wheel
QB64 Member Project - Pivot version two
QB64 Member Project - Domain
QB64 Member Project - Swirl
QB64 Member Project - Color Triangles
QB64 Member Project - Color Rotating Text
QB64 Member Project - Algeria Weather
QB64 Member Project - Line Thickness
QB64 Member Project - 9 Board
QB64 Member Project - Blokus
QB64 Member Project - Isolation
QB64 Member Project - Full Color LED Sign
QB64 Member Project - Rubix's Magic
QB64 Member Project - Basic Dithering