You are not logged in or registered. Please login or register to use the full functionality of this board...
SIGN IN Join Our Community For FREE


Nested EOF() Loops reading multiple files
09-07-2017, 03:35 PM
Post: #31
 (Print Post)
RE: Nested EOF() Loops reading multiple files
Give this a quick run and see if it doesn't meet your requirements:

Code Snippet: [Select]
DIM file(3) AS STRING
file(1) = "temp1.txt": file(2) = "temp2.txt": file(3) = "temp3.txt"

'initial data
OPEN file(1) FOR OUTPUT AS #1: PRINT #1, 2: PRINT #1, 4: PRINT #1, 5: PRINT #1, 7: PRINT #1, 8: PRINT #1, 9: PRINT #1, 10: CLOSE
OPEN file(2) FOR OUTPUT AS #1: PRINT #1, 1: PRINT #1, 3: PRINT #1, 4: PRINT #1, 6: PRINT #1, 8
CLOSE

'open files to merge
OPEN file(1) FOR INPUT AS #1
OPEN file(2) FOR INPUT AS #2
OPEN file(3) FOR OUTPUT AS #3

INPUT #1, num1: INPUT #2, num2 'start with a seed value

DO UNTIL EOF(1) OR EOF(2) 'merge until one file runs out of entries
    IF num1 < num2 THEN
        DO UNTIL EOF(1)
            IF num1 <> lastnum THEN PRINT #3, num1: lastnum = num1
            INPUT #1, num1
            IF num1 > num2 THEN EXIT DO
        LOOP
        IF EOF(1) THEN EndFlag = 1
    ELSE
        DO UNTIL EOF(2)
            IF num2 <> lastnum THEN PRINT #3, num2: lastnum = num2
            INPUT #2, num2
            IF num2 > num1 THEN EXIT DO
        LOOP
        IF EOF(2) THEN EndFlag = 2
    END IF
LOOP

IF EndFlag = 1 THEN 'merge what's left in the other file, after the first one ran out of entries.
    DO
        IF EOF(2) THEN finish = -1
        IF num2 < num1 THEN
            IF num2 <> lastnum THEN PRINT #3, num2: lastnum = num2
            INPUT #2, num2
        ELSE
            IF num1 <> lastnum THEN PRINT #3, num1
            IF num2 <> lastnum THEN PRINT #3, num2
            DO UNTIL EOF(2)
                INPUT #2, num2
                PRINT #3, num2
            LOOP
            finish = -1
        END IF
    LOOP UNTIL finish
ELSE
    DO
        IF EOF(1) THEN finish = -1
        IF num1 < num2 THEN
            IF num1 <> lastnum THEN PRINT #3, num1: lastnum = num1
            INPUT #1, num1
        ELSE
            IF num2 <> lastnum THEN PRINT #3, num2: lastnum = num2
            IF num1 <> lastnum THEN PRINT #3, num1
            DO UNTIL EOF(1)
                INPUT #1, num1
                PRINT #3, num1
            LOOP
            finish = -1
        END IF
    LOOP UNTIL finish
END IF

CLOSE


'just a little routine to display results
FOR i = 1 TO 3
    OPEN file(i) FOR INPUT AS i
NEXT

FOR i = 1 TO 3
    PRINT "FILE "; i
    DO UNTIL EOF(i)
        INPUT #i, num
        PRINT num,
    LOOP
    PRINT
NEXT

This is what's known as an external merge routine, and it's actually not that hard to do.

Basically, open your two (or more) files.
Read them in once to get initial "seed" values.

DO
   Print the lowest value one to the disk first.
  Then read the file that had the lowest value again (unless EOF).  Compare to the next lowest value.
LOOP until you reach the End of File for whichever one runs out of entries first.

(NOTE: Filesize DOES NOT mean the largest file size will run out of data first.  Take these 2 files for example -- File 1: "1, 100".   File 2:  "2, 3, 4, 5, 6, 7, 8, 9, 10")

DO
   Print the rest of the data from the file which was left over, once the dual merge was finished.
LOOP

Finish
Find all posts by this user
Like Post
09-07-2017, 05:53 PM
Post: #32
 (Print Post)
RE: Nested EOF() Loops reading multiple files
Personally, instead of EOF, I think I'd just use LOF and SEEK to get this type of job done easily for me.  Take a look at the following program, which you can set to use as many files as you want to merge (they're pre-sorted as you already mentioned).  Currently it's set to work with 5 files, but I don't foresee where it'd have issues if it ran for hundreds of files at a time.

Code Snippet: [Select]
SCREEN _NEWIMAGE(800, 600, 32)
_SCREENMOVE _MIDDLE

CONST NumberOfFiles = 5
DIM SHARED file(NumberOfFiles) AS STRING


InitFiles
MergeFiles
'CleanDrive   'Uncomment this line and run the program again to clean up the hard drive


SUB InitFiles
RANDOMIZE TIMER
FOR i = 1 TO NumberOfFiles
    file(i) = "temp" + STR$(i) + ".txt"
    OPEN file(i) FOR OUTPUT AS #i
    COLOR &HFFFF0000
    PRINT "FILE #"; i; ":";
    COLOR -1
    CN = 0
    DO UNTIL CN = 100
        CN = INT(RND * (100 - CN)) + CN + 1
        PRINT #i, CN
        PRINT CN;
    LOOP
    PRINT
NEXT
CLOSE
END SUB

SUB MergeFiles
DIM CurrentPosition(NumberOfFiles) AS LONG

FOR i = 1 TO NumberOfFiles 'prepare the files to be read and merged
    OPEN file(i) FOR INPUT AS #i
NEXT

OutFile = NumberOfFiles + 1
OPEN "merge.txt" FOR OUTPUT AS #OutFile

PRINT "MERGED FILE:"
DO
    FOR i = 1 TO NumberOfFiles
        CurrentPosition(i) = SEEK(i)
        IF CurrentPosition(i) < LOF(i) THEN
            INPUT #i, Num
            IF i = 1 THEN lowestnum = Num
            IF Num <= lowestnum THEN
                lowestnum = Num
                FOR j = 1 TO i - 1: SEEK j, CurrentPosition(j): NEXT
            ELSE
                SEEK i, CurrentPosition(i)
            END IF
        END IF
    NEXT
    IF lowestnum <> currentnum THEN
        PRINT #OutFile, lowestnum
        PRINT lowestnum;
    END IF
    currentnum = lowestnum
    finish = -1
    FOR i = 1 TO NumberOfFiles
        IF NOT EOF(i) THEN finish = 0: EXIT FOR
    NEXT
LOOP UNTIL finish
PRINT
CLOSE
END SUB

SUB CleanDrive
FOR i = 1 TO NumberOfFiles 'prepare the files to be read and merged
    KILL file(i)
NEXT
KILL "merge.txt"
END SUB

Code is broken into 3 distinct subroutines to make it easier to follow and understand:
InitFiles -- this just creates the initial files
MergeFiles -- this opens those and then merges them into "merge.txt", eliminating duplicates.
CleanDrive -- uncomment this line to clean those files off your drive so they don't take up space or clutter your folder.
Find all posts by this user
Like Post



Forum Jump:


User(s) browsing this thread: 1 Guest(s)




QB64 Member Project - Color Rotating Text
QB64 Member Project - Othello
QB64 Member Project - Bowditch curve
QB64 Member Project - Dakapo
QB64 Member Project - Full Color LED Sign
QB64 Member Project - Point Blank
QB64 Member Project - Kings Court
QB64 Member Project - Domain
QB64 Member Project - Touche
QB64 Member Project - Dreamy Clock
QB64 Member Project - Martin Fractals version three
QB64 Member Project - Inside Moves
QB64 Member Project - OpenGL Triangles
QB64 Member Project - Basic Dithering
QB64 Member Project - Swirl
QB64 Member Project - Algeria Weather
QB64 Member Project - Spiro Roses
QB64 Member Project - Martin Fractals version one
QB64 Member Project - Sabotage
QB64 Member Project - Martin Fractals version four
QB64 Member Project - Kings Vallery version two
QB64 Member Project - MAPTRIANGLE
QB64 Member Project - Pivot version two
QB64 Member Project - Rotating Background
QB64 Member Project - Quarto
QB64 Member Project - Amazon
QB64 Member Project - Kings Valley verion one
QB64 Member Project - Qubic
QB64 Member Project - 9 Board
QB64 Member Project - Overboard
QB64 Member Project - Line Thickness
QB64 Member Project - Red Scrolling LED Sign
QB64 Member Project - Isolation
QB64 Member Project - Connect Four
QB64 Member Project - Spinning Color Wheel
QB64 Member Project - Rubix's Magic
QB64 Member Project - Exit
QB64 Member Project - Score 4
QB64 Member Project - Pivet version one
QB64 Member Project - Foursight
QB64 Member Project - ARB Checkers
QB64 Member Project - Blokus
QB64 Member Project - Martin Fractals version two
QB64 Member Project - Color Triangles
QB64 Member Project - Input
QB64 Member Project - Kobolts Monopoly
QB64 Member Project - RGB Color Wheel
QB64 Member Project - Splatter
QB64 Member Project - STxAxTIC 3D World