How to Read csv Files with both Characters and Numbers?
  Home FAQ Contact Sign in
comp.lang.fortran only
 
Advanced search
POPULAR GROUPS

more...

comp.lang.fortran Profile…
 Up
How to Read csv Files with both Characters and Numbers?         


Author: qquito
Date: Mar 23, 2008 23:28

Dear Everyone:

I use Fortran 90 and need to read data from thousands of CSV files.
Each line of the data begins with DATE and TIME followed by decimal
numbers separated by commas, and the they look like

1991-01-01,1:00,23.0,34.21,100.73
1991-01-02,2:25,15.9,90.2,74.31
1991-02-23,14:23,39.23,11.90,0.0
......

As you can see, while the DATES are of the same length, the TIMES
(hour and minute separated by a colon) are NOT. Though each line has
the SAME number of fields, including the DATE and TIME.

Could you suggest a simple loop to read ALL the decimal numbers?

Thank you for reading and replying!

--Roland
12 Comments
Re: How to Read csv Files with both Characters and Numbers?         


Author: glen herrmannsfeldt
Date: Mar 24, 2008 00:32

qquito wrote:
> I use Fortran 90 and need to read data from thousands of CSV files.
> Each line of the data begins with DATE and TIME followed by decimal
> numbers separated by commas, and the they look like
> 1991-01-01,1:00,23.0,34.21,100.73
> 1991-01-02,2:25,15.9,90.2,74.31
> 1991-02-23,14:23,39.23,11.90,0.0
(snip)
> Could you suggest a simple loop to read ALL the decimal numbers?

This looks like it could be read using normal list-directed
input. The date and time into CHARACTER variables, the rest
into REAL variables (most likely) or CHARACTER if desired.

CHARACTER*20 DATE,TIME
REAL X,Y,Z

with the following will read one line of the file:

READ(UNIT,*) DATE,TIME,X,Y,Z

The actual loop form depends on what you want to do with the
data after reading it in. If you can process each line
before reading the next line, just use the variables as shown:
Show full article (1.64Kb)
no comments
Re: How to Read csv Files with both Characters and Numbers?         


Author: Kurt Kallblad
Date: Mar 24, 2008 02:10

"glen herrmannsfeldt" ugcs.caltech.edu> wrote in message
news:--idnd0ZUIJFxXranZ2dnUVZ_t6onZ2d@comcast.com...
> qquito wrote:
>
>> I use Fortran 90 and need to read data from thousands of CSV files.
>> Each line of the data begins with DATE and TIME followed by decimal
>> numbers separated by commas, and the they look like
>
>> 1991-01-01,1:00,23.0,34.21,100.73
>> 1991-01-02,2:25,15.9,90.2,74.31
>> 1991-02-23,14:23,39.23,11.90,0.0
> (snip)
>
>> Could you suggest a simple loop to read ALL the decimal numbers?
>
> This looks like it could be read using normal list-directed
> input. The date and time into CHARACTER variables, the rest
> into REAL variables (most likely) or CHARACTER if desired.
>
> CHARACTER*20 DATE,TIME ...
Show full article (3.39Kb)
no comments
Re: How to Read csv Files with both Characters and Numbers?         


Author: nospam
Date: Mar 24, 2008 02:26

glen herrmannsfeldt ugcs.caltech.edu> wrote:
> DO WHILE(.TRUE.)
...
> ENDDO

As an aside, please note that the above C idiom can be written in
Fortran as simply

DO
...
ENDDO

The while(.true.) bit reminds me somewhat of other redundant constructs
such as

if (logical_variable .eqv. .true.)

--
Richard Maine | Good judgement comes from experience;
email: last name at domain . net | experience comes from bad judgement.
domain: summertriangle | -- Mark Twain
no comments
Re: How to Read csv Files with both Characters and Numbers?         


Author: glen herrmannsfeldt
Date: Mar 24, 2008 10:28

Richard Maine wrote:
> glen herrmannsfeldt ugcs.caltech.edu> wrote:
>> DO WHILE(.TRUE.)
>> ENDDO
> As an aside, please note that the above C idiom can be written in
> Fortran as simply
> DO
> ...
> ENDDO

I used to know that, but didn't think of it last night. Though
the actual C idiom is: for(;;) (A for loop with no initialization,
test, or increment). For PL/I, a plain DO group is executed
once (used for grouping, similar to the C { }). PL/I has
DO I=1 BY 1; (an increment but no test loop). I almost wrote

DO I=1,100000000

for the post yesterday, with the assumption that a count would
come in useful. That might not be enough for the OP, though.
> The while(.true.) bit reminds me somewhat of other
> redundant constructs such as
Show full article (1.16Kb)
no comments
Re: How to Read csv Files with both Characters and Numbers?         


Author: glen herrmannsfeldt
Date: Mar 24, 2008 10:37

Kurt Kallblad wrote:
> "glen herrmannsfeldt" wrote
>>qquito wrote:
(snip)
>>>Could you suggest a simple loop to read ALL the decimal numbers?
(snip)
>> READ(UNIT,*) DATE,TIME,X,Y,Z
(snip)
> The data seems to come from a data logger and with thousands of files there
> are almost sure that some records have errors. E.g. missing values will lead
> the list-directed input to read the date in next record as a real number and
> you will have problems. Thus I prefer to read each line into a character
> buffer and then examine the buffer.

I had thought of that, but the OP did ask for a simple loop. That one
doesn't seem so simple. It should fail with a conversion error if the
date is read as a real number. One could add the ERR= test onto the
READ for it.

-- glen
no comments
Re: How to Read csv Files with both Characters and Numbers?         


Author: qquito
Date: Mar 24, 2008 19:24

Dear All:

Thank you very much for your answers! My code is working perfectly
now.

Have a nice day!

--Roland
no comments
Re: How to Read csv Files with both Characters and Numbers?         


Author: relaxmike
Date: Mar 27, 2008 06:18

Hi,
All the ideas here are interesting, but I would have chosen another
way. In other languages, one could use two string manipulations
tools which would make the current problem very simple to solve :

- string_split(string,chars,nbcomponent,component) splits
the given string everytime one character in the chars argument is
found and add it to the array of string "component".

- string_is(string,class) returns false or true if the
given string is an element of the given class, with class
is a string chosen in the list : "integer", "character", string",
"real", "double", etc...

With these basic tools, it would be easy to do an algorithm like
this (in pseudo-code):

foreach line
split the line at each ";" with string_split
foreach component
compute the class of the component with string_is
check that the current component has an acceptable value
process it
Show full article (1.03Kb)
no comments
Re: How to Read csv Files with both Characters and Numbers?         


Author: nospam
Date: Mar 27, 2008 08:33

relaxmike gmail.com> wrote:
> - string_split(string,chars,nbcomponent,component) splits
> the given string everytime one character in the chars argument is
> found and add it to the array of string "component".
>
> - string_is(string,class) returns false or true if the
> given string is an element of the given class, with class
> is a string chosen in the list : "integer", "character", string",
> "real", "double", etc...
...
> Of course, the basic blocks string_split and string_is do not
> exist, therefore one would have to develop it...

...which isn't to hard to do. I've done simillar string utility routines
myself and I use them all the time. The interface to my procedures isn't
exactly as above, but they do follow the basic idea of having string
utility procedures that get used for parsing tasks.
Show full article (1.57Kb)
no comments
Re: How to Read csv Files with both Characters and Numbers?         


Author: relaxmike
Date: Mar 28, 2008 00:53

Looking again at "iso_varying_string", I saw that that there was
a "split" command, see here :

http://www.fortran.com/iso_varying_string.f95

elemental subroutine split_VS (string, word, set, separator, back)

type(varying_string), intent(inout) :: string
type(varying_string), intent(out) :: word
type(varying_string), intent(in) :: set
type(varying_string), intent(out), optional :: separator
logical, intent(in), optional :: back

But it did not see a "string is" command in iso_varying_string.

The flibs project :
http://flibs.sourceforge.net/
contains
"tokenize : Module to split a string into pieces according to certain
fairly basic, but common criteria."

Best regards,

Michaël
no comments
1 2