> Gabor wrote:

>> Can it be some subtle gfortran bug?

>
> Unlikely. Having multiple programs write to the same file without
> locking is a bug (in your application). It is a bug that can produce
> exactly the symptoms you describe. Since you have a known bug in your
> application that matches the symptoms, but no information pointing to
> anything about the compiler, I see no basis for such a suspicion. It
> seems rather "random". While compilers do have bugs, it sometimes seems
> like those who are most prone to suspect the compiler are those least
> likely to have actually found one of the compiler bugs. As a general
> rule, one should suspect one's own code first. That's not always the
> source fo the problem, but it is still a good general rule to start
> there.
>
> In your case, you have indeed found a problem there, which makes it even
> less fruitful to go on to suspect the compiler anyway.
>

>>I am somewhat disappointed because these "long" calculations
>> often take only tens of minutes, and the bookkeeping
>>can take away most of the gain of quick answers.

>
> I'm a little surpised that the work to combine the files would take a
> significant fraction of the tens of minutes that you describe unless
> there is an awfully lot of output. If there is really that much output,
> your application might be inherently output limitted. That isn't
> horribly unusual. You say that the app is "embassassingly parallel." In
> that case may well be that serialization of the output is the limitting
> factor. The ration of I/O to computation is a fundamental factor in
> extreme parallelization.
>
> Do note that if I/O performance is being a hindrance, you might want to
> consider using unformatted I/O instead of formatted for the intermediate
> files. Formatted I/O is far slower than unformatted, often by a factor
> of 10 or so. Even if the final file needs to be formatted for human
> consumption or other purposes, the intermediate files could be
> unformatted.
>
> I suppose that another possible inefficiency could be in the OPENs.
> Opening a file typically is relatively slow, compared to such things as
> writing a single record to it. If, perhaps, each of your files is very
> small (for example, if they are only one line each), it might be that
> you are spending a lot more time on the open/close than on
> reading/writing the data. If that's the case, possibly you could benefit
> from more complicated schemes of interprocess communication... but they
> would be quite a bit more complicated (and I couldn't tell you off the
> top of my head how to do them; it would take some research).
>
> --
> Richard Maine                    | Good judgement comes from experience;
> email: last name at domain . net | experience comes from bad judgement.
> domain: summertriangle           |  -- Mark Twain