I was wondering what happens if multiple scripts are sharing same
file. I uploaded the test on remote server, where they use HDD to store
data. There were 7 tests total, but the family of 6 are compatible.
I have 7 files of different size which I uploaded to server and the
test. It is loop which reads and writes data from the files.
There is 50 microseconds delay in the loop. The loop repeats 50x.
I measure the time needed to perform every circle.
The differences in the tests (T):
Using file_get_contents/file_put_contents
T2 - SOURCE <> TARGET - reads data from original file, writes data do different (new) file
T3 - SOURCE = TARGET - 1. copies data from original
file to target; 2. reads source data -> writes data; 3. the point 3
is repeated: i.e I read the data which I have written. This test uses
same file to write data.
T4 - SOURCE = TARGET - I repeated the same test as in T3 getting shorted times.
Using fopen, flock, fread, flock, fclose, fopen, flock, fopen, fwrite,
fflush,
fclock, fclose ... This is complicated code, but here I have tested the
fflush. I also use clearstatcache, stat and touch and clearstatcache,
filesize. To check validity. The tests T5 - T7 were less reliable than
T2-T4 because sometimes the write operation failed. I tested the file
size and when it was not correct, I copied (restored) the file back from
original file.
T5: (fflush) SOURCE = TARGET
T6: (fflush) SOURCE <> TARGET
T7: (fflush) SOURCE <> TARGET + I have removed
the 50 microseconds delay from the loop (It seems like the
validity/reliability is worse when there is a delay).
I made 4 requests from 4 different browsers - so every test have 4 sets of data (7*50*4 values total).
Now I have collected all data, created tables and diagrams. This is
one diagram of many, showing minimal and maximal values of avarage
value.
T4 yellow color and
T3 green provides very small times so they are suspicious. For example T4 avarage times are these: 0,001
0.001 0.002 0.003 0.002 0.004 0.003 0.004 0.001 0.004 0.001 0.004 0.001 0.004
And T3 times:
0.002 0.003 0.001 0.001 0.003 0.003 0.006 0.007 0.002 0.003 0.004 0.004 0.019 0.019
The values of T2 seems normal, but this can be explained by the fact,
that that was read from different file than was written to.
T5-T7 just show normal times as expected - the bigger the file the
bigger the time needed to proccess. Fairly slow as expected from HDD and
4 scripts running at the same time.
So my question here is:
Does the results of T3-T4 mean, that the file_read_contents and file_put_contents are not reliable for this type of job?
To me it looks like they simply do not read the data from file but they
are copied from buffer, which means, that old data are saved, not the
current data been changed by concurent script. I would welcome more
information. I spent a lot of time searching for answers but did not
found clear answer. I did this tests because I need proofs. You man want
to use my scripts but I am not sure if can I paste here the 6 scripts?
Now I will add just the fflush test number 7 which is most useful.
PHP
clearstatcache();
$_DEBUG_ = false;
echo "Lock and flush tester.".time()."
";
die;
while ( time()<1570787996 )
{
usleep(500);
}
function test($n, $p, $_DEBUG_){
$sname = "$n"; // source
$tname = "$n.txt";// target
echo "
$n at "
.time()."
"
;
for ($i = 0; $i<50; $i++ ){
$start = microtime(true);
clearstatcache(); // needed for filesize and touch
$st = stat("$sname");
$original_size = $st['size'];
if ( $_DEBUG_ )
echo "; 1) prevAccess by ".$st['mtime']." fsize ".$st['size']."; ";
$fsize = filesize($sname);
if ( $original_size <> $fsize )
die("; fsize total FAILTURE; ");
if ($fsize === 0)
echo "! The fsize is 0: stat(): ".$st['size']." ;";
else
{
// READ OPERATION AND LOCK FOR SHARE
$locked = false;
for ($c = 0; !$locked; $c++):
if ( $c > 400)
break;
$fp = fopen($sname, "r");
$locked = flock($fp, LOCK_SH);
if ($locked)
break;
else
{
echo "failed to get LOCK_SH;
";
usleep(5000);
}
endfor;
$s = fread($fp, $fsize );
$success = flock($fp, LOCK_UN);
if ( $success === false )
die("; r flock release failed; ");
$success = fclose($fp);
if ( $success === false )
die("; fclose failed; ");
// 10 - data loaded , $p - browser
if ( $success )
{
$result = touch("$sname",strlen($s),$p);
if ( $_DEBUG_ )
echo "; TOUCH: $result;";
}
else
die("fclose FAIL.");
if ( strlen($s)<60 )
echo "*$s LENGTH:".strlen($s)."
";
}
clearstatcache();
$st = stat("$tname");
if ( $_DEBUG_ )
echo "; 2) prevAccess by ".$st['mtime']." fsize is ".$fsize."; ";
// WRITE OPERATION WITH LOC_EX
$fp = fopen($tname, "w");
$locked = false;
$locked = flock($fp, LOCK_EX);
if ( $locked ) { // acquire an exclusive lock
$success = fwrite($fp, $s);
if ( $success === false)
echo "; w FAILED;";
else
if ( $_DEBUG_ )
echo " $success B written; ";
$success = fflush($fp);// flush output before releasing the lock
if ( $success === false )
echo "; flush FAILED; ";
$success = flock($fp, LOCK_UN); // release the lock
if ( $success === false )
echo "; release FAILED; ";
$success = fclose($fp);
if ( $success === false )
echo "; fclose FAILED; ";
clearstatcache(); // needed for filesize and touch
$fsize = filesize($tname);
if ($original_size>$fsize)
{
echo "; WRITE FAILED, restoring;";
$original_fname = "$n";
$result = copy($original_fname, $tname);
if ($result == false )
die(" TOTAL FAILTURE: copy failed.");
else
echo " RESTORED;";
}
else
{
if ($fsize === 0)
echo "! THE FILE WAS NOT WRITTEN: data length: ".strlen($s)." fsize: $fsize RESOURCE: $fp
";
if ( $success )
touch("$tname",$fsize,$p);
}
} else {
echo "Couldn't get the lock!";
}
$time_elapsed_secs = microtime(true) - $start;
if ( $time_elapsed_secs === 0 )
echo " FAILED ";
echo "time: $time_elapsed_secs s
";
}
}
switch ( $_SERVER['HTTP_USER_AGENT'] ):
// FF 1:
case "Mozilla/5.0 (Windows NT 5.1; rv:49.0) Gecko/20100101 Firefox/49.0":
$p = 1; break;
// Chrome:
case "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36":
$p = 2; break;
// OPERA:
case "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36 OPR/36.0.2130.80":
$p = 3; break;
endswitch;
copy("523","523.txt");
copy("948","948.txt");
copy("1371","1371.txt");
copy("1913","1913.txt");
copy("2701","2701.txt");
copy("4495","4495.txt");
copy("6758","6758.txt");
test("523",$p,$_DEBUG_);
test("948",$p,$_DEBUG_);
test("1371",$p,$_DEBUG_);
test("1913",$p,$_DEBUG_);
test("2701",$p,$_DEBUG_);
test("4495",$p,$_DEBUG_);
test("6758",$p,$_DEBUG_);
die;
echo "php: " . phpversion();
?>
PHP echo "php: " . phpinfo();
?>
Notice: 523 means the filesize is 523 kB.
You may want to enable $
DEBUG option to monitor each proccess. Note: The touch maybe do not work correctly always.
Note: This is not a request for test, this is just request for review.
Also: Please do not be confused by the yellow color curve. There are
two yellow colors. The T4 yellow is almost no visible on the diagram
because it has very low values.
If you want to solve this question, follow the topic
here.
<50 1="" and="" by="" clearstatcache="" echo="" filesize="" for="" fsize="filesize($sname);" i="" if="" mtime="" needed="" original_size="" prevaccess="" size="" sname="" st="" start="microtime(true);" touch=""><60 br="" echo="" length:="" s="" strlen="">
