Is file_get_contents & file_put_contents reliable or can lead to loss of data? Benchmark results
I was wondering what happens if multiple scripts are sharing same
file. I uploaded the test on remote server, where they use HDD to store
data. There were 7 tests total, but the family of 6 are compatible.
I have 7 files of different size which I uploaded to server and the test. It is loop which reads and writes data from the files.
There is 50 microseconds delay in the loop. The loop repeats 50x.
I measure the time needed to perform every circle.
The differences in the tests (T):
Using file_get_contents/file_put_contents
T2 - SOURCE <> TARGET - reads data from original file, writes data do different (new) file
T3 - SOURCE = TARGET - 1. copies data from original file to target; 2. reads source data -> writes data; 3. the point 3 is repeated: i.e I read the data which I have written. This test uses same file to write data.
T4 - SOURCE = TARGET - I repeated the same test as in T3 getting shorted times.
Using fopen, flock, fread, flock, fclose, fopen, flock, fopen, fwrite, fflush, fclock, fclose ... This is complicated code, but here I have tested the fflush. I also use clearstatcache, stat and touch and clearstatcache, filesize. To check validity. The tests T5 - T7 were less reliable than T2-T4 because sometimes the write operation failed. I tested the file size and when it was not correct, I copied (restored) the file back from original file.
T5: (fflush) SOURCE = TARGET
T6: (fflush) SOURCE <> TARGET
T7: (fflush) SOURCE <> TARGET + I have removed the 50 microseconds delay from the loop (It seems like the validity/reliability is worse when there is a delay).
I made 4 requests from 4 different browsers - so every test have 4 sets of data (7*50*4 values total).
Now I have collected all data, created tables and diagrams. This is one diagram of many, showing minimal and maximal values of avarage value.
T4 yellow color and T3 green provides very small times so they are suspicious. For example T4 avarage times are these: 0,001
T5-T7 just show normal times as expected - the bigger the file the bigger the time needed to proccess. Fairly slow as expected from HDD and 4 scripts running at the same time.
So my question here is:
Does the results of T3-T4 mean, that the file_read_contents and file_put_contents are not reliable for this type of job? To me it looks like they simply do not read the data from file but they are copied from buffer, which means, that old data are saved, not the current data been changed by concurent script. I would welcome more information. I spent a lot of time searching for answers but did not found clear answer. I did this tests because I need proofs. You man want to use my scripts but I am not sure if can I paste here the 6 scripts?
Now I will add just the fflush test number 7 which is most useful.
"; usleep(5000); } endfor; $s = fread($fp, $fsize ); $success = flock($fp, LOCK_UN); if ( $success === false ) die("; r flock release failed; "); $success = fclose($fp); if ( $success === false ) die("; fclose failed; "); // 10 - data loaded , $p - browser if ( $success ) { $result = touch("$sname",strlen($s),$p); if ( $_DEBUG_ ) echo "; TOUCH: $result;"; } else die("fclose FAIL."); if ( strlen($s)<60 ) echo "*$s LENGTH:".strlen($s)."
"; } clearstatcache(); $st = stat("$tname"); if ( $_DEBUG_ ) echo "; 2) prevAccess by ".$st['mtime']." fsize is ".$fsize."; "; // WRITE OPERATION WITH LOC_EX $fp = fopen($tname, "w"); $locked = false; $locked = flock($fp, LOCK_EX); if ( $locked ) { // acquire an exclusive lock $success = fwrite($fp, $s); if ( $success === false) echo "; w FAILED;"; else if ( $_DEBUG_ ) echo " $success B written; "; $success = fflush($fp);// flush output before releasing the lock if ( $success === false ) echo "; flush FAILED; "; $success = flock($fp, LOCK_UN); // release the lock if ( $success === false ) echo "; release FAILED; "; $success = fclose($fp); if ( $success === false ) echo "; fclose FAILED; "; clearstatcache(); // needed for filesize and touch $fsize = filesize($tname); if ($original_size>$fsize) { echo "; WRITE FAILED, restoring;"; $original_fname = "$n"; $result = copy($original_fname, $tname); if ($result == false ) die(" TOTAL FAILTURE: copy failed."); else echo " RESTORED;"; } else { if ($fsize === 0) echo "! THE FILE WAS NOT WRITTEN: data length: ".strlen($s)." fsize: $fsize RESOURCE: $fp
"; if ( $success ) touch("$tname",$fsize,$p); } } else { echo "Couldn't get the lock!"; } $time_elapsed_secs = microtime(true) - $start; if ( $time_elapsed_secs === 0 ) echo " FAILED "; echo "time: $time_elapsed_secs s
"; } } switch ( $_SERVER['HTTP_USER_AGENT'] ): // FF 1: case "Mozilla/5.0 (Windows NT 5.1; rv:49.0) Gecko/20100101 Firefox/49.0": $p = 1; break; // Chrome: case "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36": $p = 2; break; // OPERA: case "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36 OPR/36.0.2130.80": $p = 3; break; endswitch; copy("523","523.txt"); copy("948","948.txt"); copy("1371","1371.txt"); copy("1913","1913.txt"); copy("2701","2701.txt"); copy("4495","4495.txt"); copy("6758","6758.txt"); test("523",$p,$_DEBUG_); test("948",$p,$_DEBUG_); test("1371",$p,$_DEBUG_); test("1913",$p,$_DEBUG_); test("2701",$p,$_DEBUG_); test("4495",$p,$_DEBUG_); test("6758",$p,$_DEBUG_); die; echo "php: " . phpversion(); ?> PHP echo "php: " . phpinfo(); ?>
Note: This is not a request for test, this is just request for review.
Also: Please do not be confused by the yellow color curve. There are two yellow colors. The T4 yellow is almost no visible on the diagram because it has very low values.
If you want to solve this question, follow the topic here.
<50 1="" and="" by="" clearstatcache="" echo="" filesize="" for="" fsize="filesize($sname);" i="" if="" mtime="" needed="" original_size="" prevaccess="" size="" sname="" st="" start="microtime(true);" touch=""><60 br="" echo="" length:="" s="" strlen="">
I have 7 files of different size which I uploaded to server and the test. It is loop which reads and writes data from the files.
There is 50 microseconds delay in the loop. The loop repeats 50x.
I measure the time needed to perform every circle.
The differences in the tests (T):
Using file_get_contents/file_put_contents
T2 - SOURCE <> TARGET - reads data from original file, writes data do different (new) file
T3 - SOURCE = TARGET - 1. copies data from original file to target; 2. reads source data -> writes data; 3. the point 3 is repeated: i.e I read the data which I have written. This test uses same file to write data.
T4 - SOURCE = TARGET - I repeated the same test as in T3 getting shorted times.
Using fopen, flock, fread, flock, fclose, fopen, flock, fopen, fwrite, fflush, fclock, fclose ... This is complicated code, but here I have tested the fflush. I also use clearstatcache, stat and touch and clearstatcache, filesize. To check validity. The tests T5 - T7 were less reliable than T2-T4 because sometimes the write operation failed. I tested the file size and when it was not correct, I copied (restored) the file back from original file.
T5: (fflush) SOURCE = TARGET
T6: (fflush) SOURCE <> TARGET
T7: (fflush) SOURCE <> TARGET + I have removed the 50 microseconds delay from the loop (It seems like the validity/reliability is worse when there is a delay).
I made 4 requests from 4 different browsers - so every test have 4 sets of data (7*50*4 values total).
Now I have collected all data, created tables and diagrams. This is one diagram of many, showing minimal and maximal values of avarage value.
T4 yellow color and T3 green provides very small times so they are suspicious. For example T4 avarage times are these: 0,001
0.001 0.002 0.003 0.002 0.004 0.003 0.004 0.001 0.004 0.001 0.004 0.001 0.004 0.002 0.003 0.001 0.001 0.003 0.003 0.006 0.007 0.002 0.003 0.004 0.004 0.019 0.019 T5-T7 just show normal times as expected - the bigger the file the bigger the time needed to proccess. Fairly slow as expected from HDD and 4 scripts running at the same time.
So my question here is:
Does the results of T3-T4 mean, that the file_read_contents and file_put_contents are not reliable for this type of job? To me it looks like they simply do not read the data from file but they are copied from buffer, which means, that old data are saved, not the current data been changed by concurent script. I would welcome more information. I spent a lot of time searching for answers but did not found clear answer. I did this tests because I need proofs. You man want to use my scripts but I am not sure if can I paste here the 6 scripts?
Now I will add just the fflush test number 7 which is most useful.
PHP
clearstatcache();
$_DEBUG_ = false;
echo "Lock and flush tester.".time()."
";
die;
while ( time()<1570787996 )
{
usleep(500);
}
function test($n, $p, $_DEBUG_){
$sname = "$n"; // source
$tname = "$n.txt";// target
echo "
$n at "
.time().""; usleep(5000); } endfor; $s = fread($fp, $fsize ); $success = flock($fp, LOCK_UN); if ( $success === false ) die("; r flock release failed; "); $success = fclose($fp); if ( $success === false ) die("; fclose failed; "); // 10 - data loaded , $p - browser if ( $success ) { $result = touch("$sname",strlen($s),$p); if ( $_DEBUG_ ) echo "; TOUCH: $result;"; } else die("fclose FAIL."); if ( strlen($s)<60 ) echo "*$s LENGTH:".strlen($s)."
"; } clearstatcache(); $st = stat("$tname"); if ( $_DEBUG_ ) echo "; 2) prevAccess by ".$st['mtime']." fsize is ".$fsize."; "; // WRITE OPERATION WITH LOC_EX $fp = fopen($tname, "w"); $locked = false; $locked = flock($fp, LOCK_EX); if ( $locked ) { // acquire an exclusive lock $success = fwrite($fp, $s); if ( $success === false) echo "; w FAILED;"; else if ( $_DEBUG_ ) echo " $success B written; "; $success = fflush($fp);// flush output before releasing the lock if ( $success === false ) echo "; flush FAILED; "; $success = flock($fp, LOCK_UN); // release the lock if ( $success === false ) echo "; release FAILED; "; $success = fclose($fp); if ( $success === false ) echo "; fclose FAILED; "; clearstatcache(); // needed for filesize and touch $fsize = filesize($tname); if ($original_size>$fsize) { echo "; WRITE FAILED, restoring;"; $original_fname = "$n"; $result = copy($original_fname, $tname); if ($result == false ) die(" TOTAL FAILTURE: copy failed."); else echo " RESTORED;"; } else { if ($fsize === 0) echo "! THE FILE WAS NOT WRITTEN: data length: ".strlen($s)." fsize: $fsize RESOURCE: $fp
"; if ( $success ) touch("$tname",$fsize,$p); } } else { echo "Couldn't get the lock!"; } $time_elapsed_secs = microtime(true) - $start; if ( $time_elapsed_secs === 0 ) echo " FAILED "; echo "time: $time_elapsed_secs s
"; } } switch ( $_SERVER['HTTP_USER_AGENT'] ): // FF 1: case "Mozilla/5.0 (Windows NT 5.1; rv:49.0) Gecko/20100101 Firefox/49.0": $p = 1; break; // Chrome: case "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36": $p = 2; break; // OPERA: case "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36 OPR/36.0.2130.80": $p = 3; break; endswitch; copy("523","523.txt"); copy("948","948.txt"); copy("1371","1371.txt"); copy("1913","1913.txt"); copy("2701","2701.txt"); copy("4495","4495.txt"); copy("6758","6758.txt"); test("523",$p,$_DEBUG_); test("948",$p,$_DEBUG_); test("1371",$p,$_DEBUG_); test("1913",$p,$_DEBUG_); test("2701",$p,$_DEBUG_); test("4495",$p,$_DEBUG_); test("6758",$p,$_DEBUG_); die; echo "php: " . phpversion(); ?> PHP echo "php: " . phpinfo(); ?>
Notice: 523 means the filesize is 523 kB.
Note: This is not a request for test, this is just request for review.
Also: Please do not be confused by the yellow color curve. There are two yellow colors. The T4 yellow is almost no visible on the diagram because it has very low values.
If you want to solve this question, follow the topic here.
<50 1="" and="" by="" clearstatcache="" echo="" filesize="" for="" fsize="filesize($sname);" i="" if="" mtime="" needed="" original_size="" prevaccess="" size="" sname="" st="" start="microtime(true);" touch=""><60 br="" echo="" length:="" s="" strlen="">
Komentáře
Okomentovat