DynaFedCopyTest < HEPrc

Tags: view all tags
---+ Test of copy tools with dynafed

---++ Version of tools needed to have it working
   * on the source site (at least if DPM)
      * latest certificates for the sites involved (under /etc/grid-security/certificates)
      * nss 3.28 or later
      * davix and davix-libs 0.6.6 or later
      * latest openssl
      * /var/www/proxycache owned by the DPM user, usually dpmmgr:dpmmgr 
      * NSSFlags and DiskFlags in the apache config need to be set to: Write RemoteCopy
   * on the destination site (dynafed)
      * latest certificates for the sites involved (under /etc/grid-security/certificates)
      * nss 3.28 or later
      * davix and davix-libs 0.6.6 or later
      * latest openssl
   * on the client machine
      * davix(-libs) at least in version 0.6.6
         * Make sure the correct version is used if CVMFS software is in the PATH/LD_LIBRARY_PATH!
      * create the entries needed to make the certificates available for authentication of sites
         * sudo cp /etc/grid-security/certificates/* /etc/pki/ca-trust/source/anchors/                                                                                         
         * sudo update-ca-trust extract                                                                                                                                        



---++ from DPM (dpmhead-trunk@cern)
source:    dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00<br>
destination:   dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX <br>
example command:   gfal-copy davs+3rd://dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11   <br>
*ls* commands are only for the source   <br>


| *tool* | *davs* | *https* | *davs+3rd* | *https+3rd*|
|davix-cp | ok  | ok | ok | ok | 
|davix-ls | ok | ok | ok | ok |
|gfal-copy | ok | ok | ok | ok |
|gfal-ls | ok | ok | ok | ok |




*---------------> old tests below

---++ from <nop>dCache (charon01)
source:    charon01.westgrid.ca:2880/pnfs/westgrid.uvic.ca/data/atlas/atlasdatadisk/rucio/mc12_8TeV/8d/c0/EVNT.01212395._000004.pool.root.1<br>
destination:   dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX <br>
example command:   gfal-copy davs+3rd://charon01.westgrid.ca:2880/pnfs/westgrid.uvic.ca/data/atlas/atlasdatadisk/rucio/mc12_8TeV/8d/c0/EVNT.01212395._000004.pool.root.1 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11   <br>
*ls* commands are only for the source   <br>
*NOTE*: The output of the _ls_ commands is different for davix-ls and gfal-ls. While the size and name are fine, all other attributes can differ, especially also the reported time stamp.<br>

| *tool* | *davs* | *https* | *davs+3rd* | *https+3rd*|
|davix-cp | error 1 | error 2| error 3 | error 3 | 
|davix-ls | ok | ok | error 3 | error 3 |
|gfal-copy | ok | ok | error 2 | error2 |
|gfal-ls | ok | ok| ok | ok |


|*error number*| *error message*|
| 1 | Error: copy HTTP 400 : Server Error|
| 2 | Error: copy Could not get the delegation id: Could not get proxy request: SOAP 1.1 fault: SOAP-ENV:Client [no subcode]     Detail: connect failed in tcp_connect()  |
| 3 | Error: copy (Neon): Could not read status line: connection was closed by server|


---++ from DPM (srm.glite.ecdf.ed.ac.uk)
source:    srm.glite.ecdf.ed.ac.uk/dpm/ecdf.ed.ac.uk/home/atlas/atlasdatadisk/rucio/data17_comm/0f/ba/data17_comm.00324195.physics_Main.merge.AOD.f822_m1799._lb0421._0001.1<br>
destination:   dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX <br>
example command:   gfal-copy davs+3rd://srm.glite.ecdf.ed.ac.uk/dpm/ecdf.ed.ac.uk/home/atlas/atlasdatadisk/rucio/data17_comm/0f/ba/data17_comm.00324195.physics_Main.merge.AOD.f822_m1799._lb0421._0001.1 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11   <br>
*ls* commands are only for the source   <br>


| *tool* | *davs* | *https* | *davs+3rd* | *https+3rd*|
|davix-cp | error 1  | error 1 | error 1 | error 1 | 
|davix-get| ok | ok | ok | ok |
|davix-ls | ok | ok  | ok | ok |
|gfal-copy | ok | ok | error 1 | error 1  |
|gfal-ls | ok | ok | ok | ok |


|*error number*| *error message*| *log message from dpm server*|
| 1 | copy HTTP 405 : Method Not Allowed, Permission refused| (115)Operation now in progress: This server does not allow remote copies ()  [405, #0]|

*NOTE*: There is no entry in http or dynafed logs about tried access for the failed connections.<br>







---++ from local file
source:    /tmp/testfile<br>
destination:   dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX <br>

| *tool* | *davs* | *https* | *davs+3rd* | *https+3rd*|
|davix-cp | -  | - | - | - | 
|gfal-copy | ok | ok | ok | ok |

davix-cp can't copy local files<br>
*davs+3rd* and *https+3rd* was used to just see how the tool itself handles that.

---++ test of get commands from charon01
| *tool* | *davs* | *https* | *davs+3rd* | *https+3rd*|
| davix-get|  ok | ok | error 1| error 1|
| davix-http| ok| ok| error 2 | error 2|
| gfal-cat | ok| ok| ok | ok |

|*error number*| *error message*|
|1| Failure (Neon): Could not read status line: connection was closed by server after 3 attempts|
|2| Error: (Neon): Could not read status line: connection was closed by server|

---++ test of get commands from dpmhead-trunk
| *tool* | *davs* | *https* |*davs+3rd* | *https+3rd*|
| davix-get|  ok | ok | ok| ok|
| davix-http| ok| ok| ok | ok|
| gfal-cat | ok| ok| ok| ok|


---++ test of put commands to dynafed02
| *tool* | *davs* | *https* | *davs+3rd* | *https+3rd*|
|davix-put| ok | ok | error 1 | error 1 |

|*error number*| *error message*|
|1| Error: HTTP 400 : Server Error|



-- %USERSIG{mebert - 2017-07-11}%

---++ Comments

%COMMENT%
Topic revision: r9 - 2017-09-11 - mebert