Test of copy tools with dynafed
Version of tools needed to have it working
- on the source site (at least if DPM)
- latest certificates for the sites involved (under /etc/grid-security/certificates)
- nss 3.28 or later
- davix and davix-libs 0.6.6 or later
- latest openssl
- /var/www/proxycache owned by the DPM user, usually dpmmgr:dpmmgr
- NSSFlags and DiskFlags in the apache config need to be set to: Write RemoteCopy
- on the destination site (dynafed)
- latest certificates for the sites involved (under /etc/grid-security/certificates)
- nss 3.28 or later
- davix and davix-libs 0.6.6 or later
- latest openssl
- on the client machine
- davix(-libs) at least in version 0.6.6
- Make sure the correct version is used if CVMFS software is in the PATH/LD_LIBRARY_PATH!
- create the entries needed to make the certificates available for authentication of sites
- sudo cp /etc/grid-security/certificates/* /etc/pki/ca-trust/source/anchors/
- sudo update-ca-trust extract
source: dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
example command: gfal-copy davs+3rd://dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11
ls commands are only for the source
*---------------> old tests below
from dCache (charon01)
source: charon01.westgrid.ca:2880/pnfs/westgrid.uvic.ca/data/atlas/atlasdatadisk/rucio/mc12_8TeV/8d/c0/EVNT.01212395._000004.pool.root.1
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
example command: gfal-copy davs+3rd://charon01.westgrid.ca:2880/pnfs/westgrid.uvic.ca/data/atlas/atlasdatadisk/rucio/mc12_8TeV/8d/c0/EVNT.01212395._000004.pool.root.1 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11
ls commands are only for the source
NOTE: The output of the
ls commands is different for davix-ls and gfal-ls. While the size and name are fine, all other attributes can differ, especially also the reported time stamp.
tool |
davs |
https |
davs+3rd |
https+3rd |
davix-cp |
error 1 |
error 2 |
error 3 |
error 3 |
davix-ls |
ok |
ok |
error 3 |
error 3 |
gfal-copy |
ok |
ok |
error 2 |
error2 |
gfal-ls |
ok |
ok |
ok |
ok |
error number |
error message |
1 |
Error: copy HTTP 400 : Server Error |
2 |
Error: copy Could not get the delegation id: Could not get proxy request: SOAP 1.1 fault: SOAP-ENV:Client [no subcode] Detail: connect failed in tcp_connect() |
3 |
Error: copy (Neon): Could not read status line: connection was closed by server |
from DPM (srm.glite.ecdf.ed.ac.uk)
source: srm.glite.ecdf.ed.ac.uk/dpm/ecdf.ed.ac.uk/home/atlas/atlasdatadisk/rucio/data17_comm/0f/ba/data17_comm.00324195.physics_Main.merge.AOD.f822_m1799._lb0421._0001.1
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
example command: gfal-copy davs+3rd://srm.glite.ecdf.ed.ac.uk/dpm/ecdf.ed.ac.uk/home/atlas/atlasdatadisk/rucio/data17_comm/0f/ba/data17_comm.00324195.physics_Main.merge.AOD.f822_m1799._lb0421._0001.1 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11
ls commands are only for the source
tool |
davs |
https |
davs+3rd |
https+3rd |
davix-cp |
error 1 |
error 1 |
error 1 |
error 1 |
davix-get |
ok |
ok |
ok |
ok |
davix-ls |
ok |
ok |
ok |
ok |
gfal-copy |
ok |
ok |
error 1 |
error 1 |
gfal-ls |
ok |
ok |
ok |
ok |
NOTE: There is no entry in http or dynafed logs about tried access for the failed connections.
from local file
source: /tmp/testfile
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
davix-cp can't copy local files
davs+3rd and
https+3rd was used to just see how the tool itself handles that.
test of get commands from charon01
error number |
error message |
1 |
Failure (Neon): Could not read status line: connection was closed by server after 3 attempts |
2 |
Error: (Neon): Could not read status line: connection was closed by server |
test of get commands from dpmhead-trunk
test of put commands to dynafed02
--
mebert - 2017-07-11
Comments