Test of copy tools with dynafed

Version of tools needed to have it working

• on the source site (at least if DPM)
  • latest certificates for the sites involved (under /etc/grid-security/certificates)
  • nss 3.28 or later
  • davix and davix-libs 0.6.6 or later
  • latest openssl
  • /var/www/proxycache owned by the DPM user, usually dpmmgr:dpmmgr
  • NSSFlags and DiskFlags in the apache config need to be set to: Write RemoteCopy
• on the destination site (dynafed)
  • latest certificates for the sites involved (under /etc/grid-security/certificates)
  • nss 3.28 or later
  • davix and davix-libs 0.6.6 or later
  • latest openssl
• on the client machine
  • davix(-libs) at least in version 0.6.6
    • Make sure the correct version is used if CVMFS software is in the PATH/LD_LIBRARY_PATH!
  • create the entries needed to make the certificates available for authentication of sites
    • sudo cp /etc/grid-security/certificates/* /etc/pki/ca-trust/source/anchors/
    • sudo update-ca-trust extract

from DPM (dpmhead-trunk@cern)

source: dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
example command: gfal-copy davs+3rd://dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11
ls commands are only for the source

tool	davs	https	davs+3rd	https+3rd
davix-cp	ok	ok	ok	ok
davix-ls	ok	ok	ok	ok
gfal-copy	ok	ok	ok	ok
gfal-ls	ok	ok	ok	ok

*---------------> old tests below

from dCache (charon01)

source: charon01.westgrid.ca:2880/pnfs/westgrid.uvic.ca/data/atlas/atlasdatadisk/rucio/mc12_8TeV/8d/c0/EVNT.01212395._000004.pool.root.1
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
example command: gfal-copy davs+3rd://charon01.westgrid.ca:2880/pnfs/westgrid.uvic.ca/data/atlas/atlasdatadisk/rucio/mc12_8TeV/8d/c0/EVNT.01212395._000004.pool.root.1 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11
ls commands are only for the source
NOTE: The output of the ls commands is different for davix-ls and gfal-ls. While the size and name are fine, all other attributes can differ, especially also the reported time stamp.

tool	davs	https	davs+3rd	https+3rd
davix-cp	error 1	error 2	error 3	error 3
davix-ls	ok	ok	error 3	error 3
gfal-copy	ok	ok	error 2	error2
gfal-ls	ok	ok	ok	ok

error number	error message
1	Error: copy HTTP 400 : Server Error
2	Error: copy Could not get the delegation id: Could not get proxy request: SOAP 1.1 fault: SOAP-ENV:Client [no subcode] Detail: connect failed in tcp_connect()
3	Error: copy (Neon): Could not read status line: connection was closed by server

from DPM (srm.glite.ecdf.ed.ac.uk)

source: srm.glite.ecdf.ed.ac.uk/dpm/ecdf.ed.ac.uk/home/atlas/atlasdatadisk/rucio/data17_comm/0f/ba/data17_comm.00324195.physics_Main.merge.AOD.f822_m1799._lb0421._0001.1
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
example command: gfal-copy davs+3rd://srm.glite.ecdf.ed.ac.uk/dpm/ecdf.ed.ac.uk/home/atlas/atlasdatadisk/rucio/data17_comm/0f/ba/data17_comm.00324195.physics_Main.merge.AOD.f822_m1799._lb0421._0001.1 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11
ls commands are only for the source

tool	davs	https	davs+3rd	https+3rd
davix-cp	error 1	error 1	error 1	error 1
davix-get	ok	ok	ok	ok
davix-ls	ok	ok	ok	ok
gfal-copy	ok	ok	error 1	error 1
gfal-ls	ok	ok	ok	ok

error number	error message	log message from dpm server
1	copy HTTP 405 : Method Not Allowed, Permission refused	(115)Operation now in progress: This server does not allow remote copies () [405, #0]

NOTE: There is no entry in http or dynafed logs about tried access for the failed connections.

from local file

source: /tmp/testfile
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX

tool	davs	https	davs+3rd	https+3rd
davix-cp	-	-	-	-
gfal-copy	ok	ok	ok	ok

davix-cp can't copy local files
davs+3rd and https+3rd was used to just see how the tool itself handles that.

test of get commands from charon01

tool	davs	https	davs+3rd	https+3rd
davix-get	ok	ok	error 1	error 1
davix-http	ok	ok	error 2	error 2
gfal-cat	ok	ok	ok	ok

error number	error message
1	Failure (Neon): Could not read status line: connection was closed by server after 3 attempts
2	Error: (Neon): Could not read status line: connection was closed by server

test of get commands from dpmhead-trunk

tool	davs	https	davs+3rd	https+3rd
davix-get	ok	ok	ok	ok
davix-http	ok	ok	ok	ok
gfal-cat	ok	ok	ok	ok

test of put commands to dynafed02

tool	davs	https	davs+3rd	https+3rd
davix-put	ok	ok	error 1	error 1

error number	error message
1	Error: HTTP 400 : Server Error

-- mebert - 2017-07-11

Comments