Test of copy tools with dynafed

Version of tools needed to have it working

  • on the source site (at least if DPM)
    • latest certificates for the sites involved (under /etc/grid-security/certificates)
    • nss 3.28 or later
    • davix and davix-libs 0.6.6 or later
    • latest openssl
    • /var/www/proxycache owned by the DPM user, usually dpmmgr:dpmmgr
    • NSSFlags and DiskFlags in the apache config need to be set to: Write RemoteCopy
  • on the destination site (dynafed)
    • latest certificates for the sites involved (under /etc/grid-security/certificates)
    • nss 3.28 or later
    • davix and davix-libs 0.6.6 or later
    • latest openssl
  • on the client machine
    • davix(-libs) at least in version 0.6.6
      • Make sure the correct version is used if CVMFS software is in the PATH/LD_LIBRARY_PATH!
    • create the entries needed to make the certificates available for authentication of sites
      • sudo cp /etc/grid-security/certificates/* /etc/pki/ca-trust/source/anchors/
      • sudo update-ca-trust extract

from DPM (dpmhead-trunk@cern)

source: dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
example command: gfal-copy davs+3rd://dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11
ls commands are only for the source

tool davs https davs+3rd https+3rd
davix-cp ok ok ok ok
davix-ls ok ok ok ok
gfal-copy ok ok ok ok
gfal-ls ok ok ok ok

*---------------> old tests below

from dCache (charon01)

source: charon01.westgrid.ca:2880/pnfs/westgrid.uvic.ca/data/atlas/atlasdatadisk/rucio/mc12_8TeV/8d/c0/EVNT.01212395._000004.pool.root.1
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
example command: gfal-copy davs+3rd://charon01.westgrid.ca:2880/pnfs/westgrid.uvic.ca/data/atlas/atlasdatadisk/rucio/mc12_8TeV/8d/c0/EVNT.01212395._000004.pool.root.1 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11
ls commands are only for the source
NOTE: The output of the ls commands is different for davix-ls and gfal-ls. While the size and name are fine, all other attributes can differ, especially also the reported time stamp.

tool davs https davs+3rd https+3rd
davix-cp error 1 error 2 error 3 error 3
davix-ls ok ok error 3 error 3
gfal-copy ok ok error 2 error2
gfal-ls ok ok ok ok

error number error message
1 Error: copy HTTP 400 : Server Error
2 Error: copy Could not get the delegation id: Could not get proxy request: SOAP 1.1 fault: SOAP-ENV:Client [no subcode] Detail: connect failed in tcp_connect()
3 Error: copy (Neon): Could not read status line: connection was closed by server

from DPM (srm.glite.ecdf.ed.ac.uk)

source: srm.glite.ecdf.ed.ac.uk/dpm/ecdf.ed.ac.uk/home/atlas/atlasdatadisk/rucio/data17_comm/0f/ba/data17_comm.00324195.physics_Main.merge.AOD.f822_m1799._lb0421._0001.1
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
example command: gfal-copy davs+3rd://srm.glite.ecdf.ed.ac.uk/dpm/ecdf.ed.ac.uk/home/atlas/atlasdatadisk/rucio/data17_comm/0f/ba/data17_comm.00324195.physics_Main.merge.AOD.f822_m1799._lb0421._0001.1 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11
ls commands are only for the source

tool davs https davs+3rd https+3rd
davix-cp error 1 error 1 error 1 error 1
davix-get ok ok ok ok
davix-ls ok ok ok ok
gfal-copy ok ok error 1 error 1
gfal-ls ok ok ok ok

error number error message log message from dpm server
1 copy HTTP 405 : Method Not Allowed, Permission refused (115)Operation now in progress: This server does not allow remote copies () [405, #0]

NOTE: There is no entry in http or dynafed logs about tried access for the failed connections.

from local file

source: /tmp/testfile
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX

tool davs https davs+3rd https+3rd
davix-cp - - - -
gfal-copy ok ok ok ok

davix-cp can't copy local files
davs+3rd and https+3rd was used to just see how the tool itself handles that.

test of get commands from charon01

tool davs https davs+3rd https+3rdSorted ascending
davix-get ok ok error 1 error 1
davix-http ok ok error 2 error 2
gfal-cat ok ok ok ok

error number error message
1 Failure (Neon): Could not read status line: connection was closed by server after 3 attempts
2 Error: (Neon): Could not read status line: connection was closed by server

test of get commands from dpmhead-trunk

tool davs https davs+3rd https+3rd
davix-get ok ok ok ok
davix-http ok ok ok ok
gfal-cat ok ok ok ok

test of put commands to dynafed02

tool davs https davs+3rd https+3rd
davix-put ok ok error 1 error 1

error number error message
1 Error: HTTP 400 : Server Error

-- mebert - 2017-07-11

Comments


This topic: HEPrc > DynaFedCopyTest
Topic revision: r9 - 2017-09-11 - mebert
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback