Difference: DynaFedCopyTest (1 vs. 9)

Revision 92017-09-11 - mebert

Line: 1 to 1
 

Test of copy tools with dynafed

Version of tools needed to have it working

Revision 82017-08-15 - mebert

Line: 1 to 1
 

Test of copy tools with dynafed

Version of tools needed to have it working

  • on the source site (at least if DPM)
Changed:
<
<
    • latest certificates for the sites involved
>
>
    • latest certificates for the sites involved (under /etc/grid-security/certificates)
 
    • nss 3.28 or later
    • davix and davix-libs 0.6.6 or later
    • latest openssl
    • /var/www/proxycache owned by the DPM user, usually dpmmgr:dpmmgr
    • NSSFlags and DiskFlags in the apache config need to be set to: Write RemoteCopy
  • on the destination site (dynafed)
Changed:
<
<
    • latest certificates for the sites involved
>
>
    • latest certificates for the sites involved (under /etc/grid-security/certificates)
 
    • nss 3.28 or later
    • davix and davix-libs 0.6.6 or later
    • latest openssl

Revision 72017-08-11 - mebert

Line: 1 to 1
 

Test of copy tools with dynafed

Version of tools needed to have it working

Line: 17 to 17
 
  • on the client machine
    • davix(-libs) at least in version 0.6.6
      • Make sure the correct version is used if CVMFS software is in the PATH/LD_LIBRARY_PATH!
Added:
>
>
    • create the entries needed to make the certificates available for authentication of sites
      • sudo cp /etc/grid-security/certificates/* /etc/pki/ca-trust/source/anchors/
      • sudo update-ca-trust extract
 

Revision 62017-08-10 - mebert

Line: 1 to 1
 

Test of copy tools with dynafed

Added:
>
>

Version of tools needed to have it working

  • on the source site (at least if DPM)
    • latest certificates for the sites involved
    • nss 3.28 or later
    • davix and davix-libs 0.6.6 or later
    • latest openssl
    • /var/www/proxycache owned by the DPM user, usually dpmmgr:dpmmgr
    • NSSFlags and DiskFlags in the apache config need to be set to: Write RemoteCopy
  • on the destination site (dynafed)
    • latest certificates for the sites involved
    • nss 3.28 or later
    • davix and davix-libs 0.6.6 or later
    • latest openssl
  • on the client machine
    • davix(-libs) at least in version 0.6.6
      • Make sure the correct version is used if CVMFS software is in the PATH/LD_LIBRARY_PATH!

from DPM (dpmhead-trunk@cern)

source: dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
example command: gfal-copy davs+3rd://dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11
ls commands are only for the source

tool davs https davs+3rd https+3rd
davix-cp ok ok ok ok
davix-ls ok ok ok ok
gfal-copy ok ok ok ok
gfal-ls ok ok ok ok

*---------------> old tests below

 

from dCache (charon01)

source: charon01.westgrid.ca:2880/pnfs/westgrid.uvic.ca/data/atlas/atlasdatadisk/rucio/mc12_8TeV/8d/c0/EVNT.01212395._000004.pool.root.1
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
Line: 20 to 57
 
2 Error: copy Could not get the delegation id: Could not get proxy request: SOAP 1.1 fault: SOAP-ENV:Client [no subcode] Detail: connect failed in tcp_connect()
3 Error: copy (Neon): Could not read status line: connection was closed by server
Deleted:
<
<

from DPM (dpmhead-trunk@cern)

source: dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
example command: gfal-copy davs+3rd://dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11
ls commands are only for the source

tool davs https davs+3rd https+3rd
davix-cp error 1 error 1 error 1 error 1
davix-ls ok ok ok ok
gfal-copy ok ok error 1 error 1
gfal-ls ok ok ok ok

error number error message
1 Transfer failed: Failed: Remote copy failed with status code 0: SSL connect error
 

from DPM (srm.glite.ecdf.ed.ac.uk)

source: srm.glite.ecdf.ed.ac.uk/dpm/ecdf.ed.ac.uk/home/atlas/atlasdatadisk/rucio/data17_comm/0f/ba/data17_comm.00324195.physics_Main.merge.AOD.f822_m1799._lb0421._0001.1

Revision 52017-07-14 - mebert

Line: 1 to 1
 

Test of copy tools with dynafed

from dCache (charon01)

Line: 16 to 16
 

error number error message
Changed:
<
<
1 (Davix::HttpRequest) Error: copy HTTP 400 : Server Error
2 (Davix::HttpThirdPartyCopy::Delegation) Error: copy Could not get the delegation id: Could not get proxy request: SOAP 1.1 fault: SOAP-ENV:Client [no subcode] Detail: connect failed in tcp_connect()
3 (Davix::HttpRequest) Error: copy (Neon): Could not read status line: connection was closed by server
>
>
1 Error: copy HTTP 400 : Server Error
2 Error: copy Could not get the delegation id: Could not get proxy request: SOAP 1.1 fault: SOAP-ENV:Client [no subcode] Detail: connect failed in tcp_connect()
3 Error: copy (Neon): Could not read status line: connection was closed by server
 

from DPM (dpmhead-trunk@cern)

source: dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00
Line: 83 to 83
 
gfal-cat ok ok ok ok

error number error message
Changed:
<
<
1 (Davix::HttpRequest) Error: Failure (Neon): Could not read status line: connection was closed by server after 3 attempts
2 (Davix::HttpRequest) Error: (Neon): Could not read status line: connection was closed by server
>
>
1 Failure (Neon): Could not read status line: connection was closed by server after 3 attempts
2 Error: (Neon): Could not read status line: connection was closed by server
 

test of get commands from dpmhead-trunk

tool davs https davs+3rd https+3rd
Line: 98 to 98
 
davix-put ok ok error 1 error 1

error number error message
Changed:
<
<
1 (Davix::HttpRequest) Error: HTTP 400 : Server Error
>
>
1 Error: HTTP 400 : Server Error

 -- mebert - 2017-07-11

Comments

Revision 42017-07-12 - mebert

Line: 1 to 1
 

Test of copy tools with dynafed

from dCache (charon01)

Line: 30 to 30
 
tool davs https davs+3rd https+3rd
davix-cp error 1 error 1 error 1 error 1
davix-ls ok ok ok ok
Changed:
<
<
gfal-copy ok ok error 2 error 2
>
>
gfal-copy ok ok error 1 error 1
 
gfal-ls ok ok ok ok

error number error message
Changed:
<
<
1 (Davix::HttpThirdPartyCopy) Error: Transfer failed: Failed: Remote copy failed with status code 0: SSL connect error
2 gfal-copy error: 5 (Input/output error) - TRANSFER Transfer failed: Failed: Remote copy failed with status code 0: SSL connect error
>
>
1 Transfer failed: Failed: Remote copy failed with status code 0: SSL connect error
 

Revision 32017-07-12 - mebert

Line: 1 to 1
 

Test of copy tools with dynafed

from dCache (charon01)

Line: 40 to 40
 
Added:
>
>

from DPM (srm.glite.ecdf.ed.ac.uk)

source: srm.glite.ecdf.ed.ac.uk/dpm/ecdf.ed.ac.uk/home/atlas/atlasdatadisk/rucio/data17_comm/0f/ba/data17_comm.00324195.physics_Main.merge.AOD.f822_m1799._lb0421._0001.1
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
example command: gfal-copy davs+3rd://srm.glite.ecdf.ed.ac.uk/dpm/ecdf.ed.ac.uk/home/atlas/atlasdatadisk/rucio/data17_comm/0f/ba/data17_comm.00324195.physics_Main.merge.AOD.f822_m1799._lb0421._0001.1 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11
ls commands are only for the source

tool davs https davs+3rd https+3rd
davix-cp error 1 error 1 error 1 error 1
davix-get ok ok ok ok
davix-ls ok ok ok ok
gfal-copy ok ok error 1 error 1
gfal-ls ok ok ok ok

error number error message log message from dpm server
1 copy HTTP 405 : Method Not Allowed, Permission refused (115)Operation now in progress: This server does not allow remote copies () [405, #0]

NOTE: There is no entry in http or dynafed logs about tried access for the failed connections.

 

from local file

source: /tmp/testfile
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX

Revision 22017-07-12 - frank

Line: 1 to 1
 

Test of copy tools with dynafed

Changed:
<
<

from charon01

>
>

from dCache (charon01)

 source: charon01.westgrid.ca:2880/pnfs/westgrid.uvic.ca/data/atlas/atlasdatadisk/rucio/mc12_8TeV/8d/c0/EVNT.01212395._000004.pool.root.1
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
example command: gfal-copy davs+3rd://charon01.westgrid.ca:2880/pnfs/westgrid.uvic.ca/data/atlas/atlasdatadisk/rucio/mc12_8TeV/8d/c0/EVNT.01212395._000004.pool.root.1 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11
Line: 20 to 20
 
2 (Davix::HttpThirdPartyCopy::Delegation) Error: copy Could not get the delegation id: Could not get proxy request: SOAP 1.1 fault: SOAP-ENV:Client [no subcode] Detail: connect failed in tcp_connect()
3 (Davix::HttpRequest) Error: copy (Neon): Could not read status line: connection was closed by server
Changed:
<
<

from dpmhead-trunk@cern

>
>

from DPM (dpmhead-trunk@cern)

 source: dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
example command: gfal-copy davs+3rd://dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11

Revision 12017-07-11 - mebert

Line: 1 to 1
Added:
>
>

Test of copy tools with dynafed

from charon01

source: charon01.westgrid.ca:2880/pnfs/westgrid.uvic.ca/data/atlas/atlasdatadisk/rucio/mc12_8TeV/8d/c0/EVNT.01212395._000004.pool.root.1
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
example command: gfal-copy davs+3rd://charon01.westgrid.ca:2880/pnfs/westgrid.uvic.ca/data/atlas/atlasdatadisk/rucio/mc12_8TeV/8d/c0/EVNT.01212395._000004.pool.root.1 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11
ls commands are only for the source
NOTE: The output of the ls commands is different for davix-ls and gfal-ls. While the size and name are fine, all other attributes can differ, especially also the reported time stamp.

tool davs https davs+3rd https+3rd
davix-cp error 1 error 2 error 3 error 3
davix-ls ok ok error 3 error 3
gfal-copy ok ok error 2 error2
gfal-ls ok ok ok ok

error number error message
1 (Davix::HttpRequest) Error: copy HTTP 400 : Server Error
2 (Davix::HttpThirdPartyCopy::Delegation) Error: copy Could not get the delegation id: Could not get proxy request: SOAP 1.1 fault: SOAP-ENV:Client [no subcode] Detail: connect failed in tcp_connect()
3 (Davix::HttpRequest) Error: copy (Neon): Could not read status line: connection was closed by server

from dpmhead-trunk@cern

source: dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX
example command: gfal-copy davs+3rd://dpmhead-trunk.cern.ch/dpm/cern.ch/home/atlas/atlasdyanfed/testfile00 davs://dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copy11
ls commands are only for the source

tool davs https davs+3rd https+3rd
davix-cp error 1 error 1 error 1 error 1
davix-ls ok ok ok ok
gfal-copy ok ok error 2 error 2
gfal-ls ok ok ok ok

error number error message
1 (Davix::HttpThirdPartyCopy) Error: Transfer failed: Failed: Remote copy failed with status code 0: SSL connect error
2 gfal-copy error: 5 (Input/output error) - TRANSFER Transfer failed: Failed: Remote copy failed with status code 0: SSL connect error

from local file

source: /tmp/testfile
destination: dynafed02.heprc.uvic.ca:8443/atlas/scratchdisk/test_copyXX

tool davs https davs+3rd https+3rd
davix-cp - - - -
gfal-copy ok ok ok ok

davix-cp can't copy local files
davs+3rd and https+3rd was used to just see how the tool itself handles that.

test of get commands from charon01

tool davs https davs+3rd https+3rd
davix-get ok ok error 1 error 1
davix-http ok ok error 2 error 2
gfal-cat ok ok ok ok

error number error message
1 (Davix::HttpRequest) Error: Failure (Neon): Could not read status line: connection was closed by server after 3 attempts
2 (Davix::HttpRequest) Error: (Neon): Could not read status line: connection was closed by server

test of get commands from dpmhead-trunk

tool davs https davs+3rd https+3rd
davix-get ok ok ok ok
davix-http ok ok ok ok
gfal-cat ok ok ok ok

test of put commands to dynafed02

tool davs https davs+3rd https+3rd
davix-put ok ok error 1 error 1

error number error message
1 (Davix::HttpRequest) Error: HTTP 400 : Server Error
-- mebert - 2017-07-11

Comments

<--/commentPlugin-->
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback