... | ... | @@ -12,21 +12,21 @@ To install the component you have to do following steps: |
|
|
* clone “b2share” branch of the B2SAFE core from GitHub https://github.com/EUDAT-B2SAFE/B2SAFE-core.git to get the scripts and rules you need for B2SHARE connection establishing.
|
|
|
* add following scripts to \<your path to B2SAFE\>/B2SAFE-core/cmd:
|
|
|
* b2shareclient.py
|
|
|
* b2shareclientworker.py
|
|
|
* b2shareclientCLI.py
|
|
|
* configuration.py
|
|
|
* create_md_file.py
|
|
|
* and the whole directory “manifest”
|
|
|
* add the rule “b2share.re” from B2SAFE-core/rulebase folder to the "rulebase" folder of your iRODS instance \<your path to B2SAFE\>/B2SAFE-core/rulebase
|
|
|
* add to your configuration folder \<your path to B2SAFE\>/B2SAFE-core/conf and modify the configuration files “b2safe_b2share.conf” and “create_md_file.conf” according to your environment as described in "Configuration" section of this wiki page
|
|
|
* check for missing python libraries trying to run the major scripts with -d (dry run) option
|
|
|
* b2shareclient.py
|
|
|
* b2shareclientCLI.py
|
|
|
* create_md_file.py
|
|
|
Try to install missing packages with the standard package manager like apt, yum, zypper etc.
|
|
|
If packages are not within the standards install and use pip to install the missing packages.
|
|
|
As described in the “Example Workflow” section, the iRODS rules will trigger these scripts according to the flags user specify in the imeta of the collection.
|
|
|
|
|
|
# Configuration
|
|
|
There are two major scripts for the B2SHARE connection component b2shareclient.py and create_md_file.py that are using the configuration files b2safe_b2share.conf” and “create_md_file.conf” accordingly. The configurable options are identical. The configurations were only separated as the scripts could be used independently. It consists of 3 parts. The first tow sections `Logging` and `B2SHARE_HTTP_API` are prefilled with default values.
|
|
|
There are two major scripts for the B2SHARE connection component b2shareclientCLI.py and create_md_file.py that are using the configuration files “b2safe_b2share.conf” and “create_md_file.conf” accordingly. The configurable options are identical. The configurations were only separated as the scripts could be used independently. It consists of 3 parts. The first tow sections `Logging` and `B2SHARE_HTTP_API` are prefilled with default values.
|
|
|
* `Logging` - with 2 values needed to be specified: log level, default is **log_level=DEBUG**, and the file where to safe the logging information, default value is **log_file=log/ b2safe_b2share.log**, so in a file named b2safe_b2share.log in folder log under the installation path.
|
|
|
* `B2SHARE_HTTP_API` - with 1 value needed to be specified: host_name of the B2SHARE instance, default is the address of the training instance of B2SHARE **host_name=https://trng-b2share.eudat.eu/**. Should the HTTP API of B2SHARE be changed in the meantime, than the values for the required endpoints are needed to be adapted.
|
|
|
* `iRODS` - connection information of the iRODS instance: the name of the iRODS zone like **zone_name=Zone1** and the endpoint consisting of the host_name:port information like **zone_ep=localhost:1247** and the information about the iRODS **resources** to be used as well as the logical home directory in iRODS like **irods_home_dir=/Zone1/home**.
|
... | ... | @@ -50,7 +50,7 @@ Following workflow was considered during the component development: |
|
|
value: EUDAT
|
|
|
units:
|
|
|
```
|
|
|
4. The user can modify the draft via B2SHARE or the drafted collection in B2SAFE. What can be done by creating and filling a specific metadata file in the collection and using the **b2shareclient.py** to add these metadata to the draft. Once she is ready, she can add a specific metadata attribute to the collection:
|
|
|
4. The user can modify the draft via B2SHARE or the drafted collection in B2SAFE. What can be done by creating and filling a specific metadata file in the collection and using the **b2shareclientCLI.py** to add these metadata to the draft. Once she is ready, she can add a specific metadata attribute to the collection:
|
|
|
`imeta add -C \<collection_X\> EUDAT_B2SHARE_PUBLISHME null`
|
|
|
|
|
|
5. The B2SAFE administrator executes a rule to scan the repository for publications:
|
... | ... | @@ -70,10 +70,10 @@ Following assumptions were made: |
|
|
* the collections are copied and published just one time, even if the rule to publish them is executed multiple times.
|
|
|
|
|
|
# Available functions of the python scripts
|
|
|
The B2SAFE B2SHARE client **b2shareclient.py** has 3 major functions: draft, meta, pub; to create a draft, add metadata to and publish it. Further it has helping functions to list all names of communities available in B2SHARE, to get the schema of a community, to understand possible key value pairs of metadata file user has to fill, to get a specific draft to check it before publishing and to delete a draft if it is not correct. The arguments that are needed to be specified for every call are the **--confpath** the path to the configuration file for the b2shareclient.py and the **--userName** name if the user in iRODS, to get the B2SHARE access token of the user from his meta data.
|
|
|
The B2SAFE B2SHARE client **b2shareclientCLI.py** has 3 major functions: draft, meta, pub; to create a draft, add metadata to and publish it. Further it has helping functions to list all names of communities available in B2SHARE, to get the schema of a community, to understand possible key value pairs of metadata file user has to fill, to get a specific draft to check it before publishing and to delete a draft if it is not correct. The arguments that are needed to be specified for every call are the **--confpath** the path to the configuration file for the b2shareclientCLI.py and the **--userName** name if the user in iRODS, to get the B2SHARE access token of the user from his meta data.
|
|
|
```
|
|
|
python b2shareclient.py -h
|
|
|
usage: b2shareclient.py [-h] [--confpath CONFPATH] [-dbg] [-d] [-u USERNAME]
|
|
|
python b2shareclientCLI.py -h
|
|
|
usage: b2shareclientCLI.py [-h] [--confpath CONFPATH] [-dbg] [-d] [-u USERNAME]
|
|
|
{draft,meta,pub,listCommunities,communitySchema,getDraft,deleteDraft}
|
|
|
B2SAFE B2SHARE client
|
|
|
positional arguments:
|
... | ... | @@ -96,8 +96,8 @@ optional arguments: |
|
|
```
|
|
|
Depending on what function is called further options are needed to be specified.
|
|
|
```
|
|
|
python b2shareclient.py draft -h
|
|
|
usage: b2shareclient.py draft [-h] (-c COMMUNITYNAME | -i COMMUNITYID) [-ti TITLE] -cp COLLECTIONPATH
|
|
|
python b2shareclientCLI.py draft -h
|
|
|
usage: b2shareclientCLI.py draft [-h] (-c COMMUNITYNAME | -i COMMUNITYID) [-ti TITLE] -cp COLLECTIONPATH
|
|
|
optional arguments:
|
|
|
-h, --help show this help message and exit
|
|
|
-c COMMUNITYNAME, --communityName COMMUNITYNAME B2Share community name
|
... | ... | @@ -105,44 +105,48 @@ optional arguments: |
|
|
-ti TITLE, --title TITLE title of the record
|
|
|
-cp COLLECTIONPATH, --collectionPath COLLECTIONPATH path to the collection in iRODS with files
|
|
|
|
|
|
python b2shareclient.py meta -h
|
|
|
usage: b2shareclient.py meta [-h] -id RECORD_ID -md METADATA
|
|
|
python b2shareclientCLI.py meta -h
|
|
|
usage: b2shareclientCLI.py meta [-h] -id RECORD_ID -md METADATA
|
|
|
optional arguments:
|
|
|
-h, --help show this help message and exit
|
|
|
-id RECORD_ID, --record_id RECORD_ID the b2share id of the record
|
|
|
-md METADATA, --metadata METADATA path to the metadata JSON file of the record
|
|
|
|
|
|
python b2shareclient.py pub -h
|
|
|
usage: b2shareclient.py pub [-h] -pi REC_ID
|
|
|
python b2shareclientCLI.py pub -h
|
|
|
usage: b2shareclientCLI.py pub [-h] -pi REC_ID
|
|
|
optional arguments:
|
|
|
-h, --help show this help message and exit
|
|
|
-pi REC_ID, --rec_id REC_ID the b2share id of the record
|
|
|
|
|
|
python b2shareclient.py listCommunities -h
|
|
|
usage: b2shareclient.py listCommunities [-h]
|
|
|
python b2shareclientCLI.py listCommunities -h
|
|
|
usage: b2shareclientCLI.py listCommunities [-h]
|
|
|
optional arguments:
|
|
|
-h, --help show this help message and exit
|
|
|
|
|
|
python b2shareclient.py communitySchema -h
|
|
|
usage: b2shareclient.py communitySchema [-h] (-cn COMMNAME | -ci COMMID)
|
|
|
python b2shareclientCLI.py communitySchema -h
|
|
|
usage: b2shareclientCLI.py communitySchema [-h] (-cn COMMNAME | -ci COMMID)
|
|
|
optional arguments:
|
|
|
-h, --help show this help message and exit
|
|
|
-cn COMMNAME, --commName COMMNAME B2Share community name
|
|
|
-ci COMMID, --commID COMMID B2Share community id
|
|
|
|
|
|
python b2shareclient.py getDraft -h
|
|
|
usage: b2shareclient.py getDraft [-h] -di DRAFT_ID
|
|
|
python b2shareclientCLI.py getDraft -h
|
|
|
usage: b2shareclientCLI.py getDraft [-h] -di DRAFT_ID
|
|
|
optional arguments:
|
|
|
-h, --help show this help message and exit
|
|
|
-di DRAFT_ID, --draft_id DRAFT_ID the b2share id of the record
|
|
|
|
|
|
python b2shareclient.py deleteDraft -h
|
|
|
usage: b2shareclient.py deleteDraft [-h] -ddi DRAFT_TO_DELETE_ID
|
|
|
python b2shareclientCLI.py deleteDraft -h
|
|
|
usage: b2shareclientCLI.py deleteDraft [-h] -ddi DRAFT_TO_DELETE_ID
|
|
|
optional arguments:
|
|
|
-h, --help show this help message and exit
|
|
|
-ddi DRAFT_TO_DELETE_ID, --draft_to_delete_id DRAFT_TO_DELETE_ID the b2share id of the record
|
|
|
|
|
|
e.g.
|
|
|
python b2shareclient.py -dbg -d --confpath conf/b2share_connection.conf -u julia draft -c EUDAT -ti testCollectionDarft5 -cp /JuliasZone/home/julia/testcollection/collection_A
|
|
|
|
|
|
```
|
|
|
To add some describing meta data to the collection of files, the user want to publish on B2SHARE, it is possible to add a metadata file directly into the collection before triggering the publication process. This file needs to be written in a specific way so it can be processed by the **b2shareclient.py** and its information send to B2SHARE over the REST API. The script **create_md_file.py** can help one to create such a file for specified community. It is requesting the schema of the community and saves optional and required options. The user needs then fill the values in for the options he wants to be saved in the meta data of the draft.
|
|
|
To add some describing meta data to the collection of files, the user want to publish on B2SHARE, it is possible to add a metadata file directly into the collection before triggering the publication process. This file needs to be written in a specific way so it can be processed by the **b2shareclientCLI.py** and its information send to B2SHARE over the REST API. The script **create_md_file.py** can help one to create such a file for specified community. It is requesting the schema of the community and saves optional and required options. The user needs then fill the values in for the options he wants to be saved in the meta data of the draft.
|
|
|
```
|
|
|
python create_md_file.py -h
|
|
|
usage: create_md_file.py [-h] [--confpath CONFPATH] [-c COMMUNITYNAME] [-dbg] [-d] [-u USERNAME] [--collectionName COLLECTIONNAME]
|
... | ... | @@ -218,4 +222,22 @@ language |
|
|
German [deu]
|
|
|
```
|
|
|
|
|
|
# iRODS rules |
|
|
\ No newline at end of file |
|
|
# iRODS rules
|
|
|
|
|
|
# Unit tests
|
|
|
There are unit tests for the 3 scripts of the connection component in the repository under /scripts/tests/testB2shareConnectionClient:
|
|
|
* b2shareclienttest.py for the worker class B2shareClient in b2shareclient.py
|
|
|
* b2shareclient_cli_test.py for the b2shareclientCLI.py
|
|
|
* configurationtest.py for the configuration.py
|
|
|
All this test can be executed at ones with the "b2shareclient_testsuite.py", that can be find one folder back in /scripts/tests, or each for its own.
|
|
|
|
|
|
e.g.
|
|
|
```
|
|
|
cd /home/irods/B2SAFE-core/scripts/tests/
|
|
|
python b2shareclient_testsuite.py
|
|
|
|
|
|
.......................
|
|
|
----------------------------------------------------------------------
|
|
|
Ran 23 tests in 0.107s
|
|
|
OK
|
|
|
``` |
|
|
\ No newline at end of file |