ibmos2spark

Type Members

1. class CloudObjectStorage extends AnyRef

   

   This class allows you to connect to an IBM cloud object storage (COS) instance.

   This class allows you to connect to an IBM cloud object storage (COS) instance. It also support connecting to a COS instance that is being hosted on bluemix.

   Constructor arguments:

   sparkcontext: a SparkContext object.

   credentials: a dictionary with the following required keys to connect to COS. The required keys differ according to the type of COS.

   • for COS type "softlayer_cos" the following key are required: endPoint [required] accessKey [required] secretKey [required]
   • for COS type "bluemix_cos", here are the required/optional key: endPoint [required] serviceId [required] apiKey OR iamToken depends on the selected authorization method (authMethod) [required] iamServiceEndpoint [optional] (default: https://iam.ng.bluemix.net/oidc/token) v2SignerType [optional]

   configurationName [optional]: string that identifies this configuration. You can use any string you like. This allows you to create multiple configurations to different Object Storage accounts. if a configuration name is not passed the default one will be used "service".

   cosType [optional]: string that identifies the type of COS to connect to. The supported types of COS are "softlayer_cos" and "bluemix_cos". "softlayer_cos" will be chosen as default if no cosType is passed.

   authMethod [optional]: string that identifies the type of authorization method to use when connecting to COS. This parameter is not reqired for softlayer_cos but only needed for bluemix_cos. Two options can be chosen for this params "api_key" or "iam_token". "api_key" will be chosen as default if the value is not set.

2. class bluemix extends AnyRef

   

   bluemix class sets up a swift connection between an IBM Spark service instance and an Object Storage instance provisioned through IBM Bluemix.

   bluemix class sets up a swift connection between an IBM Spark service instance and an Object Storage instance provisioned through IBM Bluemix.

   Constructor arguments:

   sparkcontext: a SparkContext object.

   credentials: a dictionary with the following required keys:

   auth_url

   project_id (or projectId)

   user_id (or userId)

   password

   region

   name: string that identifies this configuration. You can use any string you like. This allows you to create multiple configurations to different Object Storage accounts. This is not required at the moment, since credentials['name'] is still supported.

   When using this from a IBM Spark service instance that is configured to connect to particular Bluemix object store instances, the values for these credentials can be obtained by clicking on the 'insert to code' link just below a data source.

3. class softlayer extends AnyRef

   

   softlayer class sets up a swift connection between an IBM Spark service instance and Softlayer Object Storgae instance.

   softlayer class sets up a swift connection between an IBM Spark service instance and Softlayer Object Storgae instance.

   Constructor arguments

   sparkcontext: a SparkContext object.

   name: string that identifies this configuration. You can use any string you like. This allows you to create multiple configurations to different Object Storage accounts.

   auth_url, tenant, username and password: string credentials for your Softlayer Object Store