Generate extraction scripts based on a given domain
The schemas should at least, specify :
a table name (schemas.name)
a file pattern (schemas.pattern) which is used as the export file base name
a write mode (schemas.metadata.write): APPEND or OVERWRITE
a delta column (schemas.merge.timestamp) if in APPEND mode : the column which is used to determine new rows for each exports
the columns to extract (schemas.attributes.name*)
You also have to provide a Mustache (http://mustache.github.io/mustache.5.html) template file.
Here you'll write your extraction export process (sqlplus for Oracle, pgsql for PostgreSQL as an example).
In that template you can use the following parameters:
table_name -> the table to export
delimiter -> the resulting dsv file delimiter
columns -> the columns to export
columns is a Mustache map, it gives you access, for each column, to:
name -> the column name
trailing_col_char -> the separator to append to the column (, if there are more columns to come, "" otherwise)
Here is an example how to use it in a template:
SELECT
{{#columns}}
TO_CHAR({{name}}){{trailing_col_char}}
{{/columns}}
FROM
{{table_name}};
export_file -> the export file name
full_export -> if the export is a full or delta export (the logic is to be implemented in your script)
Usage: comet [script-gen] [options]
Command: script-gen
--domain <value> The domain for which to generate extract scripts
--templateFile <value> Script template file
--scriptsOutputDir <value> Scripts output folder
Linear Supertypes
StrictLogging, App, DelayedInit, AnyRef, Any
Ordering
Alphabetic
By Inheritance
Inherited
Main
StrictLogging
App
DelayedInit
AnyRef
Any
Hide All
Show All
Visibility
Public
All
Value Members
final def!=(arg0: Any): Boolean
Definition Classes
AnyRef → Any
final def##(): Int
Definition Classes
AnyRef → Any
final def==(arg0: Any): Boolean
Definition Classes
AnyRef → Any
valarglist: List[String]
defargs: Array[String]
Attributes
protected
Definition Classes
App
Annotations
@deprecatedOverriding("args should not be overridden", "2.11.0")
Generate extraction scripts based on a given domain The schemas should at least, specify :
You also have to provide a Mustache (http://mustache.github.io/mustache.5.html) template file.
Here you'll write your extraction export process (sqlplus for Oracle, pgsql for PostgreSQL as an example). In that template you can use the following parameters:
table_name -> the table to export delimiter -> the resulting dsv file delimiter columns -> the columns to export columns is a Mustache map, it gives you access, for each column, to:
Usage: comet [script-gen] [options]
Command: script-gen --domain <value> The domain for which to generate extract scripts --templateFile <value> Script template file --scriptsOutputDir <value> Scripts output folder