Talend Open Studio Cookbook

Primarily designed as a reference book, simple and effective exercises based upon genuine real-world tasks enable the developer to reduce the time to deliver the results. Presentation of the activities in a recipe format will enable the readers to grasp even the complex concepts with consummate ease...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
1. Verfasser:	Barton, Rick
Format:	E-Book
Sprache:	Englisch
Veröffentlicht:	Birmingham Packt Publishing, Limited 2013 Packt Publishing
Ausgabe:	1
Schlagworte:	Data integration (Computer science) Database management
ISBN:	9781782167266, 1782167269
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Inhaltsangabe:

Intro -- Talend Open Studio Cookbook -- Table of Contents -- Talend Open Studio Cookbook -- Credits -- About the Author -- About the Reviewers -- www.PacktPub.com -- Support files, eBooks, discount offers and more -- Why Subscribe? -- Free Access for Packt account holders -- Preface -- What this book covers -- What you need for this book -- Who this book is for -- Conventions -- Reader feedback -- Customer support -- Downloading the example code -- Errata -- Piracy -- Questions -- 1. Introduction and General Principles -- Before you begin -- Installing the software -- How to do it… -- Enabling tHashInput and tHashOutput -- How to do it… -- 2. Metadata and Schemas -- Introduction -- Schema metadata -- Schemas -- Repository schemas -- Generic schemas -- Shared schemas -- Generated data sources -- Fixed schemas and columns -- Hand-cranking a built-in schema -- Getting ready -- How to do it… -- How it works… -- There's more... -- Date patterns -- Nullable elements -- Propagating schema changes -- Getting ready -- How to do it… -- How it works… -- There's more… -- Creating a generic schema from the existing metadata -- How to do it… -- How it works… -- Cutting and pasting schema information -- Getting ready -- How to do it… -- How it works… -- There's more… -- Dropping schemas to empty components -- Getting ready -- How to do it… -- How it works… -- There's more… -- Creating schemas from lists -- Getting ready -- How to do it... -- How it works… -- There's more… -- 3. Validating Data -- Introduction -- Enabling and disabling reject flows -- Getting ready -- How to do it… -- How it works… -- There's more... -- See also -- Gathering all rejects prior to killing a job -- Getting ready -- How to do it… -- How it works… -- There's more... -- See also -- Validating against the schema -- Getting ready -- How to do it… -- How it works
How it works... -- Setting the context file location in the operating system -- Getting ready -- How to do it... -- How it works… -- There's more… -- Variable not present -- Implicit context load -- 7. Working with Databases -- Introduction -- Setting up a database connection -- Getting ready -- How to do it... -- How it works... -- There's more… -- Using the connection -- Always create database connections -- Connection names -- Context -- Importing the table schemas -- Getting ready -- How to do it… -- How it works... -- There's more… -- Reading from database tables -- Getting ready -- How to do it… -- Selected rows and columns -- Multiple tables and complex queries -- How it works… -- There's more… -- Efficiency versus readability -- SQL string -- SQL style -- Using context and globalMap variables in SQL queries -- Getting ready -- How to do it… -- How it works… -- There's more… -- The globalMap variables -- Developing the query -- Reloading at each row -- Printing your input query -- Getting ready -- How to do it… -- How it works… -- There's more… -- Writing to a database table -- Getting ready -- How to do it… -- How it works… -- There's more… -- Creating tables -- Update and delete keys -- Batches -- Bulk loading -- Bulk loading to temp table -- Printing your output query -- Getting ready -- How to do it… -- How it works… -- There's more… -- Managing database sessions -- Getting ready -- How to do it… -- How it works… -- Executions -- There's more… -- Multiple outputs -- Don't forget the commit -- Committing but not closing -- Passing a session to a child job -- Getting ready -- How to do it… -- How it works… -- Selecting different fields and keys for insert, update, and delete -- Getting ready -- How to do it… -- How it works… -- There's more... -- Updating -- Deleting -- Capturing individual rejects and errors -- Getting ready
How to do it... -- How it works… -- Loading the data into memory -- The globalMap key -- The WHERE clause -- The result -- There's more… -- 5. Using Java in Talend -- Introduction -- Performing one-off pieces of logic using tJava -- Getting ready -- How to do it… -- How it works… -- See also -- Setting the context and globalMap variables using tJava -- Getting ready -- How to do it… -- How it works… -- There's more… -- See also -- Adding complex logic into a flow using tJavaRow -- Getting ready -- How to do it… -- How it works… -- Creating pseudo components using tJavaFlex -- Getting ready -- How to do it… -- How it works… -- There's more… -- Creating custom functions using code routines -- Getting ready -- How to do it… -- How it works… -- There's more… -- See also -- Importing JAR files to allow use of external Java classes -- Getting ready -- How to do it… -- How it works… -- There's more… -- 6. Managing Context Variables -- Introduction -- Transportable code -- Context variables -- Common values in contexts -- Passing command line parameters -- Setting context variables in the code -- Database context variables -- Creating a context group -- How to do it... -- How it works... -- There's more… -- Context types -- Prompt for variable values using the tree mode -- Adding a context group to your job -- Getting ready -- How to do it... -- How it works… -- There's more… -- Adding contexts to a context group -- Getting ready -- How to do it... -- There's more… -- Using tContextLoad to load contexts -- Getting ready -- How to do it... -- How it works... -- There's more… -- Print operations -- Warnings -- Context file location -- Using implicit context loading to load contexts -- Getting ready -- How to do it... -- How it works... -- There's more… -- Turning implicit context loading on and off in a job -- Getting ready -- How to do it...
Moving, copying, renaming, and deleting files and folders
How to do it… -- How it works… -- There's more… -- Die on error -- Efficiency -- Error management -- Database and table management -- Getting ready -- How to do it… -- How it works… -- There's more… -- Managing surrogate keys for parent and child tables -- Getting ready -- How to do it… -- How it works… -- There's more... -- Added efficiency using hashMap key table -- Ranges -- Sequences -- Auto increment keys -- The LastInsertId component -- Auto increment procedure -- Rewritable lookups using an in-process database -- Background -- Getting ready -- How to do it… -- How it works… -- In-memory components -- Initialize the data -- tMap -- Write back -- There's more… -- Memory -- See also -- 8. Managing Files -- Introduction -- Appending records to a file -- Getting ready -- How to do it... -- How it works... -- There's more… -- Concatenating files using the append method -- Reading rows using a regular expression -- Getting ready -- How to do it... -- How it works... -- There's more… -- Using temporary files -- Getting ready -- How to do it... -- How it works... -- There's more… -- See also -- Storing intermediate data in the memory using tHashMap -- Getting ready -- How to do it... -- How it works... -- There's more… -- Reading headers and trailers using tMap -- Getting ready -- How to do it... -- How it works... -- There's more… -- Reading headers and trailers with no identifiers -- Getting ready -- How to do it... -- How it works... -- Using the information in the header and trailer -- Getting ready -- How to do it... -- Validation subjob -- Use the header information subjob -- How it works... -- Validating using the trailer information -- Using the header information in the detail -- There's more… -- Adding a header and trailer to a file -- Getting ready -- How to do it... -- How it works... -- There's more… -- See also
Rejecting rows using tMap -- Getting ready -- How to do it… -- How it works… -- There's more… -- Checking a column against a list of allowed values -- Getting ready -- How to do it… -- How it works… -- There's more… -- See also -- Checking a column against a lookup -- Getting ready -- How to do it… -- How it works… -- Creating validation rules for more complex requirements -- Getting ready -- How to do it… -- How it works… -- There's more… -- See also -- Creating binary error codes to store multiple test results -- Getting ready -- How to do it… -- How it works… -- There's more… -- Decrypting the error code -- 4. Mapping Data -- Introduction -- The tMap component -- Single line of code -- Batch versus real time -- Simple mapping and tMap time savers -- Getting ready -- How to do it... -- How it works... -- There's more… -- Creating tMap expressions -- Getting ready -- How to do it... -- How it works... -- There's more… -- Testing expressions -- Expression editor -- Getting around the 'one line' limitation -- See Also -- Using the ternary operator for conditional logic -- Getting ready -- How to do it... -- Single ternary expression: if-then-else -- Ternary in ternary: if-then-elsif-then-else -- How it works… -- There's more… -- Using intermediate variables in tMap -- Getting ready -- How to do it… -- How it works… -- There's more… -- Filtering input rows -- Getting ready -- How to do it... -- How it works… -- There's more… -- Splitting an input row into multiple outputs based on input conditions -- Getting ready -- How to do it... -- How it works… -- There's more… -- Joining data using tMap -- Getting ready -- How to do it... -- How it works… -- There's more… -- See Also -- Hierarchical joins using tMap -- Getting ready -- How to do it... -- How it works… -- Using reload at each row to process real-time / near real-time data -- Getting ready