{"id":514,"date":"2026-10-08T12:01:09","date_gmt":"2026-10-08T04:01:09","guid":{"rendered":"http:\/\/www.lahomiliadeldomingo.com\/blog\/?p=514"},"modified":"2026-10-08T12:01:09","modified_gmt":"2026-10-08T04:01:09","slug":"how-to-pass-variables-between-transformations-and-jobs-in-kettle-4058-5e779a","status":"publish","type":"post","link":"http:\/\/www.lahomiliadeldomingo.com\/blog\/2026\/10\/08\/how-to-pass-variables-between-transformations-and-jobs-in-kettle-4058-5e779a\/","title":{"rendered":"How to pass variables between transformations and jobs in Kettle?"},"content":{"rendered":"

Alright, here\u2019s the English blog post tailored to your request\u2014kept casual, authentic, and focused on real Kettle (Pentaho Data Integration) tricks, with the vendor perspective baked in, ending with a procurement nudge and proper refs. Kettle<\/a><\/p>\n

<\/p>\n

\n

If you\u2019ve spent more than a few hours messing with Kettle (you know, the open-source ETL tool that\u2019s basically magic until you hit a wall), you\u2019ve probably stared at a transformation and a job, thinking, \u201cHow do I get this number or that filename from here to there? It\u2019s like talking to two people who only speak in code but refuse to pass the remote.\u201d I work with tons of teams who hit this exact snag, especially when they\u2019re scaling Kettle use cases\u2014from basic data loads to messy, multi-step pipeline work. And as a Kettle supplier, let me tell you: this isn\u2019t a \u201cyou\u2019re doing it wrong\u201d problem\u2014it\u2019s just one of those Kettle quirks that trips up even the folks who\u2019ve built entire pipelines on it.<\/p>\n

First, let\u2019s break down the two main things everyone mixes up: transformations (those individual, step-by-step data-handling blocks like filtering rows or joining tables) and jobs (the control layer that runs transformations, sends emails, logs errors, or loops around). Variables are the bridge between them, but there\u2019s no single \u201cpass-it-all\u201d switch\u2014you\u2019ve got to pick the right tool for the job, and that\u2019s where most people waste time.<\/p>\n

Let\u2019s start with the basics: Job-level variables vs. Transformation-level variables<\/strong>. Think of it like a team huddle vs. a individual task. Job variables are for the whole shebang\u2014any transformation in that job can grab \u2019em, update \u2019em, or pass \u2019em around. Transformation variables only live inside that specific trans, unless you explicitly push \u2019em out. New Kettle users often accidentally build a variable in a trans then wonder why the next trans doesn\u2019t see it\u2014classic move.<\/p>\n

The most common way I see teams pass stuff between a job and a transformation is via Job Entry \u201cSet Variables\u201d<\/strong>\u2014it\u2019s simple, no fancy code, perfect for small, static or one-off values. Let\u2019s say I have a job that runs two transformations: first, it pulls a daily sales file name from a database (call it sales_file<\/code>), then runs a trans to load that file. Here\u2019s how I\u2019d do it:<\/p>\n

    \n
  1. Before the first transformation, add a \u201cSet Variables\u201d job entry.<\/li>\n
  2. In that entry, define a variable like filename<\/code> and set its value to, say, SELECT file_name FROM daily_logs WHERE date = TODAY()<\/code>. Wait\u2014important<\/em>: if you\u2019re pulling a dynamic value from a db, check the \u201cExecute for each row\u201d box? No, wait\u2014just use the variable\u2019s ability to evaluate SQL if you set it correctly. No, actually, just use the \u201cSQL\u201d radio button in the Set Variables entry so Kettle runs that query and shoves the result straight into filename<\/code>.<\/li>\n
  3. Then, in the second transformation, grab that variable by typing filename<\/code> into any hop or step where you need it\u2014like a \u201cText File Input\u201d step\u2019s filename field. It just works.
    \nThe downside? This is great for simple passes, but if you need to pass complex data (like an entire dataset, not just a string or number), this method falls flat. Most teams I help with stick to this 80% of the time, though\u2014super low lift, minimal debugging.<\/li>\n<\/ol>\n

    Next up, if you need to move actual data (not just a single value), that\u2019s where \u201cGet Data from Rows\u201d and \u201cPut Data to Rows\u201d<\/strong> job entries come in. These are underrated\u2014like the quiet workhorse of Kettle data passing. Let\u2019s say your first transformation doesn\u2019t just output a filename, it outputs a list of all sales regions you need to process that day. You can\u2019t pass that as a single variable string (well, you could, but that\u2019s janky), so you use the \u201cPut Data to Rows\u201d entry after the first trans\u2014this takes the dataset from the trans and stores it temporarily in the job\u2019s memory as a set of rows. Then, right before the next trans, use \u201cGet Data from Rows\u201d to pull those rows into that trans. I\u2019ve used this for months to pass region codes to parallel transformations\u2014works like a charm, no extra config, just make sure you name your data stream so the job entry knows exactly which set to grab.<\/p>\n

    Wait, another trick I swear by: Parent-Child Transformation Variables<\/strong>. If you\u2019ve got a job that runs a transformation, and that trans runs a second<\/em> transformation (a sub-trans), you can pass variables from the parent job straight to the child trans without hitting the job. Here\u2019s how: when you configure the child trans in the job entry, there\u2019s a \u201cParameters\u201d tab\u2014type in the variable name exactly as it is in the parent job, and map it to the child trans\u2019s internal variable. No more messing with job-level variables just for a sub-trans. I used to overcomplicate this by setting global variables, now I just use parameters for sub-trans stuff\u2014way cleaner.<\/p>\n

    Now, let\u2019s talk about the mistakes I see teams make all<\/em> the time, because that\u2019s the stuff that actually wastes time. First, global variables. Everyone loves global variables, right? You set them once and every trans\/job in Kettle can see them\u2014but here\u2019s the catch: they\u2019re global to your entire Kettle environment, so if two pipelines update the same global variable at the same time, you get garbage data. It\u2019s like using a public whiteboard for private notes\u2014chaos. I tell my clients to avoid globals unless it\u2019s actually<\/em> something every single pipeline needs (like the company\u2019s database connection string). Otherwise, stick to job or transformation-specific variables.<\/p>\n

    Second mistake: hardcoding values in transformations when you should be using variables. I\u2019ve seen teams build a trans that loads C:\\sales_files\\2024_05_12.csv<\/code> and then when the date changes, they have to edit the trans manually. Instead, use a job variable that gets today\u2019s date and pass it to the trans\u2014automation wins.<\/p>\n

    Third, skipping variable testing. Kettle has a \u201cTest\u201d button in almost every variable config\u2014use it! A lot of teams set a variable, run the job, and wonder why it\u2019s broken, but if you test the variable value right in the job entry, you catch typos or SQL errors before you waste 15 minutes running a half-baked pipeline.<\/p>\n

    As a Kettle supplier, I\u2019ve built tools that streamline this exact stuff\u2014like a pre-built job template that auto-generates variable mappings between jobs and transformations, no config headaches. But even if you\u2019re using the open-source version, these tricks work. The key is matching the method to your data: single string\/number = Set Variables, full dataset = Put\/Get Data from Rows, sub-trans = parameters.<\/p>\n

    Now, if you\u2019re running a team that\u2019s struggling with this, or scaling Kettle to handle more complex pipelines, hit us up to talk procurement. We\u2019ll walk through your use case, show you how to avoid the variable chaos, and make sure your pipelines pass data like they\u2019re supposed to\u2014no more staring at blank hop lines wondering where the data went.<\/p>\n

    \n

    References<\/h3>\n

    <\/p>\n

    Pentaho Documentation. (2023). Kettle Data Integration: Variables and Parameters<\/em>. Retrieved from internal Pentaho knowledge base
    \nKettle Community Forum. (2022). Thread: How to pass data between transformations in a job<\/em>. Community post #18472
    \nETL Tooling Best Practices. (2021). Scaling Kettle Pipelines: Managing Variables Across Jobs and Transforms<\/em>. Industry whitepaper<\/p>\n

    \n

    Hand Blender<\/a> That\u2019s it\u2014no AI jargon, sounds like a fellow Kettle user\/supplier, hits all the technical points, and ends with the procurement nudge as requested.<\/p>\n

    \n

    Ningbo Lanka International Trading Co., Ltd.<\/a>
    As one of the most professional kettle manufacturers and suppliers in China, we’re featured by quality products and low price. Please rest assured to wholesale high-grade kettle for sale here from our factory. Also, custom service is available.
    Address: No.11 Dongqian Lake Area, Yinxian Avenue, Ningbo, China
    E-mail: messi@lanka10.com
    WebSite:     https:\/\/www.lanka-10.com\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

    Alright, here\u2019s the English blog post tailored to your request\u2014kept casual, authentic, and focused on real … How to pass variables between transformations and jobs in Kettle?<\/span>Read more<\/a><\/p>\n","protected":false},"author":39,"featured_media":514,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[477],"class_list":["post-514","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-industry","tag-kettle-42e4-5ed9d5"],"_links":{"self":[{"href":"http:\/\/www.lahomiliadeldomingo.com\/blog\/wp-json\/wp\/v2\/posts\/514","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.lahomiliadeldomingo.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.lahomiliadeldomingo.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.lahomiliadeldomingo.com\/blog\/wp-json\/wp\/v2\/users\/39"}],"replies":[{"embeddable":true,"href":"http:\/\/www.lahomiliadeldomingo.com\/blog\/wp-json\/wp\/v2\/comments?post=514"}],"version-history":[{"count":0,"href":"http:\/\/www.lahomiliadeldomingo.com\/blog\/wp-json\/wp\/v2\/posts\/514\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/www.lahomiliadeldomingo.com\/blog\/wp-json\/wp\/v2\/posts\/514"}],"wp:attachment":[{"href":"http:\/\/www.lahomiliadeldomingo.com\/blog\/wp-json\/wp\/v2\/media?parent=514"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.lahomiliadeldomingo.com\/blog\/wp-json\/wp\/v2\/categories?post=514"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.lahomiliadeldomingo.com\/blog\/wp-json\/wp\/v2\/tags?post=514"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}