configuring-imports — Skillopedia

<!-- TIER:1 -- Configuring Imports An import is the data destination in a Celigo integration. It takes records from an upstream step and writes them to an external system -- REST APIs, databases, ERPs, file servers, or AI models. Every import has one adaptor type and, apart from the connection-less kinds in Minimum Required Fields, is bound to exactly one connection. Imports handle six concerns: - Field mapping -- transforming source fields into the destination system's expected format (including value resolution via static maps and lookup tables). Uses Mapper 2.0 ( array) by default, includi…

— the record root. An object record is addressed as\n '$.field'; a grouped (array) record as '$[0].field' / '$[*].field'\n - Used for precisely targeting data in structured JSON objects\n - Examples: '$.customer.firstName', '$.items[0].price', '$.addresses[*].street',\n '$[*].Tax' (every row of a grouped record)\n - Wildcards like [*] extract multiple values/objects\n - A key that contains a dot needs bracket notation: `$['P.O. Number']` (or\n `$[\"P.O. Number\"]`, `$.[P.O. Number]`). `$.P.O. Number` is read as a nested\n path: it maps nothing and the record still succeeds. Spaces, hyphens and `#`\n need no brackets (`$.Ship To Name`, `$.Tracking #`)\n - Filter expressions such as `$.items[?(@.active)]` fail the record with\n \"Eval [?(expr)] prevented in JSONPath expression.\"\n\n ```json\n \"extract\": \"$.customer.addresses[*]\" // Extracts all addresses\n ```\n\n **2. Handlebars Template Syntax**\n - Contains '{{' and '}}' pattern\n - Evaluated by the AFE 2.0 handlebars template engine\n - Can include logic, formatting, and computation\n - Access input record fields with {{record.fieldName}} notation; a grouped\n (array) record binds as `rows` instead — {{rows.0.fieldName}} / {{#each rows}}\n - Examples: \"{{record.firstName}} {{record.lastName}}\", \"{{#if record.isActive}}Active{{else}}Inactive{{/if}}\"\n - Valid on scalars and on `object` rows, where the rendered output must\n parse into an object. On an array row the template goes in a\n `buildArrayHelper` entry instead: on an `objectarray` row's own\n `extract` it is refused with 422 (\"buildArrayHelper\" or \"hardCodedValue\"\n must be present for object array datatype.), and on a primitive-array\n row's own `extract` it yields nothing (the field is left out) without an\n error\n\n ```json\n \"extract\": \"{{record.price}} {{record.currency}}\" // Combines two fields\n ```\n\n **3. Hard-Coded Value (literal string)**\n - Does NOT start with '

Important: agents should read /llm.txt, /llms.txt, or /.well-known/skills.json to discover the public Skillopedia API.

\n - Does NOT contain handlebars '{{' syntax\n - System treats it as a literal string value\n - VERY COMMON for setting static/constant values\n - Examples: \"Active\", \"USD\", \"Completed\", \"true\"\n\n ```json\n \"extract\": \"primary\" // Sets field value to the literal string \"primary\"\n \"extract\": \"true\" // Sets field value to the literal string \"true\"\n \"extract\": \"N/A\" // Sets field value to the literal string \"N/A\"\n ```\n\n This third pattern is the simplest and most efficient way to set hard-coded values in your mappings.\n AI agents should use this pattern whenever a field needs a static value that doesn't come from\n the input record or require computation.\n\n **Important implementation details**\n\n - JSON path patterns ALWAYS execute from the TOP-LEVEL root of the input record\n - The system maintains this context even in deeply nested mappings\n - For object mappings without child mappings, extract should return a complete object\n - An `object` row takes `extract` (to copy a whole object) or child `mappings`, not\n both: a row with both fails with 422 (`\"extract\" should not be present if\n \"mappings\" field is present`). A `buildArrayHelper` entry is different: there\n `extract` and `mappings` work together.\n - The root is the record as it arrives: an object record roots at `$.field`; a\n grouped record (an array of rows — grouped exports, file key columns, NetSuite\n grouped saved searches) roots at the array, so its rows are addressed as\n `$[0].field` (one row) or `$[*].field` (every row). A root that contradicts the\n record's shape resolves to nothing without an error.\n\n For most simple field-to-field mappings, prefer JSON path syntax for its clarity and performance.\n For hard-coded values, simply use the literal string as the extract value.\n x-celigo-ai-guidance:\n - |-\n ## GROUPED (ARRAY) RECORDS\n Decide the JSONPath root from the record's shape before writing\n any extract. When the record is an array of rows, `$.field`,\n `$.properties.field` and `$.rows[*].field` all resolve to\n nothing — the processor still reports success and the request\n body ships as `{}`. Address rows as `$[0].field` (that row's\n value, typed) or `$[*].field` (every row: joined with commas\n into a `string` destination; rejected with\n `incompatible_destination_conversion` into a `number`\n destination). Column labels with spaces work as\n `$[*][Order Number]`, `$[*].[Order Number]`, or\n `$[*]['Order Number']`. For one destination element per row\n the two array kinds take different helper forms: a primitive\n array (`stringarray` / `numberarray`) uses `buildArrayHelper:\n [{\"extract\": \"$[*].field\"}]`; an `objectarray` uses\n `buildArrayHelper: [{\"extract\": \"$[*]\", \"mappings\": [...]}]`\n with the children addressing the row relatively (`$.field`,\n `{{rows.field}}`). An `extract` placed directly on the\n array-typed field yields nothing. Handlebars\n extracts bind the array record as `rows` (`{{rows.0.field}}`),\n an object record as `record`. The reverse holds too: `$[*]` on\n an object record iterates the object's values, not rows.\n - |-\n ## SIZE LIMIT\n The platform's import save validator caps this field at **1024\n characters**. Saves with a longer `extract` are rejected\n with `[size_limit_exceeded] The maximum size of the field:\n mappings.\u003cindex>.extract in the Import document should be\n 1024 characters`. This typically only matters for\n Handlebars templates that build complex strings inline -- if\n a template is approaching the cap, prefer one of:\n\n - **Decompose into nested mappings** (set `dataType:\n \"object\"` and use child `mappings` -- each child gets\n its own 1024-char budget).\n - **Move the logic to a `preMap` hook** so the heavy\n transformation runs in script and the mapping just reads\n `record.\u003ccomputed>`.\n - **Use a lookup** (`lookupName` + a small `lookups` entry)\n for value translation that would otherwise need a long\n `{{#if ...}}{{else if ...}}` chain.\n examples: [\"$.fullName\", \"{{record.fullName}}\", \"primary\", \"$.lineItems[*].unitPrice\"]\n extractDateFormat:\n type: string\n description: |\n Specifies the format pattern of the input date string to ensure proper parsing.\n\n Used on string-typed mappings whose `extract` yields a date. Uses\n Moment.js-compatible formatting tokens to describe how the incoming date\n string is structured.\n examples: [\"YYYY-MM-DD\", \"MM/DD/YYYY\", \"DD-MMM-YYYY HH:mm:ss\"]\n extractDateTimezone:\n type: string\n description: |\n Specifies the timezone of the input date string using Olson/IANA timezone identifiers.\n\n Used on string-typed mappings whose `extract` yields a date; tells the system\n how to interpret timestamp values from the input system.\n examples: [\"UTC\", \"America/New_York\", \"Europe/London\"]\n generateDateFormat:\n type: string\n description: |\n Specifies the output format pattern when generating a date string or converting\n from a Date type to String type.\n\n Uses Moment.js-compatible formatting tokens to define the structure of the resulting\n date string.\n examples: [\"YYYY-MM-DD\", \"MM/DD/YYYY\", \"DD-MMM-YYYY HH:mm:ss\"]\n generateDateTimezone:\n type: string\n description: |\n Specifies the timezone to apply when generating or converting timestamp values\n using Olson/IANA timezone identifiers.\n\n Controls timezone conversion when producing date output.\n examples: [\"UTC\", \"America/New_York\", \"Europe/London\"]\n default:\n type: [string, 'null']\n description: |\n Specifies a fallback value to use when extract returns empty/null or when conditional\n logic fails and no other mapping supplies a value.\n\n Explicit JSON `null` is itself a valid fallback: the destination field is written as\n JSON null when the extract yields nothing (the Mapper UI's \"Use null as default value\"\n action). Omitting the key entirely means no fallback — the field is left out of the\n output when the extract is missing or an empty string (the UI's \"Do nothing\" action),\n while a null source is still written as null unless `conditional.when` is\n `extract_not_empty`. Failing the record instead is `conditional.when:\n fail_record_when_empty`, which excludes a default.\n x-celigo-ai-guidance:\n - |-\n `\"default\": null` and `\"default\": \"\"` are distinct saved values, not\n absent keys: null renders the destination field as JSON null and the\n empty string renders it as `\"\"` when the extract yields nothing.\n Preserve them verbatim when editing or migrating a mapping — dropping\n the key changes runtime behavior from \"use null/empty string as\n default value\" to \"do nothing\" (a missing or `\"\"` source is left out\n of the output, while a null source is written as null unless\n `conditional.when` is `extract_not_empty`).\n - |-\n The four \"Action to take if source field has no value\" choices map to:\n do nothing → no `default` key (add `conditional.when:\n extract_not_empty` to leave a null source out too; without it a null\n source is written as null); use null → `\"default\": null`; use empty string or\n a custom value → `\"default\": \"\u003cvalue>\"`; cause the record to fail →\n `conditional.when: fail_record_when_empty` with no `default` key.\n examples: [\"N/A\", \"0\", \"false\"]\n lookupName:\n type: string\n description: |\n **Purpose**\n References a lookup table for transforming values during the mapping process.\n\n **Usage**\n\n The lookupName refers to a named lookup defined in the lookups array of the same resource.\n On an HTTP import that is `http.lookups`: a `lookups` array sent only at the top level\n is dropped on save. On an import, a name that matches no lookup fails every record\n with \"Unable to find lookup in mapping configs - \u003cname>\", and a key the lookup\n doesn't hold follows the lookup's `allowFailures`: false fails the record; true, or\n the field left out, lets the record succeed with the lookup's `default`, or without\n the mapped field when there is none.\n\n ```json\n {\n \"generate\": \"countryName\",\n \"dataType\": \"string\",\n \"extract\": \"$.countryCode\",\n \"lookupName\": \"countryCodeToName\"\n }\n ```\n\n During processing, the system:\n 1. Extracts the value from the input record (e.g., \"US\")\n 2. Finds the lookup table with the specified name\n 3. Uses the extracted value as a key in the lookup\n 4. Returns the corresponding value (e.g., \"United States\")\n\n **Benefits**\n\n - **Standardization**: Ensures consistent value translation across mappings\n - **Centralization**: Define translations once and reference them in multiple places\n - **Maintainability**: Update all mappings by changing the lookup definition\n - **Readability**: Makes mappings more descriptive and self-documenting\n\n The specific lookup capabilities depend on the context where mappings are used.\n examples: [\"countryCodeToName\", \"statusMapping\", \"departmentCodes\"]\n hardCodedValue:\n type: string\n description: |\n Constant written to the destination instead of an extracted value. On a primitive\n array (`stringarray`, `numberarray`, `booleanarray`) the string is split on commas,\n verbatim and untrimmed, into elements of the array's type (`\"A,B\"` gives `[\"A\", \"B\"]`).\n On an `object` it must be a JSON object string and on an `objectarray` a JSON array\n string (an object string becomes one element); the save parses it and refuses non-JSON\n or the wrong shape with 422, and an array row with a constant needs no `buildArrayHelper`.\n examples: [\"USD\", \"A,B\", \"[{\\\"code\\\": \\\"A\\\"}, {\\\"code\\\": \\\"B\\\"}]\"]\n description:\n type: string\n description: |\n Optional free-text annotation that appears in the Mapper sidebar to provide context about\n the mapping's purpose for collaboration and documentation.\n\n Has no functional impact on the mapping behavior.\n examples: [\"Maps customer name\", \"Converts price to target currency\", \"Formats date for target system\"]\n sourceDataType:\n type: string\n description: |\n Declares the JSON type of the value produced by `extract`, before any\n conversion to `dataType`. Same enum as `dataType`. Set on leaf mappings\n only — parent mappings (with child `mappings` or `buildArrayHelper`)\n have no extracted value of their own; the children carry their own\n `sourceDataType`.\n\n For date fields use `string` (JSON represents dates as strings); the\n parsing/formatting lives in `extractDateFormat` / `generateDateFormat` /\n `extractDateTimezone` / `generateDateTimezone`.\n enum:\n - string\n - number\n - boolean\n - object\n - stringarray\n - numberarray\n - booleanarray\n - objectarray\n - arrayarray\n x-enumDescriptions:\n string: Source value is a text string.\n number: Source value is numeric.\n boolean: Source value is true or false.\n object: Source value is a JSON object.\n stringarray: Source value is an array of strings.\n numberarray: Source value is an array of numbers.\n booleanarray: Source value is an array of booleans.\n objectarray: Source value is an array of objects.\n arrayarray: Source value is an array of arrays.\n examples: [\"string\", \"number\", \"boolean\", \"object\"]\n mappings:\n type: array\n description: |\n **Purpose**\n Enables recursive definition of nested object structures through child mapping objects.\n\n **Guidance**\n This is the KEY FIELD that implements the recursive pattern at the core of this schema:\n\n **When to Use**\n - REQUIRED when dataType = \"object\" (unless you are copying an entire object from the input record)\n - REQUIRED in buildArrayHelper.mappings when defining complex object array elements\n - NEVER used with simple types (string, number, boolean)\n\n **Behavior**\n - Each mapping in this array becomes a property of the parent object\n - The full Mappings schema is repeated recursively at each level\n - Can be nested to any depth for complex hierarchical structures\n\n **Context Handling**\n - Each level of nesting changes the mapping CONTEXT for 'generate'\n - The extraction CONTEXT always remains the original input record\n - This means child mappings can pull data from anywhere in the input record\n\n **Common Patterns**\n\n **Nested Objects**\n ```json\n {\n \"generate\": \"customer\",\n \"dataType\": \"object\",\n \"mappings\": [\n {\n \"generate\": \"contact\",\n \"dataType\": \"object\",\n \"mappings\": [\n {\"generate\": \"email\", \"dataType\": \"string\", \"extract\": \"$.customerEmail\"}\n ]\n }\n ]\n }\n ```\n\n **Multiple Fields in Object**\n ```json\n {\n \"generate\": \"address\",\n \"dataType\": \"object\",\n \"mappings\": [\n {\"generate\": \"street\", \"dataType\": \"string\", \"extract\": \"$.address.line1\"},\n {\"generate\": \"city\", \"dataType\": \"string\", \"extract\": \"$.address.city\"},\n {\"generate\": \"country\", \"dataType\": \"string\", \"extract\": \"$.address.country\"}\n ]\n }\n ```\n\n IMPORTANT: When analyzing or generating mappings, ALWAYS check if parent.dataType = \"object\"\n or if you're inside buildArrayHelper.mappings for objectarray elements. These are the only\n valid contexts for the mappings array.\n items:\n $ref: '#/Mappings/items'\n examples: [[{\"generate\": \"firstName\", \"extract\": \"$.name.first\", \"dataType\": \"string\", \"status\": \"Active\"}, {\"generate\": \"lastName\", \"extract\": \"$.name.last\", \"dataType\": \"string\", \"status\": \"Active\"}]]\n buildArrayHelper:\n type: array\n x-celigo-ai-guidance:\n - |-\n Entry-level `extract` accepts an iterating array path\n (`$.items[*]`), `\"$\"` (the record root), or omission — a\n constructed entry (child `mappings` with the extract omitted or\n `\"$\"`; the two spellings are equivalent) contributes exactly one\n element, and entries append in order. On a grouped (array)\n record the record itself is the array to iterate: `\"$[*]\"`\n yields one `objectarray` element per row (children address the\n row as `$.field`), and `\"$[*].field\"` yields one primitive-array\n value per row. Indexed or comma-joined\n entry extracts (`\"$[0]\"`, `\"$[0],$[0]\"`) are not valid: the\n processor renders the entire destination array as null with no\n validation error. To emit a fixed number of static elements, add\n that many sibling constructed entries — never index the extract.\n - |-\n An entry's `extract` can be a Handlebars template. On an\n `objectarray` it must render a JSON object (one element) or a JSON\n array (one element per item). On a primitive array each entry\n renders one element, so a serialized array becomes one string\n element. A bare literal entry extract is refused with 422 (\"extract\n only supports json path or handlebars expression\"); set a constant\n with the row's `hardCodedValue` instead.\n description: |\n **Purpose**\n Configures how to construct arrays in the output record, handling various array types and inputs.\n\n **Guidance**\n This is the mechanism for building array data types:\n\n **When to Use**\n - REQUIRED when dataType ends with \"array\" (stringarray, objectarray, etc.), unless the\n row sets `hardCodedValue`: an array row's own `extract` is never read (see `dataType`)\n - Each entry in this array contributes elements to the output array\n - Multiple entries allow combining data from different input arrays\n\n **Array Type Handling**\n\n **For Simple Arrays (stringarray, numberarray, booleanarray)**\n - Only the 'extract' field is used to pull values\n - JSON path with wildcards (e.g., $.items[*].name) returns multiple values;\n on a grouped (array) record the record itself is the array, so the\n path roots at it ($[*].name — one value per row)\n - Each result is converted to the appropriate primitive type\n ```json\n {\n \"generate\": \"productNames\",\n \"dataType\": \"stringarray\",\n \"buildArrayHelper\": [\n {\"extract\": \"$.products[*].name\"}\n ]\n }\n ```\n\n **For Object Arrays (objectarray) - three patterns**\n\n 1. Extract Only (existing objects):\n ```json\n {\n \"generate\": \"contacts\",\n \"dataType\": \"objectarray\",\n \"buildArrayHelper\": [\n {\"extract\": \"$.account.primaryContacts[*]\"}, // Pull primary contact objects\n {\"extract\": \"$.account.secondaryContacts[*]\"}, // Pull secondary contact objects\n {\"extract\": \"$.vendor.contactPersons[*]\"}, // Pull vendor contact objects\n {\"extract\": \"$.subsidiaries[*].mainContact\"} // Pull main contact from each subsidiary\n ]\n }\n ```\n\n 2. Mappings Only (constructed object):\n ```json\n {\n \"generate\": \"contactInfo\",\n \"dataType\": \"objectarray\",\n \"buildArrayHelper\": [\n {\n \"mappings\": [ // Creates one object in the array\n {\"generate\": \"type\", \"dataType\": \"string\", \"extract\": \"primary\"},\n {\"generate\": \"email\", \"dataType\": \"string\", \"extract\": \"$.primaryEmail\"}\n ]\n },\n {\n \"mappings\": [ // Creates another object in the array\n {\"generate\": \"type\", \"dataType\": \"string\", \"extract\": \"secondary\"},\n {\"generate\": \"email\", \"dataType\": \"string\", \"extract\": \"$.secondaryEmail\"}\n ]\n }\n ]\n }\n ```\n Each constructed entry contributes exactly one element, in entry\n order; writing the entry with `\"extract\": \"$\"` is equivalent to\n omitting the extract (both anchor the element at the record root).\n A fixed number of static elements is built from that many sibling\n constructed entries.\n\n 3. Extract AND Mappings (transform input arrays):\n ```json\n {\n \"generate\": \"lineItems\",\n \"dataType\": \"objectarray\",\n \"buildArrayHelper\": [\n {\n \"extract\": \"$.order.items[*]\", // For each item in the array\n \"mappings\": [ // Transform to this structure using the composite object\n {\"generate\": \"sku\", \"dataType\": \"string\", \"extract\": \"$.order.items.productId\"}, // Notice: items is singular\n {\"generate\": \"quantity\", \"dataType\": \"number\", \"extract\": \"$.order.items.qty\"}, // Notice: items is singular\n {\"generate\": \"orderNumber\", \"dataType\": \"string\", \"extract\": \"$.order.id\"}, // Access parent data\n {\"generate\": \"customerName\", \"dataType\": \"string\", \"extract\": \"$.customerName\"} // Access root data\n ]\n }\n ]\n }\n ```\n\n **Arrays of Arrays**\n - The mapper has no working type for them: `arrayarray` saves, but every record\n fails with `Destination datatype is a required field.` Build an array of arrays\n in a script hook instead.\n\n **Important details**\n\n - When both extract and mappings are provided, the system creates special composite objects\n that maintain hierarchical context during processing\n - This enables accessing both the current array element AND its parent context\n - An ITERATING entry's extract MUST be a JSON path that iterates an array:\n '$.items[*]' when the array is a field of an object record, or '$[*]'\n (objectarray) / '$[*].field' (primitive arrays) when the record itself is a\n grouped array of rows. A CONSTRUCTED entry (child mappings building one\n element) instead omits extract or sets it to exactly \"$\" — never rewrite \"$\"\n into an indexed or comma-joined form ('$[0]', '$[0],$[0]'): those render the\n entire destination array as null without raising a validation error\n - Each array helper entry acts independently, potentially adding multiple elements\n\n The buildArrayHelper is the most complex part of the mappings system - always analyze the\n dataType first to understand which pattern is appropriate.\n items:\n type: object\n properties:\n extract:\n type: string\n description: |\n JSON path expression that identifies the input array or values to extract.\n\n For objectarray with mappings, this defines which input objects to iterate through.\n The JSON path must return either a single object or an array of objects.\n\n For a CONSTRUCTED entry (child mappings building exactly one element from\n record-root fields and constants), omit this field or set it to exactly \"$\" —\n the two spellings are equivalent. Indexed or comma-joined forms ('$[0]',\n '$[0],$[0]') are not valid at the entry level: the processor renders the\n entire destination array as null without raising a validation error.\n\n The system creates special composite objects during processing to maintain\n hierarchical relationships, allowing easy access to both the current array item\n and its parent contexts.\n\n An iterating entry keeps the `[*]`: `$.line_items` without it yields one empty\n element per item, with no error. Child extracts drop the `[*]` but keep the\n array's path (`$.line_items.sku`); `$.line_items[*].sku` maps nothing, and an\n element-relative `$.sku` reads the record root instead of the element.\n examples: [\"$.items[*]\", \"$.order.lineItems[*]\", \"$.parents[*].children[*]\", \"$\"]\n name:\n type: string\n maxLength: 200\n description: |\n Display label for this source object in the mapper, so the sources of\n an object array can be told apart instead of all showing as ` configuring-imports — Skillopedia .\n Has no effect at runtime; stored only when non-empty.\n x-celigo-ai-guidance:\n - |-\n Carry an existing `name` through every edit — it is the user's label\n for the tab, and an entry sent without it loses the label. Duplicate\n names across entries are allowed. Set one only when the user names\n the source object; never invent labels.\n examples: [\"Billing address\", \"Shipping address\"]\n sourceDataType:\n type: string\n description: |\n Declares the JSON type of the input array being iterated, to ensure\n proper type handling during array construction. Same enum as `dataType`.\n enum:\n - string\n - number\n - boolean\n - object\n - stringarray\n - numberarray\n - booleanarray\n - objectarray\n - arrayarray\n x-enumDescriptions:\n string: Input array contains string elements.\n number: Input array contains numeric elements.\n boolean: Input array contains boolean elements.\n object: Input array contains object elements.\n stringarray: Input array contains arrays of strings.\n numberarray: Input array contains arrays of numbers.\n booleanarray: Input array contains arrays of booleans.\n objectarray: Input array contains arrays of objects.\n arrayarray: Input array contains arrays of arrays.\n examples: [\"string\", \"number\", \"boolean\", \"object\"]\n default:\n type: [string, 'null']\n description: |\n Specifies a fallback value when the extracted array element is empty or\n not found in the input data.\n\n Explicit JSON `null` is itself a valid fallback (the element is written as\n JSON null); omitting the key means no fallback is applied.\n examples: [\"N/A\", \"0\", \"false\"]\n conditional:\n type: object\n description: |\n Defines conditional rules for including each array element in the result.\n x-celigo-ai-guidance:\n - |-\n Judged against this entry's own `extract` (the array source). The\n design scopes the UI's tab-level \"Action to take if source field has\n no value\" to copy-as-is tabs (no child `mappings`); the API accepts\n `fail_record_when_empty` on a mapped entry as well, and the mapper\n processor honors it there — the record fails when that array is\n missing or empty (`errors[]` entry `code: extract_empty`, `fatal:\n true`, message `\u003cgenerate>: Source field has no value`; probed\n 2026-10-09). Say so when writing it on a mapped entry: the user may\n not find the setting in the mapper's tab. To fail on an empty field\n inside each element, set the action on that child mapping instead.\n properties:\n when:\n type: string\n description: |\n Condition judged on the entry's extract — skip the elements the\n extract does not return, or fail the whole record when the source\n array is missing or empty.\n enum:\n - extract_not_empty\n - fail_record_when_empty\n x-enumDescriptions:\n extract_not_empty: Include an array element only when the extract returns a non-empty value for it\n fail_record_when_empty: Fail the record when the entry's source array is missing or empty\n examples: [\"extract_not_empty\"]\n mappings:\n type: array\n description: |\n Contains recursive mapping definitions for complex array element transformations.\n\n **Composite object mechanism**\n\n When both 'extract' and 'mappings' are used together, the system implements a sophisticated\n \"composite object\" approach that is crucial for AI agents to understand:\n\n 1. The system starts with the complete input record\n\n 2. For each array element matched by the extract path, it creates a modified version of\n the input record where:\n - Array paths in the extract JSON path are REPLACED with single objects\n - Each array ([]) in the path is converted to a single object ({})\n - This preserves the hierarchical relationship between nested arrays\n\n **Example**\n\n Given an input record:\n ```json\n {\n \"customer\": {\n \"name\": \"John Doe\",\n \"orders\": [\n {\n \"id\": \"O-001\",\n \"items\": [\n {\"sku\": \"ABC\", \"qty\": 2},\n {\"sku\": \"XYZ\", \"qty\": 1}\n ]\n },\n {\n \"id\": \"O-002\",\n \"items\": [\n {\"sku\": \"DEF\", \"qty\": 3}\n ]\n }\n ]\n }\n }\n ```\n\n With extract path: `$.customer.orders[*].items[*]`\n\n For each item, the system creates a composite object like:\n ```json\n {\n \"customer\": {\n \"name\": \"John Doe\",\n \"orders\": { // Note: Array replaced with single object\n \"id\": \"O-001\",\n \"items\": { // Note: Array replaced with single object\n \"sku\": \"ABC\",\n \"qty\": 2\n }\n }\n }\n }\n ```\n\n Then in your mappings, you can access:\n - The current item: `$.customer.orders.items.sku`\n - The parent order: `$.customer.orders.id`\n - Top-level data: `$.customer.name`\n\n This approach allows for precise mapping from deeply nested structures while maintaining\n access to all contextual parent data, without requiring complex array index management.\n\n **Implementation guidance**\n\n When implementing the composite object mechanism:\n\n 1. Analyze the extract path to identify all array patterns (`[*]` or `[number]`)\n 2. For each array in the path, understand that it will be replaced with a single object\n 3. In the mappings, use paths that reference these arrays as if they were objects\n 4. Remember that every mapping still has access to the full input record context\n 5. This mechanism is especially powerful when mapping hierarchical data like:\n - Order → Line Items → Taxes/Discounts\n - Customer → Addresses → Address Lines\n - Invoice → Line Items → Serial Numbers\n\n The extract path effectively tells the system \"iterate through these arrays\",\n while the composite object mechanism ensures you can still access both the\n current array item AND its parent context during mapping.\n items:\n $ref: '#/Mappings/items'\n examples: [[{\"generate\": \"itemName\", \"extract\": \"$.name\", \"dataType\": \"string\", \"status\": \"Active\"}, {\"generate\": \"itemPrice\", \"extract\": \"$.price\", \"dataType\": \"number\", \"status\": \"Active\"}]]\n # The save rejects the fail-record action paired with a default on an\n # entry ([unsupported_mapping]); the entry schema has no lookupName or\n # hardCodedValue, so this is the only pair at this level.\n if:\n properties:\n conditional:\n properties:\n when:\n const: fail_record_when_empty\n required: [when]\n required: [conditional]\n then:\n not:\n required: [default]\n status:\n type: string\n description: |\n **Purpose**\n Controls whether the mapping is applied. Send it on every mapping entry.\n\n **Guidance**\n Only `Active` rows run. `Draft` and `Inactive` rows keep their\n configuration, are saved with minimal validation, and are skipped at\n runtime and in preview; a non-Active parent skips its whole subtree\n whatever its children say. Imports (on save and in the mapper processor)\n and API response mappings reject a row missing this field (validation\n error: \"Mapping object must have status field present.\"); a Tool's\n output mapping saves and runs one, which is why `required` doesn't\n list it.\n x-celigo-ai-guidance:\n - |-\n Emit `\"Active\"` for mappings that should run; `\"Draft\"` is also\n accepted and saves the mapping while bypassing the Active-only\n field validations. The value is case-sensitive. The API rejects:\n - Missing field: `[status_field_absent] Mapping object must have status field present.`\n - Wrong case (e.g. `\"active\"`): `[invalid_status_enums] active is not a valid enum for status field.`\n - |-\n `\"Inactive\"` is the user's \"Set inactive\" on a row: use it only when\n asked to disable or park a row, never to mean incomplete (that is\n `\"Draft\"`). Setting a parent `Inactive` means no `Active` row may\n remain under it — the save rejects an `Active` row under an\n `Inactive` parent (`[inactive_parent_active_child]` on `mappings`,\n no message text) and never cascades or promotes on its own; `Draft`\n descendants are kept as they are, and a `Draft` parent does not gate\n its children. The UI's Set inactive on a parent marks the parent and\n every child `Inactive`. Re-enabling a child\n means re-enabling its ancestors in the same payload. Edits that keep\n existing rows must carry their stored status forward — rewriting a\n `Draft` or `Inactive` row to `Active` turns a row the user parked\n back on.\n enum:\n - Active\n - Draft\n - Inactive\n x-enumDescriptions:\n Active: The mapping is enabled and applied during processing\n Draft: In-progress mapping; saved without the Active-only field validations and skipped at runtime\n Inactive: Disabled by the user; configuration kept, skipped at runtime and in preview\n examples: [\"Active\"]\n conditional:\n type: object\n description: |\n **Purpose**\n Defines conditional processing rules for the entire mapping.\n\n **Guidance**\n These conditions determine whether the mapping is applied based on record\n state or field content:\n\n **When to Use**\n - When a mapping should only be applied in specific circumstances\n - To implement conditional logic without using complex handlebars expressions\n - For creating mappings that only run during create or update operations\n\n **Available Conditions**\n\n - `record_created`: Apply only when creating a new record\n Useful for setting initial values that should not be overwritten during updates\n\n - `record_updated`: Apply only when updating an existing record\n Useful for transformation logic that should only run during updates\n\n - `extract_not_empty`: Apply only when the extract field returns a value\n Useful for conditional mapping based on input data availability\n\n - `fail_record_when_empty`: Fail the record when the extract field returns\n no value (the UI's \"Cause record to fail\" action when the source field\n has no value)\n\n **Example**\n ```json\n {\n \"generate\": \"statusMessage\",\n \"dataType\": \"string\",\n \"status\": \"Active\",\n \"extract\": \"$.status.message\",\n \"conditional\": {\n \"when\": \"extract_not_empty\" // Only map when status.message exists\n }\n }\n ```\n x-celigo-ai-guidance:\n - |-\n `when` holds one value, and two different settings share it: the\n composite-import operation gate (`record_created` / `record_updated`,\n the UI's \"Only perform mapping when\") and the empty-source action\n (`extract_not_empty` / `fail_record_when_empty`, the UI's \"Action to\n take if source field has no value\"). A row cannot carry both — writing\n one replaces the other and the save succeeds, so a composite import\n row that needs both has to choose. Never author `fail_record_when_empty`\n on a row that already carries `record_created` or `record_updated`.\n properties:\n when:\n type: string\n description: |\n Condition that gates this mapping — either an operation gate on a\n composite (create-or-update) import or the action to take when the\n extract returns no value. Empty means null, undefined or an empty\n string; `0`, `false` and whitespace are values.\n x-celigo-ai-guidance:\n - |-\n `fail_record_when_empty` is the fourth \"Action to take if source field\n has no value\" and is mutually exclusive with the other three: send it\n with no `default` key. A `default`, a `lookupName` or a `hardCodedValue`\n on the same row — or a `default` on a `buildArrayHelper` entry — is\n rejected at save with `[unsupported_mapping] \"\u003ckey>\" and conditional.when\n \"fail_record_when_empty\" cannot be present in mapping simultaneously.`\n (the lookup variant adds `Use lookup \"allowFailures\" instead.`);\n lookups fail through `allowFailures: false`, and a hard-coded row\n always has a value. At runtime an empty extract fails the record with\n an `errors[]` entry `{code: \"extract_empty\", fatal: true, source:\n \"mapping\", message: \"\u003cgenerate>: Source field has no value\"}`\n (\"Input field has no value\" in a Transformation 2.0); `0` and `false`\n are values and map through.\n enum:\n - record_created\n - record_updated\n - extract_not_empty\n - fail_record_when_empty\n x-enumDescriptions:\n record_created: Apply this mapping only when creating a new record\n record_updated: Apply this mapping only when updating an existing record\n extract_not_empty: Apply this mapping only when the extract field returns a non-empty value\n fail_record_when_empty: Fail the record when the extract field returns no value\n examples: [\"record_created\", \"record_updated\", \"extract_not_empty\", \"fail_record_when_empty\"]\n required:\n - dataType\n if:\n properties:\n conditional:\n properties:\n when:\n const: fail_record_when_empty\n required: [when]\n required: [conditional]\n then:\n not:\n anyOf:\n - required: [default]\n - required: [lookupName]\n - required: [hardCodedValue]\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":52991,"content_sha256":"52bc801b81ac5f934dda6ea0e194361e1aad6592840f911cf948db2fbb4ca41d"},{"filename":"references/common-schemas/one-to-many.yml","content":"OneToMany:\n type: boolean\n description: |\n When true, the step runs once per child record instead of once per incoming record.\n `pathToMany` names the array field that holds the children inside an object record; when\n the incoming record is itself an array (grouped or row-based data), leave `pathToMany` blank\n and each element becomes a record. The fan-out is scoped to this step: afterwards the children\n are re-joined into the record's original shape — the object with its array, or the array of rows —\n carrying any response-mapping enrichment, and that re-joined record is what the next step\n receives. Not for locating records in an export's HTTP response; use\n `http.response.resourcePath` for that. Applies to steps that receive a record — imports and\n lookups (an export referenced as a page processor of a flow, API, or Tool). A standalone export\n (a flow's page generator) has no incoming record, so the fields are saved but have no effect on\n the records it produces; to emit one record per array element from a standalone export, use a\n `hooks.preSavePage` script, or for a file-definition export make the repeating segment the\n record boundary in the definition's rules.\n x-celigo-ai-guidance:\n - |-\n Only imports and lookups (exports placed as page processors) fan out. On a standalone export\n (a flow's page generator) the API saves ``oneToMany`` / ``pathToMany`` and applies neither:\n the records reach the next step unchanged and no error is raised. Live-verified 2026-09-18 on\n a flow test run — an order record with three ``lines`` reached the import as ONE record with\n the setting on the export, and as THREE child records with the same setting on the import.\n Never propose these fields as the way to split a standalone export's records; that is\n ``hooks.preSavePage`` work (or, for a file definition, the definition's record boundary).\n The rules below apply only on a step that receives a record.\n - |-\n Two modes, decided by the SHAPE of the record entering the step. Object record with a child\n array under a field → set ``pathToMany`` to that field. Array record (a ``groupByFields``\n export, a CSV/file group, a script that returns an array) → leave ``pathToMany`` blank; each\n element becomes one record. Live-verified 2026-09-10 on flows, Tools, and APIs.\n - |-\n Blank ``pathToMany`` on an OBJECT record fails every record with code ``oneToMany_error``:\n \"Failed to extract child records from oneToMany import data. Record at index 0 was not an\n array, and no PathToMany was provided\". Builder previews return only the code, with no\n message. The fix is never \"turn oneToMany off\" by default — first determine whether the\n record arriving at the step is an object (set a path) or an array (blank is correct; a\n preview error on an array-fed step is the preview panel re-applying one-to-many to the\n already-fanned-out record — verify with a test run or a real run).\n - |-\n Child record shape differs by mode. Path mode: each child is the array element plus\n ``_PARENT`` (the parent record minus the array). Blank/row mode: each child is the element\n itself, with no ``_PARENT``. In both modes templates on the step reference the child as\n ``{{record.\u003cfield>}}``; a response mapping on a one-to-many lookup merges into each child and\n the downstream step receives the re-joined parent (path mode) or the re-joined array (row\n mode) with every element enriched.\n - |-\n One-to-many never batches by itself: ``batch_of_records`` appears as the HTTP body template\n root only when ``http.batchSize`` is greater than 1 (children are batched after fan-out).\n With the default batch size the template root is ``record``.\n default: false\n examples: [true, false]\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":3900,"content_sha256":"9c57c8003be6b14dd6d1f0f1320ddf2a1cf1bc95f496b5cb968d599408650b15"},{"filename":"references/common-schemas/path-to-many.yml","content":"PathToMany:\n type: string\n description: |\n Path to the array of child records inside an object record when `oneToMany` is true, in dot\n notation (`items`, `lines.lineItems`). Leave blank when the incoming record is itself an array\n (grouped or row-based data) — each element is then a child record. A path that does not resolve\n to an array processes zero records and reports success. Read only on steps that receive a\n record (imports, lookup exports); on a standalone export it is saved and ignored — see `oneToMany`.\n x-celigo-ai-guidance:\n - |-\n Inert on a standalone export (a flow's page generator) exactly like ``oneToMany`` — the value\n is saved, the export's records reach the next step unchanged, and nothing errors\n (live-verified 2026-09-18). Splitting a standalone export's records into one per element is\n ``hooks.preSavePage`` work, or the file definition's record boundary; never this field.\n - |-\n Set this ONLY when the record entering the step is an object whose children sit under a field.\n On an array record (a ``groupByFields`` export, a CSV/file group, a script returning an array)\n any path resolves to nothing: the step processes ZERO children, builds no request, and reports\n success with no error. Live-verified 2026-09-10 — the failure is silent, so confirm the arriving\n record's shape (test run stage data, or the flow builder's input panel: ``rows`` means array).\n - |-\n Blank with an object record is the ``oneToMany_error`` failure (\"Record at index 0 was not an\n array, and no PathToMany was provided\"); blank with an array record is the correct row-based\n configuration. An empty string behaves exactly like an absent field.\n - |-\n Use the bare field name or dotted path (``lineItems``, ``details.items``), not a JSONPath\n expression. Each child in path mode carries ``_PARENT`` (the parent minus the array), so\n parent-level fields are read as ``{{record._PARENT.\u003cfield>}}``.\n examples: [\"items\", \"line_items\", \"orderLines\", \"lines.lineItems\"]\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":2095,"content_sha256":"d2e78377a11ab6f901a8834c658d80da27216888de4fe915e592546b2737b938"},{"filename":"references/common-schemas/pre-save.yml","content":"PreSave:\n type: object\n description: |\n Defines a JavaScript hook that executes before the resource is saved.\n\n This hook allows for programmatic validation, transformation, or enrichment of the\n resource itself before it is persisted. It can be used to enforce business rules,\n set derived properties, or implement cross-field validations that can't be expressed\n through the standard UI.\n\n **Function signature**\n\n The preSave function is invoked with a single 'options' argument containing:\n ```javascript\n function preSave(options) {\n // Process options and return the modified resource\n return options.newResource;\n }\n ```\n\n **Available context**\n\n The 'options' argument provides access to:\n - `options.newResource` - The resource being saved (with pending changes)\n - `options.oldResource` - The previous version of the resource (before changes)\n - `options.sandbox` - Boolean flag indicating whether the script is running in a sandbox environment\n\n\n **Common uses**\n\n - Enforcing complex business rules across multiple fields\n - Automatically deriving field values based on other configuration\n - Performing validation that depends on external systems or data\n - Normalizing or standardizing configuration values\n - Adding computed or derived properties\n - Implementing versioning or change tracking\n - Dynamically looking up data using the Celigo API module to enrich configuration\n\n **Return value**\n\n The function must return the newResource object (potentially modified) to be saved.\n Throwing an exception will prevent saving and signal an error to the user.\n properties:\n function:\n type: string\n description: |\n The name of the function to execute within the referenced script.\n\n This property specifies which function to invoke from the script\n referenced by _scriptId. The function will be called just before\n the resource is saved.\n\n The function must follow the expected signature and return the resource object.\n examples: [\"preSave\", \"validateResource\", \"prepareResourceForSave\"]\n _scriptId:\n type: string\n format: objectId\n x-celigo-refModel: scripts\n description: |\n Reference to a predefined script resource containing the preSave function.\n\n The referenced script should contain the function specified in the\n 'function' property. This script must be accessible within the user's account\n and have appropriate permissions.\n examples: [\"60a2c4e6f321d800129a1a3c\", \"5f8e7d2c1b54e6003a9c7e12\"]","content_type":"application/yaml; charset=utf-8","language":"yaml","size":2618,"content_sha256":"60e1b1de4be3c2404f72c6701b986109a3c3aaabf69b27dba14820cf519a361f"},{"filename":"references/common-schemas/resource-response.yml","content":"ResourceResponse:\n type: object\n description: Response\n x-celigo-ai-guidance:\n - Core response fields shared by all Celigo resources\n properties:\n _id:\n type: string\n format: objectId\n readOnly: true\n description: Unique identifier for the resource. Format is a 24-character hexadecimal string.\n x-celigo-ai-guidance:\n - |-\n The _id is used in:\n - API endpoints that operate on a specific resource (e.g., GET, PUT, DELETE)\n - References from other resources (e.g., flows that use this resource)\n - Job history and error tracking\n examples: [\"5f8d43a1b9e5a80011a35f2c\"]\n createdAt:\n type: string\n format: date-time\n readOnly: true\n description: Timestamp when the resource was created. Set automatically and cannot be modified.\n x-celigo-ai-guidance:\n - Recorded in ISO 8601 with a UTC timezone (Z suffix).\n examples: [\"2023-04-01T09:15:32Z\"]\n lastModified:\n type: string\n format: date-time\n readOnly: true\n description: Timestamp when the resource was last updated. Changes whenever any property is modified.\n x-celigo-ai-guidance:\n - Recorded in ISO 8601 with a UTC timezone (Z suffix); always equal to or later than `createdAt`.\n - |-\n Records that something was written, not what — a settings edit, a flag flip, or a\n platform workflow all move it. What changed and when is in the audit log: the\n per-resource route `GET /v1/{resourceType}/{_id}/audit` for the types its\n `resourceType` enum lists, or the account-wide\n `GET /v1/audit?resourceType=\u003csingular>&_resourceId={_id}` for the further types that\n endpoint's enum accepts (users, environments, notifications among them). A type absent\n from both enums — jobs, connectors, templates — has only this timestamp.\n examples: [\"2023-04-15T14:30:15Z\"]\n deletedAt:\n type: [string, 'null']\n format: date-time\n readOnly: true\n description: Timestamp when the resource was soft-deleted. When null or absent, the resource is active.\n x-celigo-ai-guidance:\n - Non-null means the resource is soft-deleted (in the recycle bin) — recoverable within the retention period before permanent removal.\n examples: [\"2023-05-20T11:45:32Z\"]\n required:\n - _id\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":2380,"content_sha256":"545364158eb63d4407ea79b5957cc357b55411ee852257bb240a60d88741bb70"},{"filename":"references/common-schemas/settings.yml","content":"Settings:\n type: object\n description: |\n Configuration settings that can be accessed by hooks, filters, mappings and handlebars templates at runtime.\n\n It enables customization of the resource's logic, allowing hooks, mappings, filters, and\n handlebars to access and apply the settings at runtime.\n\n **Usage**\n\n The settings object can store arbitrary JSON data that you want to save with the resource.\n While it's often populated through a form defined in the `settingsForm` field, you can also:\n\n - Directly provide JSON settings without using a form\n - Store configuration values used by hooks and templates\n - Create resource-specific constants and parameters\n - Maintain lookup tables or mapping structures\n - Define conditional logic parameters\n\n **Accessibility**\n\n Settings are available in:\n - All handlebars fields for building dynamic payloads\n - Field mapping expressions\n - JavaScript hooks via the options object\n - Filters and transformations\n\n At runtime, the settings objects in the step's execution chain are gathered into a\n single `settings` context keyed by fixed scope keys. Flow runs populate the full\n chain: `settings.integration.*`, `settings.flowGrouping.*` (the flow group the flow\n belongs to), `settings.flow.*`, `settings.connection.*`, `settings.iClient.*`, and\n the running step's own scope — `settings.export.*` on exports/lookups,\n `settings.import.*` on imports (the key matches the step type). Steps executing\n inside a My API or a Tool receive only the step's own scope plus\n `settings.connection.*` / `settings.iClient.*`; the integration, flowGrouping, and\n flow scopes resolve empty there. The scope key is literal — resource display names\n and settingsForm section labels are never part of the path. The segments after the\n scope mirror the stored settings JSON key path exactly: a top-level field is\n `settings.\u003cscope>.\u003cfieldId>`, and nested objects add one segment per JSON key\n (e.g. `settings.flowGrouping.Customer.region` when the group's settings JSON nests\n `region` under `Customer`). References without a scope key\n (e.g. `{{settings.myField}}`) resolve to empty strings.\n\n **Best practices**\n\n For non-technical users, create a custom form instead of editing the JSON directly.\n This provides a user-friendly interface for updating settings without requiring JSON knowledge.\n additionalProperties: true\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":2467,"content_sha256":"7fb3eb881109b6912af008c054c5d5f237fdb1c36430c16e3e43d18114bee12b"},{"filename":"references/common-schemas/transform.yml","content":"Transform:\n type: object\n description: |\n Configuration for transforming data during processing operations. This object enables\n reshaping of records.\n\n **Transformation capabilities**\n\n Celigo's transformation engine offers powerful features for data manipulation:\n - Precise field mapping with JSONPath expressions\n - Support for any level of nested arrays\n - Formula-based field value generation\n - Dynamic references to flow and integration settings\n\n **Implementation approaches**\n\n There are two distinct transformation mechanisms available:\n\n **Rule-Based Transformation (`type: \"expression\"`)**\n - **Best For**: Most transformation scenarios from simple to complex\n - **Capabilities**: Field mapping, formula calculations, lookups, nested data handling\n - **Advantages**: Visual configuration, no coding required, intuitive interface\n - **Configuration**: Define rules in the `expression` object\n - **Use When**: You have clear mapping requirements or need to reshape data structure\n\n **Script-Based Transformation (`type: \"script\"`)**\n - **Best For**: Extremely complex logic or proprietary algorithms\n - **Capabilities**: Full programmatic control, custom processing, complex business rules\n - **Advantages**: Maximum flexibility, can implement any transformation logic\n - **Configuration**: Reference a script in the `script` object\n - **Use When**: Visual transformation tools aren't sufficient for your use case\n properties:\n type:\n type: string\n description: |\n Determines which transformation mechanism to use. This choice affects which properties\n must be configured and how transformation logic is implemented.\n\n **Available types**\n\n **Rule-Based Transformation (`\"expression\"`)**\n - **Required Config**: The `expression` object with mapping definitions\n - **Behavior**: Applies declarative rules to reshape data\n - **Best For**: Most transformation scenarios from simple to complex\n - **Advantages**: Visual configuration, no coding required\n\n **Script-Based Transformation (`\"script\"`)**\n - **Required Config**: The `script` object with _scriptId and function\n - **Behavior**: Executes custom JavaScript to transform data\n - **Best For**: Extremely complex logic or proprietary algorithms\n - **Advantages**: Maximum flexibility, can implement any logic\n\n **Implementation guidance**\n\n 1. For standard data transformations, use `\"expression\"`\n 2. For complex logic or specialized processing, use `\"script\"`\n 3. When selecting a type, you must configure the corresponding object:\n - `type: \"expression\"` requires the `expression` object\n - `type: \"script\"` requires the `script` object\n enum: [\"expression\", \"script\"]\n x-enumDescriptions:\n expression: Rule-based transformation using declarative mapping definitions.\n script: Script-based transformation using custom JavaScript logic.\n examples: [\"expression\", \"script\"]\n expression:\n type: object\n description: |\n Configuration for declarative rule-based transformations. This object enables reshaping data\n without requiring custom code.\n\n **Usage context**\n\n This object is REQUIRED when `transform.type` is set to \"expression\" and should not be\n configured otherwise. It provides a standardized way to define transformation rules that\n can map, modify, and generate data elements.\n\n **Implementation guidance**\n\n The expression system uses a rule-based approach where:\n - Field mappings define how input data is transformed to target fields\n - Formulas can be used to calculate or generate new values\n - Lookups can enrich data by fetching related information\n - Mode determines how records are processed (create new or modify existing)\n properties:\n version:\n type: string\n description: |\n Version of the expression format. Determines which rules\n property contains the transformation logic.\n enum: [\"1\", \"2\"]\n x-enumDescriptions:\n \"1\": \"Legacy format. Rules are in the `rules` array (array of arrays of extract/generate objects).\"\n \"2\": \"Current format. Rules are in the `rulesTwoDotZero` object with mode, mappings, and lookups.\"\n examples: [\"1\", \"2\"]\n rules:\n type: array\n description: |\n Transformation rules for version 1 expressions. An array of\n rule groups; each group is an array of field-mapping objects.\n Most transforms have a single group. Present when `version`\n is `\"1\"`. The output record contains ONLY the generated\n fields — every unmapped field is dropped (v1 has no\n equivalent of Transform 2.0's `modify` mode), and the\n record's trace key does not survive the rebuild.\n x-celigo-ai-guidance:\n - |-\n v1 rules REPLACE the record: a one-field rule group drops\n every other field, silently breaking downstream filters,\n router branch rules, and templates that referenced the\n dropped fields. Either map every field downstream logic\n needs forward, or author Transform 2.0 with\n ``mode: \"modify\"`` instead.\n - |-\n An import's ``responseTransform`` is the exception: it accepts\n version ``\"1\"`` only (``\"2\"`` is refused with 422, verified live\n 2026-10-21), so there, map every response field that response\n mapping needs.\n items:\n type: array\n items:\n type: object\n properties:\n extract:\n type: string\n description: |\n Source field path to read from. Supports multiple\n syntaxes: bare field names (`id`), dot notation\n (`fulfillment.shipment_id`), slash-prefixed paths\n for XML (`/FeedProcessingStatus`), wildcards (`*.id`,\n `*.[Internal ID]`), and array indexing (`SDF[0]`).\n generate:\n type: string\n description: |\n Target field name to write to. Typically a bare name\n (`id`) or dot path (`SDF.Filter.ID`).\n key:\n type: string\n description: |\n Auto-generated identifier for this rule, used by the\n UI to track individual rules for editing and reordering.\n required:\n - extract\n - generate\n rulesTwoDotZero:\n type: object\n description: |\n Configuration for version 2 transformation rules. This object contains the core logic\n for how data is mapped, enriched, and transformed.\n\n **Capabilities**\n\n Transformation 2.0 provides:\n - Precise field mapping with JSONPath expressions\n - Support for deeply nested data structures\n - Formula-based field generation\n - Dynamic lookups for data enrichment\n - Multiple operating modes to fit different scenarios\n properties:\n mode:\n type: string\n description: |\n Transformation mode that determines how records are handled during processing.\n\n **Available modes**\n\n **Create Mode (`\"create\"`)**\n - **Behavior**: Builds entirely new output records from inputs\n - **Use When**: Output structure differs significantly from input\n - **Advantage**: Clean slate approach, no field inheritance\n\n **Modify Mode (`\"modify\"`)**\n - **Behavior**: Makes targeted edits to existing records\n - **Use When**: Output structure should remain similar to input\n - **Advantage**: Preserves unmapped fields from the original record\n enum: [\"create\", \"modify\"]\n x-enumDescriptions:\n create: Builds entirely new output records from inputs, ignoring unmapped fields.\n modify: Makes targeted edits to existing records, preserving unmapped fields.\n x-celigo-ai-guidance:\n - |-\n On a grouped record (the export groups rows, so the record is an\n array) only `create` can author output — build it from the rows with\n `$[0].field`, `$[*].field`, or a `buildArrayHelper` over `$[*]`.\n `modify` has no object to overlay onto an array: the record passes\n through untouched and the processor reports success. Either mode\n reshapes the inside of one record; neither changes how many records\n the page carries (that is the export's `preSavePage` hook) or whether\n the export groups at all (that is its grouping setting).\n examples: [\"create\", \"modify\"]\n mappings:\n $ref: './mappings.yml#/Mappings'\n lookups:\n allOf:\n - description: |\n Shared lookup tables used across all mappings defined in the transformation rules.\n\n **Purpose**\n\n Lookups provide centralized value translation that can be referenced from any mapping\n in your transformation configuration. They enable consistent translation of codes, IDs,\n and values between systems without duplicating translation logic.\n\n **Usage in transformations**\n\n Lookups are particularly valuable in transformations for:\n\n - **Data Normalization**: Standardizing values from diverse source systems\n - **Code Translation**: Converting between different coding systems (e.g., status codes)\n - **Field Enrichment**: Adding descriptive values based on ID or code lookups\n - **Cross-Reference Resolution**: Mapping identifiers between integrated systems\n\n **Implementation**\n\n Lookups are defined once in this array and referenced by name in mappings:\n\n ```json\n \"lookups\": [\n {\n \"name\": \"statusMapping\",\n \"map\": {\n \"A\": \"Active\",\n \"I\": \"Inactive\",\n \"P\": \"Pending\"\n },\n \"default\": \"Unknown Status\"\n }\n ]\n ```\n\n Then referenced in mappings using the lookupName property:\n\n ```json\n {\n \"generate\": \"status\",\n \"dataType\": \"string\",\n \"extract\": \"$.statusCode\",\n \"lookupName\": \"statusMapping\"\n }\n ```\n\n The system automatically applies the lookup during transformation processing.\n\n For complete details on lookup properties and behavior, see the Lookups schema.\n - $ref: './lookups.yml#/Lookups'\n inputContext:\n type: string\n enum:\n - record\n - envelope\n x-enumDescriptions:\n record: The transform processor evaluates `mappings[].extract` JSONPath against the flat record itself (`$.field`, no `record.` segment) — the default when absent.\n envelope: The transform processor evaluates JSONPath against the wrapped runtime envelope `{record, job, testMode, settings, headers, queryParams}` (file-provider steps also carry `fileMeta`) — record fields need the `record.` segment (`$.record.field`), while `headers` and `queryParams` sit beside `record` (`$.headers['X-Id']`, `$.queryParams.src`).\n description: |\n Controls the JSON shape the transformTwoDotZero processor\n evaluates `mappings[].extract` JSONPath values against at\n flow runtime. Applies only to Transform 2.0 (v2,\n `rulesTwoDotZero`); v1 transforms (the `rules` array on\n `transform.expression.rules`) and script-mode transforms\n ignore this field.\n x-celigo-ai-guidance:\n - |-\n ## Extract anchoring by mode\n\n - `record` (default — absent / empty / undefined): the\n processor sees the FLAT record as its evaluation root.\n `mappings[].extract` JSONPath MUST anchor directly at\n ` configuring-imports — Skillopedia with NO `record.` segment — e.g. `\"$.orderId\"`,\n `\"$.customer.email\"`; PLACEHOLDER extracts use\n `\"$.PLACEHOLDER_\u003cfield>\"`.\n This is the dominant Celigo idiom and the right choice for new transforms.\n - `envelope`: the processor sees a wrapped object\n `{record: {...}, job, testMode, settings, headers,\n queryParams}` (plus `fileMeta` on file-provider steps).\n `mappings[].extract` JSONPath MUST include the\n `record.` segment for record fields — e.g.\n `\"$.record.orderId\"` — and reads `headers` and\n `queryParams` beside it (`$.headers['X-Id']`);\n PLACEHOLDER extracts use `\"$.record.PLACEHOLDER_\u003cfield>\"`.\n Pick this only when the transform was authored against\n the envelope shape on purpose.\n - |-\n Handlebars expressions inside `extract`\n (`{{record.field}}`, `{{dateFormat ... record.field}}`)\n ALWAYS reference the `record` root regardless of\n `inputContext` — only JSONPath anchoring flips.\n - |-\n In `envelope` mode the output is still a bare record: a\n `modify` transform returns the record with its mapped\n fields merged in (no `record` wrapper, no `job` or\n `settings` keys), and a `create` transform returns only\n the mapped fields.\n - |-\n Emit `inputContext` explicitly on new v2 transforms:\n unset is read as `record`, but stating it makes the\n extract-anchoring contract self-documenting and immune\n to a platform default change.\n - |-\n ## Allowed values\n\n - ``record`` (default — emitted when the field is absent\n / empty / undefined): the transform processor sees\n the FLAT record itself as its evaluation root.\n ``mappings[].extract`` JSONPath values MUST anchor\n directly at `` configuring-imports — Skillopedia ` with NO ``record.`` segment — for\n example ``\"$.orderId\"`` or\n ``\"$.customer.email\"``. PLACEHOLDER extracts use\n ``\"$.PLACEHOLDER_\u003cfield>\"``. This is the dominant\n Celigo idiom and the right choice for new transforms.\n\n - ``envelope``: the transform processor sees a wrapped\n object of the shape\n ``{record: {...}, job, testMode, settings, headers,\n queryParams}`` (plus ``fileMeta`` on file-provider\n steps; ``headers`` and ``queryParams`` sit beside\n ``record``) — the\n same runtime-context envelope Handlebars / SQL /\n GraphQL templates render against. ``mappings[].extract``\n JSONPath values for record fields MUST include the\n ``record.`` segment — for example ``\"$.record.orderId\"``\n or ``\"$.record.customer.email\"`` — while ``headers`` and\n ``queryParams`` are read beside it\n (``\"$.headers['X-Id']\"``). PLACEHOLDER extracts\n use ``\"$.record.PLACEHOLDER_\u003cfield>\"``. Pick this\n mode only when the export was authored against the\n envelope shape on purpose (typically because an\n upstream step intentionally feeds the wrapped\n runtime context through to the transform).\n\n ## Handlebars expressions are NOT affected\n\n Handlebars expressions inside ``extract``\n (``{{record.field}}``,\n ``{{dateFormat ... record.field}}``, etc.) ALWAYS\n reference the runtime context's ``record`` root\n regardless of ``inputContext`` — the Handlebars runtime\n always sees the wrapped envelope. Only JSONPath\n behavior flips with this field.\n\n ## Agent guidance\n\n Set this field explicitly on every new v2 transform\n the agent authors. Leaving it unset is interpreted as\n ``record`` by the platform, but emitting\n ``inputContext: record`` makes the contract\n self-documenting and prevents the transform from\n drifting into envelope mode if an upstream change ever\n changes the platform default. Read this value before\n writing ``mappings[].extract`` so the JSONPath form\n matches the runtime root.\n - |-\n ## fileMeta on file-provider steps\n\n File-provider exports and lookups populate a\n ``fileMeta`` envelope key describing each record's\n source file: ``fileName``, ``fileSize``, and\n ``lastModifiedTime`` (epoch milliseconds). Reach it\n as ``$.fileMeta.\u003cfield>`` in envelope mode, or as\n ``{{fileMeta.\u003cfield>}}`` from a Handlebars extract in\n either mode. The keys are exactly these three —\n lowercase variants such as ``filesize`` or\n ``filename`` resolve empty. The standalone transform\n processor substitutes sample fileMeta values when the\n posted data lacks them, so a resolving editor render\n is not proof the step provides fileMeta — only\n file-provider steps populate it at flow runtime.\n examples: [\"record\", \"envelope\"]\n script:\n type: object\n description: |\n Configuration for programmable script-based transformations. This object enables complex, custom\n transformation logic beyond what expression-based transformations can provide.\n\n **Usage context**\n\n This object is REQUIRED when `transform.type` is set to \"script\" and should not be configured\n otherwise. It provides a way to execute custom JavaScript code to transform data according to\n specialized business rules or complex algorithms.\n\n **Implementation approach**\n\n Script-based transformation works by:\n 1. Executing the specified function from the referenced script\n 2. Passing input data to the function\n 3. Using the function's return value as the transformed output\n\n **Common use cases**\n\n Script transformation is ideal for:\n - Complex business logic that can't be expressed through mappings\n - Algorithmic transformations requiring computation\n - Dynamic transformations based on external factors\n - Legacy system data format compatibility\n - Multi-stage processing with intermediate steps\n\n Only use script-based transformation when expression-based transformation is insufficient.\n Script transformation requires maintaining custom code, which adds complexity to the integration.\n properties:\n _scriptId:\n type: string\n description: |\n Reference to a predefined script resource containing the transformation logic.\n\n The referenced script should contain the function specified in the\n 'function' property. It must be a script in this account: any other id fails the save\n with 422 (\"Only a script owned by the user or a script shared by the Integration App can\n be used at transform.script._scriptId.\").\n format: objectid\n x-celigo-refModel: scripts\n examples: [\"60a2c4e6f321d800129a1a3c\", \"5f8d43a1b9e5a80011a35f2c\"]\n function:\n type: string\n description: |\n Name of the function within the script to execute for transformation. This function\n must exist in the script referenced by _scriptId.\n examples: [\"transformData\", \"convertRecord\", \"applyMapping\"]\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":21239,"content_sha256":"0f2a1bdcf3fd4c5a271d5fc97ba73ce416e3c9a25c3767407d9d0eac9738578f"},{"filename":"references/guardrail-schemas/guardrail-config.yml","content":"# Guardrail-owned contract for the `guardrail` object. It mirrors the import spec's\n# GuardrailConfig field for field with one deliberate difference: the AI-agent branch\n# (GuardrailAiAgentConfig, below) is composed without the `tools` arrays that the server\n# rejects on guardrails, instead of $ref-ing the tool-bearing AiAgentConfig. Keep the shared\n# fields (type, confidenceThreshold, pii, moderation, and the provider blocks) in step with\n# specs/v1/import/components/schemas/{guardrail,aiagent}.yml when either side changes.\nGuardrailConfig:\n type: object\n description: |-\n Selects the check a guardrail applies and carries its settings. `type` picks the check\n and the matching sub-object (`pii`, `moderation`, or `aiAgent`) configures it.\n Responses echo the active sub-object with `confidenceThreshold` defaulted and also\n return the inactive `pii` / `moderation` siblings as empty stubs\n (`pii: {entities: [], mask: false}`, `moderation: {categories: []}`) — only the active\n type's sub-object is meaningful. Guardrails created before the AI-agent stub was\n stripped on write may still return an inert `aiAgent` on `pii` / `moderation` guardrails.\n x-celigo-ai-guidance:\n - |-\n Pick exactly one type. Prefer the deterministic local classifiers (`pii`, then\n `moderation`) whenever the brief's intent fits; use `ai_agent` only for custom\n judgement that has no `pii.entities` / `moderation.categories` analogue — the full\n decision rule is on `type`.\n - |-\n Guardrails never invoke tools. A non-empty `tools` array anywhere under `aiAgent`\n (`aiAgent.tools`, `aiAgent.openai.tools`, `aiAgent.litellm._overrides.gemini.tools`,\n `aiAgent.litellm._overrides.anthropic.tools`) fails the save with 422\n `guardrail_tools_not_allowed`. MCP `prompts` and `resources` are still accepted on\n save, but a guardrail's run never uses them: it gives the model only its instructions\n and input, whether the MCP server can be reached or not. Put the policy the guardrail\n must apply in its instructions.\n properties:\n type:\n type: string\n enum: [\"ai_agent\", \"pii\", \"moderation\"]\n x-enumDescriptions:\n ai_agent: Evaluate data using an AI model with custom instructions.\n pii: Detect personally identifiable information in data fields.\n moderation: Check content for harmful or inappropriate categories.\n description: |-\n The type of guardrail to apply. Each type requires its corresponding\n sub-configuration object (`aiAgent`, `pii`, or `moderation`).\n x-celigo-ai-guidance:\n - |-\n The type of guardrail to apply.\n Pick exactly one — `pii` / `moderation` / `ai_agent` are mutually exclusive at\n the type level (a brief that wants both runs as two guardrails in sequence,\n not one).\n - |-\n ## How to choose\n\n Apply these rules **in order** and pick the first match. Do\n NOT skip ahead to `ai_agent` just because the brief is in\n natural language — `pii` and `moderation` are also expressed\n as natural-language briefs. The decision is about WHAT the\n brief asks the guardrail to detect, not how the brief is\n worded.\n\n 1. **`pii`** — pick this when the brief names ANY personally\n identifiable information, OR when the brief names entity\n categories (SSN, email, phone, credit card, address, name,\n passport, license, account number, etc.) that match\n `pii.entities`. Examples that route here:\n \"detect SSN and credit card numbers\", \"mask customer\n emails and phone numbers\", \"scan for PII\".\n\n 2. **`moderation`** — pick this when the brief asks the\n guardrail to detect harmful, inappropriate, unsafe, or\n policy-violating CONTENT (as opposed to PII data fields)\n AND the intent maps to any value in `moderation.categories`\n (`sexual`, `hate`, `harassment`, `self_harm`, `violence`,\n `illicit`, etc.) — including via close synonyms. Apply\n the synonym mapping liberally:\n - \"explicit\", \"obscene\", \"vulgar\", \"profane\",\n \"inappropriate language\", \"swearing\", \"NSFW\" → typically\n maps to `sexual` and/or `harassment`\n - \"abusive\", \"insulting\", \"bullying\", \"toxic\" → `harassment`\n - \"hateful\", \"discriminatory\", \"racist\", \"sexist\" → `hate`\n - \"threatening\", \"intimidating\" → `harassment_threatening`\n or `violence`\n - \"graphic violence\", \"gore\" → `violence_graphic`\n - \"suicide\", \"self-injury\" → `self_harm`\n - \"drugs\", \"weapons\", \"illegal activity\" → `illicit`\n Briefs like \"screen email body for explicit language\",\n \"block toxic chat messages\", \"flag harassment in support\n tickets\", \"moderate user-generated content for unsafe\n material\" all route to `moderation`. When the brief uses\n a synonym, populate `moderation.categories` with the\n closest enum value(s) — do NOT invent new category\n strings; the enum is fixed.\n\n 3. **`ai_agent`** — pick this ONLY when the brief asks for\n judgement that doesn't fit `pii` or `moderation`. Typical\n triggers: domain-specific compliance checks (HIPAA / SOX\n rule evaluation), business-policy validation (price\n bounds, approval thresholds), data-quality assertions\n (required fields populated, values plausible), or any\n \"evaluate against these custom rules\" framing where the\n rules are domain-specific and have no `pii.entities` /\n `moderation.categories` analogue. If you can map the\n brief's intent to a moderation category (even via a\n synonym above), use `moderation` instead — `ai_agent` is\n the catch-all, not the default.\n\n Tie-breaker for borderline cases: prefer the more specific\n type (`pii` > `moderation` > `ai_agent`). `pii` and\n `moderation` are local deterministic classifiers with fixed\n enums — they're cheaper, faster, and more predictable than\n `ai_agent` (which calls a real LLM per record). Default to\n the cheaper path whenever the intent fits.\n - |-\n ## Sub-configuration requirements (one per type)\n\n - **ai_agent** requires the `aiAgent` sub-configuration.\n - **pii** requires the `pii` sub-configuration with at least\n one entity from `pii.entities`.\n - **moderation** requires the `moderation` sub-configuration\n with at least one category from `moderation.categories`.\n confidenceThreshold:\n type: number\n minimum: 0\n maximum: 1\n default: 0.7\n x-celigo-ui-override: >-\n Required by the guardrail form (confidenceThreshold is required:true once a type is\n selected, default 0.7). Encoded to mirror the form so builders produce connectable\n configurations.\n description: |-\n Confidence threshold (0 to 1). Detections below this threshold are\n ignored. Lower values catch more issues but increase false positives.\n x-celigo-ai-guidance:\n - Confidence threshold for guardrail detection (0 to 1).\n - |-\n Only detections with confidence at or above this threshold will be flagged.\n Lower values catch more potential issues but may increase false positives.\n examples: [0.7, 0.5, 0.9]\n aiAgent:\n type: object\n description: |-\n LLM configuration for the check; set when `type` is `ai_agent` and validated\n against `GuardrailAiAgentConfig`. On `pii` / `moderation` guardrails a legacy\n server-written stub may appear here — it is inert, and current servers strip it on\n write.\n pii:\n type: object\n required: [entities]\n description: |-\n PII detection configuration. Required when `type` is `pii`.\n x-celigo-ai-guidance:\n - Configuration for PII (Personally Identifiable Information) detection.\n - |-\n Required when `guardrail.type` is \"pii\".\n At least one entity type must be specified.\n properties:\n entities:\n type: array\n description: |-\n PII entity types to detect. When `type` is `pii`, at least one\n entry is required; the inactive sibling on other guardrail types\n may be served with an empty list.\n x-celigo-ai-guidance:\n - PII entity types to detect in the data.\n - At least one entity must be specified when using PII guardrails.\n items:\n type: string\n enum:\n - credit_card_number\n - card_security_code_cvv_cvc\n - cryptocurrency_wallet_address\n - date_and_time\n - email_address\n - iban_code\n - bic_swift_bank_identifier_code\n - ip_address\n - location\n - medical_license_number\n - national_registration_number\n - persons_name\n - phone_number\n - url\n - us_bank_account_number\n - us_drivers_license\n - us_itin\n - us_passport_number\n - us_social_security_number\n - uk_nhs_number\n - uk_national_insurance_number\n - spanish_nif\n - spanish_nie\n - italian_fiscal_code\n - italian_drivers_license\n - italian_vat_code\n - italian_passport\n - italian_identity_card\n - polish_pesel\n - finnish_personal_identity_code\n - singapore_nric_fin\n - singapore_uen\n - australian_abn\n - australian_acn\n - australian_tfn\n - australian_medicare\n - indian_pan\n - indian_aadhaar\n - indian_vehicle_registration\n - indian_voter_id\n - indian_passport\n - korean_resident_registration_number\n x-enumDescriptions:\n credit_card_number: Credit or debit card number (e.g., Visa, Mastercard, Amex).\n card_security_code_cvv_cvc: Card security code printed on credit or debit cards (CVV/CVC).\n cryptocurrency_wallet_address: Cryptocurrency wallet address (e.g., Bitcoin, Ethereum).\n date_and_time: Date and/or time values that could identify an individual.\n email_address: Email address in standard format.\n iban_code: International Bank Account Number used for cross-border payments.\n bic_swift_bank_identifier_code: BIC/SWIFT code identifying a specific bank for international transfers.\n ip_address: IPv4 or IPv6 network address.\n location: Physical location or geographic coordinates.\n medical_license_number: Medical professional license or registration number.\n national_registration_number: National identification or registration number.\n persons_name: Full or partial name of a person.\n phone_number: Telephone number in any format.\n url: Web URL or URI.\n us_bank_account_number: United States bank account number.\n us_drivers_license: United States driver's license number.\n us_itin: United States Individual Taxpayer Identification Number.\n us_passport_number: United States passport number.\n us_social_security_number: United States Social Security Number (SSN).\n uk_nhs_number: United Kingdom National Health Service number.\n uk_national_insurance_number: United Kingdom National Insurance number.\n spanish_nif: Spanish tax identification number (NIF).\n spanish_nie: Spanish foreigner identification number (NIE).\n italian_fiscal_code: Italian fiscal code (Codice Fiscale).\n italian_drivers_license: Italian driver's license number.\n italian_vat_code: Italian VAT identification number (Partita IVA).\n italian_passport: Italian passport number.\n italian_identity_card: Italian national identity card number.\n polish_pesel: Polish national identification number (PESEL).\n finnish_personal_identity_code: Finnish personal identity code (henkilotunnus).\n singapore_nric_fin: Singapore National Registration Identity Card or Foreign Identification Number.\n singapore_uen: Singapore Unique Entity Number for business registration.\n australian_abn: Australian Business Number.\n australian_acn: Australian Company Number.\n australian_tfn: Australian Tax File Number.\n australian_medicare: Australian Medicare card number.\n indian_pan: Indian Permanent Account Number for tax purposes.\n indian_aadhaar: Indian Aadhaar unique identity number.\n indian_vehicle_registration: Indian vehicle registration number.\n indian_voter_id: Indian voter identification card number.\n indian_passport: Indian passport number.\n korean_resident_registration_number: South Korean resident registration number.\n examples:\n - [\"email_address\", \"phone_number\", \"persons_name\"]\n - [\"us_social_security_number\", \"credit_card_number\"]\n mask:\n type: boolean\n default: false\n description: |-\n When true, the result also carries `masked`: the evaluated input with each detected\n value replaced by asterisks. The record itself is unchanged. With whole-record input\n (no `mapping`, and no top-level `text` field on the record), `masked` is the record\n serialized as one JSON string. When false, PII is flagged only.\n x-celigo-ai-guidance:\n - |-\n Masking doesn't rewrite the record. To pass masked text on, select the field as the\n `text` input (`mapping`) and response-map `masked` back onto that field on the\n guardrail's flow step (`{\"extract\": \"masked\", \"generate\": \"\u003cfield>\"}`).\n moderation:\n type: object\n required: [categories]\n description: |-\n Content moderation configuration. Required when `type` is `moderation`.\n x-celigo-ai-guidance:\n - |-\n Required when `guardrail.type` is \"moderation\".\n At least one category must be specified.\n properties:\n categories:\n type: array\n description: |-\n Content moderation categories to check. When `type` is\n `moderation`, at least one entry is required; the inactive\n sibling on other guardrail types may be served with an empty\n list.\n x-celigo-ai-guidance:\n - At least one category must be specified when using moderation guardrails.\n items:\n type: string\n enum:\n - sexual\n - sexual_minors\n - hate\n - hate_threatening\n - harassment\n - harassment_threatening\n - self_harm\n - self_harm_intent\n - self_harm_instructions\n - violence\n - violence_graphic\n - illicit\n - illicit_violent\n x-enumDescriptions:\n sexual: Content depicting sexual activity or explicit sexual material.\n sexual_minors: Sexual content involving minors.\n hate: Content expressing hatred toward a group based on protected characteristics.\n hate_threatening: Hateful content that includes threats of violence or serious harm.\n harassment: Content that targets, intimidates, or bullies an individual.\n harassment_threatening: Harassment content that includes threats of violence or serious harm.\n self_harm: Content that promotes or depicts self-harm behaviors.\n self_harm_intent: Content expressing intent to engage in self-harm.\n self_harm_instructions: Content providing instructions for self-harm methods.\n violence: Content depicting or promoting physical violence.\n violence_graphic: Graphic or gory depictions of violence or injury.\n illicit: Content promoting illegal activities or unlawful behavior.\n illicit_violent: Content promoting illegal activities that involve violence.\n examples:\n - [\"hate\", \"violence\", \"harassment\"]\n - [\"sexual\", \"self_harm\", \"illicit\"]\n required:\n - type\n - confidenceThreshold\n if:\n required: [type]\n properties:\n type:\n const: pii\n then:\n required: [pii]\n properties:\n pii:\n required: [entities]\n properties:\n entities:\n minItems: 1\n else:\n if:\n required: [type]\n properties:\n type:\n const: moderation\n then:\n required: [moderation]\n properties:\n moderation:\n required: [categories]\n properties:\n categories:\n minItems: 1\n else:\n required: [aiAgent]\n properties:\n aiAgent:\n $ref: '#/GuardrailAiAgentConfig'\n\nGuardrailAiAgentConfig:\n type: object\n description: |\n LLM configuration for an `ai_agent` guardrail — which provider and model evaluate each\n record, with instructions, parameter tuning, and output format. Guardrails never invoke\n tools, so unlike an AI agent import no block here accepts a `tools` array. MCP `prompts`\n and `resources` are accepted on save but never used at run time, so put the policy the\n guardrail must apply in its instructions. Providers come in two families with two\n configuration formats:\n\n - **Built-in providers** — each has its own configuration block.\n - **openai**: OpenAI models (GPT-4.1, GPT-5, etc.). Configure via the `openai` object.\n - **gemini**: Google Gemini models via the LiteLLM proxy. Configure via `litellm` with overrides in `litellm._overrides.gemini`.\n - **anthropic**: Anthropic Claude models via the LiteLLM proxy. Configure via `litellm` with overrides in `litellm._overrides.anthropic`.\n - **Catalog providers** — `mistral`, `xai`, `huggingface`, `deepseek`, `cohere`, `groq`. Added\n as catalog data rather than platform code, they share one generic flat format:\n `model`, `modelOptions`, `instructions`, `output`, and the `prompts` / `resources`\n arrays directly on this object.\n\n A `_connectionId` on the parent import selects bring-your-own-key (BYOK)\n credentials. Without one, built-in providers run on platform-managed\n credentials; catalog providers have no platform-managed credentials, so the\n save requires one — without it the guardrail fails to save with 422 on\n `guardrail.aiAgent.model`: `ai_model_not_available` for a catalog-listed model,\n `ai_model_not_in_catalog` for one the catalog does not list (the model\n catalog check answers before any `connection_required`).\n\n **Model lifecycle.** Every model the platform offers carries a lifecycle in\n the platform's model catalog: `active`, `deprecated` (still usable; a\n retirement date and suggested replacement models may be published), or\n `retired`, plus an availability switch. The save and every run of an\n `ai_agent` guardrail check the configured model against it exactly as for an\n AI agent import: a model that is not in the catalog, switched off, not offered\n for the guardrail's credential type, or retired (by status, or once its\n retirement date has passed) is refused — the save with 422\n `ai_model_not_in_catalog`, `ai_model_disabled`, `ai_model_not_available`, or\n `ai_model_retired` on the model field, the run with a `Run blocked:` error\n carrying the same code before any model call (`Run blocked: model\n \"gpt-4.1-nano\" is retired on \"Oct 23, 2026\". Please choose a supported model\n and test your agent configuration.` — the run-time message names the date\n only). A deprecated model saves and runs until\n its retirement date. The platform never substitutes a model: once a model is\n retired, every save of the guardrail is refused until the model changes.\n Guardrails on affected models are listed by `GET /v1/aiagents/aiModelHealth`.\n With a `_connectionId`, a model the catalog does not list is passed through to\n the vendor as-is; a catalog-listed model remains governed.\n x-celigo-ai-guidance:\n - |-\n Do not send `tools` on any block. A non-empty array at `tools`, `openai.tools`,\n `litellm._overrides.gemini.tools`, or `litellm._overrides.anthropic.tools` fails the\n save with 422 `guardrail_tools_not_allowed` naming that path. Responses still echo\n `openai.tools: []` on OpenAI guardrails; do not copy it back into a write.\n required: [provider]\n properties:\n provider:\n type: string\n enum: [\"openai\", \"gemini\", \"anthropic\", \"mistral\", \"xai\", \"huggingface\", \"deepseek\", \"cohere\", \"groq\"]\n x-enumDescriptions:\n openai: Use OpenAI models via the OpenAI Responses API (built-in; `openai` block).\n gemini: Use Google Gemini models via the LiteLLM proxy (built-in; `litellm` block).\n anthropic: Use Anthropic Claude models via the LiteLLM proxy (built-in; `litellm` block).\n mistral: Use Mistral models (catalog provider; generic flat format, BYOK).\n xai: Use xAI Grok models (catalog provider; generic flat format, BYOK).\n huggingface: Use Hugging Face inference endpoint models (catalog provider; generic flat format, BYOK).\n deepseek: Use DeepSeek models (catalog provider; generic flat format, BYOK).\n cohere: Use Cohere models (catalog provider; generic flat format, BYOK).\n groq: Use Groq-hosted models (catalog provider; generic flat format, BYOK).\n description: AI provider to use.\n x-celigo-ai-guidance:\n - |-\n The provider decides the configuration format, and the server rejects a\n mismatch with 422 `invalid_provider_config_format`: `openai` needs the\n `openai` block, `gemini` and `anthropic` need the `litellm` block (Gemini- or\n Claude-specific settings under `litellm._overrides`), and the catalog\n providers (`mistral`, `xai`, `huggingface`, `deepseek`, `cohere`, `groq`) need the\n generic flat format — `model` and `modelOptions` directly on this object,\n never an `openai` or `litellm` block.\n - |-\n Catalog providers are BYOK-only, and the save enforces it through the model\n catalog: without a `_connectionId` the guardrail fails to save with 422\n `ai_model_not_available` on `guardrail.aiAgent.model` (\"Model \\\"grok-4.6\\\" is\n not supported. Please choose a supported model and test your agent\n configuration.\") — the model is a catalog row that is not offered on\n platform-managed credentials. Attach an HTTP connection to the vendor's API\n (token auth) before saving a catalog-provider guardrail.\n model:\n type: string\n description: |-\n Model identifier for a catalog provider (generic flat format) — for example\n `grok-4.6`. Built-in providers set the model inside their own block\n (`openai.model`, `litellm.model`) instead.\n x-celigo-ai-guidance:\n - |-\n Required for every catalog provider — omitting it or sending an empty string\n fails the save with 422 `missing_required_field`.\n - |-\n Without a `_connectionId` on the parent import the model check answers first:\n a catalog-provider guardrail fails to save with 422 on `guardrail.aiAgent.model` —\n `ai_model_not_available` for a catalog-listed model (catalog providers are not\n offered on platform-managed credentials), `ai_model_not_in_catalog` for an id\n the catalog does not list.\n With a `_connectionId` (BYOK) a model the catalog does not list saves as-is —\n the server does not check it against the catalog or the vendor; a\n catalog-listed model is still governed by its lifecycle row (see the\n schema description — a retired or switched-off model is refused on save).\n modelOptions:\n type: object\n description: |-\n Model tuning for a catalog provider (generic flat format). Which keys, values, and\n ranges are accepted is defined per catalog model and validated on save when the\n model is a catalog entry.\n properties:\n maxOutputTokens:\n type: number\n description: Maximum number of tokens the model may generate.\n temperature:\n type: number\n description: Sampling temperature.\n topP:\n type: number\n description: Nucleus sampling threshold.\n reasoning_effort:\n type: string\n enum: [\"low\", \"medium\", \"high\", \"xhigh\"]\n x-enumDescriptions:\n low: Minimal reasoning before answering.\n medium: Moderate reasoning.\n high: Extensive reasoning.\n xhigh: Maximum reasoning.\n description: Reasoning depth for models that expose it.\n x-celigo-ai-guidance:\n - |-\n When the model is a catalog entry, the server validates this object against\n that model's schema and fails the save with 422 `invalid_model_options`\n naming the key: unknown keys are rejected, `reasoning_effort` must be one of\n the enum values where the model accepts it at all (xAI's `grok-4.6` does;\n DeepSeek's `deepseek-chat` rejects the key), and numeric bounds apply —\n on `grok-4.6`, `temperature` ≤ 2, `topP` ≤ 1, `maxOutputTokens` ≥ 100.\n - |-\n With a `_connectionId` and a model the catalog does not list, the object is\n passed through unvalidated — send only options the vendor's API accepts.\n instructions:\n type: string\n description: |-\n System prompt for a catalog provider (generic flat format). Built-in providers\n set it inside their own block (`openai.instructions`,\n `litellm._overrides.anthropic.systemInstruction`).\n output:\n type: object\n description: |-\n Output format for a catalog provider (generic flat format). Built-in providers\n configure it inside their own block (`openai.output`, `litellm.responseFormat`).\n properties:\n format:\n type: object\n description: Controls the structure of the model's output.\n properties:\n type:\n type: string\n enum: [\"text\", \"json_schema\"]\n x-enumDescriptions:\n text: Free-form text.\n json_schema: Structured JSON conforming to `jsonSchema`.\n description: Output type.\n x-celigo-ai-guidance:\n - |-\n `blob` is not accepted here — the save fails with 422\n `unsupported_output_format` naming the model (verified on `grok-4.6`).\n name:\n type: string\n description: Name of the JSON schema, for `json_schema` output.\n strict:\n type: boolean\n description: When true, the model must conform exactly to `jsonSchema`.\n jsonSchema:\n type: object\n additionalProperties: true\n description: JSON Schema the structured output must conform to, for `json_schema` output.\n verbose:\n type: string\n description: Level of detail in the model's response, for models that expose it.\n tools:\n type: array\n deprecated: true\n description: |-\n Not available on guardrails: a non-empty array fails the save with 422\n `guardrail_tools_not_allowed`. Kept in source only because the field exists on the\n shared import model.\n items:\n type: object\n additionalProperties: true\n prompts:\n type: array\n description: |-\n MCP prompt entries saved with a catalog-provider guardrail (generic flat format).\n Each entry references one MCP connection and prompt names; a guardrail's run never fetches them.\n items:\n type: object\n properties:\n type:\n type: string\n enum: [\"mcp\"]\n x-enumDescriptions:\n mcp: Reference prompts exposed by an MCP server.\n description: Type of prompt entry. Always \"mcp\".\n mcp:\n type: object\n properties:\n _mcpConnectionId:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Connection to the MCP server that exposes the prompts.\n allowedPrompts:\n type: array\n description: Prompt names saved with the entry; a guardrail's run never fetches them.\n items:\n type: string\n x-celigo-ai-guidance:\n - |-\n An unknown `_mcpConnectionId` fails the save with 422 `invalid_ref` naming\n the array index.\n resources:\n $ref: '../schemas/aiagent.yml#/McpResources'\n openai:\n type: object\n description: |\n OpenAI-specific configuration. Used when `provider` is \"openai\".\n required: [model, instructions]\n properties:\n instructions:\n type: string\n maxLength: 1000000\n description: |\n System prompt that defines the check's behavior, goals, and constraints.\n examples: [\"You are a data validation agent. Check each record for completeness.\"]\n x-celigo-ui-override: >-\n Required by the AI agent configuration form (the model's system prompt). Encoded\n here to mirror the form so builders produce connectable configurations.\n model:\n type: string\n description: |-\n OpenAI model identifier. Open string (not an enum) — model names change frequently.\n examples: [\"gpt-5.4\", \"gpt-5.4-pro\", \"gpt-5-mini\", \"gpt-5-nano\", \"gpt-4.1-mini\"]\n x-celigo-ui-override: >-\n Required by the AI agent configuration form. Encoded here to mirror the form so\n builders produce connectable configurations.\n x-celigo-ai-guidance:\n - |-\n With platform-managed credentials (no `_connectionId` on the parent\n import), the model must be a catalog row the platform offers on its own\n credentials. The save checks this — a model that is not in the catalog\n fails with 422 `ai_model_not_in_catalog` on\n `guardrail.aiAgent.openai.model` (\"Model \\\"gpt-4\\\" is not supported.\n Please choose a supported model and test your agent configuration.\"). A\n retired model is refused the same way with `ai_model_retired` — the\n message names the retirement date and, when any remain eligible, the\n suggested next models (\"Model \\\"gpt-4.1-nano\\\" is retired on \\\"Oct 23,\n 2026\\\" and cannot be used. Suggested next models: gpt-5.6-luna. Please\n test and update your agent configuration.\") — and so is a switched-off\n one with `ai_model_disabled`; a deprecated model saves. Never swap the\n model on the user's behalf — relay the message and let them choose.\n - |-\n With a `_connectionId` (BYOK) a model the catalog does not list is passed\n through to the provider as-is; a catalog-listed model is still governed by\n its lifecycle row.\n reasoning:\n type: object\n description: |-\n Controls depth of reasoning for complex tasks.\n x-celigo-ai-guidance:\n - |-\n Supported on reasoning-capable OpenAI models only (currently the GPT-5 family and the o-series).\n Setting `reasoning` on a non-reasoning model (e.g.\n `gpt-4.1-mini`) is rejected by the provider at runtime — omit the field on those models.\n properties:\n effort:\n type: string\n enum: [\"none\", \"minimal\", \"low\", \"medium\", \"high\", \"xhigh\", \"max\"]\n x-enumDescriptions:\n none: No additional reasoning effort.\n minimal: Least reasoning effort for simple, straightforward tasks.\n low: Light reasoning for moderately simple tasks.\n medium: Balanced reasoning effort for typical tasks.\n high: Extensive reasoning effort for complex, multi-step tasks.\n xhigh: Extended reasoning effort beyond high for demanding tasks.\n max: Highest available reasoning effort.\n description: How much reasoning effort the model should invest\n x-celigo-ai-guidance:\n - |-\n How much reasoning effort the model should invest.\n Reasoning- capable models only — see `reasoning` for the gating rule.\n summary:\n type: string\n enum: [\"concise\", \"auto\", \"detailed\"]\n x-enumDescriptions:\n concise: Brief, high-level reasoning summary.\n auto: Let the model decide the appropriate level of detail.\n detailed: Comprehensive reasoning summary with full explanations.\n description: Level of detail in reasoning summaries\n x-celigo-ai-guidance:\n - |-\n Level of detail in reasoning summaries.\n Reasoning-capable models only — see `reasoning` for the gating rule.\n temperature:\n type: number\n minimum: 0\n maximum: 2\n description: |\n Sampling temperature. Higher values (e.g. 1.5) produce more creative output,\n lower values (e.g. 0.2) produce more focused and deterministic output.\n topP:\n type: number\n minimum: 0.1\n maximum: 1\n description: Nucleus sampling parameter\n topLogprobs:\n type: number\n minimum: 0\n maximum: 20\n description: Number of most likely tokens to return log probabilities for at each output position.\n maxOutputTokens:\n type: number\n minimum: 100\n maximum: 128000\n description: Maximum number of tokens in the model's response. Unlike an AI agent import, a guardrail does not receive a server-written value when this is omitted.\n serviceTier:\n type: string\n enum: [\"auto\", \"default\", \"priority\"]\n x-enumDescriptions:\n auto: Let OpenAI automatically select the appropriate service tier.\n default: Standard service tier with normal rate limits and latency.\n priority: Premium tier with higher rate limits and lower latency at increased cost.\n description: |-\n OpenAI service tier. \"priority\" provides higher rate limits and\n lower latency at increased cost.\n x-celigo-ai-guidance:\n - |-\n BYOK only.\n Platform-managed credentials (no `_connectionId` on the parent import) always\n run at `default` regardless of what's set here.\n output:\n type: object\n description: |-\n Output format configuration. A guardrail stores `format` but doesn't use it: the\n request to the model always carries the guardrail's own `validation_result` JSON\n schema (`flagged`, `reasoning`, `confidence`), and the result returns `flagged` and\n `reasoning` whatever `format` holds.\n properties:\n format:\n type: object\n description: |\n Stored output structure. The request to the model uses the guardrail's own\n verdict schema instead (see `output`).\n properties:\n type:\n type: string\n enum: [\"text\", \"json_schema\", \"blob\"]\n x-enumDescriptions:\n text: Stored format type; the model request still uses the guardrail's verdict schema.\n json_schema: A stored JSON schema (`jsonSchema`); the model request still uses the guardrail's verdict schema.\n blob: Stored format type; the model request still uses the guardrail's verdict schema.\n description: Output format type, as stored.\n schemaMode:\n type: string\n enum: [\"manual\", \"json\"]\n x-enumDescriptions:\n manual: The schema editor opens `jsonSchema` in the visual field-by-field view\n json: The schema editor opens `jsonSchema` in the raw JSON view\n description: |-\n Which view a schema editor opens `jsonSchema` in. Stored and echoed on\n guardrails for parity with the AI-agent format block — the guardrail form\n does not set it today, and neither view changes the model request.\n x-celigo-ai-guidance:\n - |-\n Resend the stored value on every update — a full PUT that omits the field\n clears it — and never invent one: a value outside the enum fails the save with\n 422 `enum` on `guardrail.aiAgent.openai.output.format.schemaMode`.\n name:\n type: string\n description: Name for the output format (used with json_schema)\n x-celigo-ui-override: >-\n Required by the AI agent form when output format type is json_schema. Encoded\n here to mirror the form so builders produce connectable configurations.\n strict:\n type: boolean\n description: Stored strict-validation flag for `jsonSchema`. A guardrail's model request uses its own verdict schema, so this doesn't apply there.\n x-celigo-ai-guidance:\n - Stored only on a guardrail; it doesn't change the guardrail's verdict.\n jsonSchema:\n type: object\n description: |\n Stored JSON Schema. Required when `format.type` is \"json_schema\". A guardrail\n doesn't send it to the model; the request carries the `validation_result`\n verdict schema instead.\n x-celigo-ui-override: >-\n Required by the AI agent form when output format type is json_schema. Encoded\n here to mirror the form so builders produce connectable configurations.\n properties:\n type:\n type: string\n description: Root type of the stored JSON Schema.\n enum: [\"object\", \"array\", \"string\", \"number\", \"integer\", \"boolean\"]\n x-enumDescriptions:\n object: JSON object with named properties.\n array: Ordered list of values.\n string: Text string value.\n number: Numeric value including decimals.\n integer: Whole number value without decimals.\n boolean: True or false value.\n properties:\n type: object\n additionalProperties: true\n description: Field definitions in the stored JSON Schema.\n required:\n type: array\n description: Property names the stored JSON Schema marks as required.\n items:\n type: string\n additionalProperties:\n type: boolean\n description: Whether the stored JSON Schema allows properties beyond those in `properties`.\n schema:\n type: object\n readOnly: true\n additionalProperties: true\n description: |-\n Server-written copy of `jsonSchema`, present in responses once `jsonSchema`\n is set. Send `jsonSchema`, not this field.\n if:\n properties:\n type:\n const: json_schema\n required: [type]\n then:\n required: [name, jsonSchema]\n verbose:\n type: string\n enum: [\"low\", \"medium\", \"high\"]\n x-enumDescriptions:\n low: Minimal detail in the model's response.\n medium: Moderate detail in the model's response, the default level.\n high: Maximum detail and verbosity in the model's response.\n description: Level of detail in the model's response\n x-celigo-ai-guidance:\n - |-\n Supported on the GPT-5 family only. Other OpenAI models (e.g.\n `gpt-4.1`, `gpt-4.1-mini`,\n `gpt-4.1-nano`) accept only `medium` (the no-op default) — `low` and `high` are\n rejected by the provider at runtime.\n Omit the field entirely on non-GPT-5 models rather than relying on the default.\n tools:\n type: array\n deprecated: true\n description: |-\n Always an empty array on guardrails saved today: the server writes `[]` and\n rejects a non-empty array with 422 `guardrail_tools_not_allowed`. Guardrails\n saved before the restriction may still return their stored entries.\n items:\n type: object\n additionalProperties: true\n prompts:\n type: array\n description: |-\n MCP prompt entries saved with the guardrail. Each item references one\n MCP connection and prompt names; a guardrail's run never fetches them.\n items:\n type: object\n properties:\n type:\n type: string\n enum: [\"mcp\"]\n x-enumDescriptions:\n mcp: Connect to an MCP server to use its exposed prompts.\n description: Type of prompt entry. Always \"mcp\".\n mcp:\n type: object\n description: MCP server prompt configuration.\n properties:\n _mcpConnectionId:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Connection to the MCP server.\n allowedPrompts:\n type: array\n description: Prompt names saved with the entry; a guardrail's run never fetches them.\n items:\n type: string\n resources:\n $ref: '../schemas/aiagent.yml#/McpResources'\n litellm:\n type: object\n description: |\n LiteLLM proxy configuration. Used when `provider` is \"gemini\" or \"anthropic\".\n\n LiteLLM provides a unified interface to multiple AI providers. Gemini-specific\n settings are in `_overrides.gemini`; Claude-specific settings are in\n `_overrides.anthropic`.\n\n `model` is required when litellm is the active provider path.\n properties:\n model:\n type: string\n description: |-\n LiteLLM model identifier. For Gemini, models are stored without the `gemini/`\n prefix; for Anthropic, use the Claude model id (e.g. `claude-sonnet-4-6`).\n examples: [\"gemini-2.5-flash\", \"gemini-2.5-pro\", \"gemini-2.5-flash-lite\", \"claude-sonnet-4-6\", \"claude-opus-4-7\"]\n x-celigo-ui-override: >-\n Required by the AI agent configuration form. Encoded here to mirror the form so\n builders produce connectable configurations.\n x-celigo-ai-guidance:\n - |-\n With platform-managed credentials (no `_connectionId` on the parent\n import), the model must be a catalog row the platform offers for the\n provider on its own credentials — Gemini and Anthropic Claude each have\n their own rows. The save checks this: a model that is not in the catalog\n fails with 422 `ai_model_not_in_catalog` on\n `guardrail.aiAgent.litellm.model`, a retired one with `ai_model_retired`\n (the message names the retirement date and, when any remain eligible, the\n suggested next models), a switched-off one with `ai_model_disabled`; a\n deprecated model saves. Never\n swap the model on the user's behalf — relay the message and let them choose.\n - |-\n With a `_connectionId` (BYOK) a model the catalog does not list is passed\n through to the provider as-is; a catalog-listed model is still governed by\n its lifecycle row.\n temperature:\n type: number\n minimum: 0\n maximum: 2\n description: Sampling temperature\n x-celigo-ai-guidance:\n - |-\n When `provider` is \"anthropic\", the value must be between 0 and 1 — a higher value\n fails the save with 422 `invalid_field_value`. Gemini accepts the full 0-2 range.\n maxCompletionTokens:\n type: number\n minimum: 100\n maximum: 128000\n description: Maximum number of tokens in the response\n topP:\n type: number\n minimum: 0.1\n maximum: 1\n description: Nucleus sampling parameter\n seed:\n type: number\n description: Random seed for reproducible outputs\n responseFormat:\n type: object\n description: Output format configuration\n x-celigo-ai-guidance:\n - |-\n When `provider` is \"anthropic\", `type` must be \"text\" or \"json_schema\" — \"blob\" is\n rejected with 422 `invalid_response_format`. As with the other providers, \"json_schema\"\n requires `name` and `jsonSchema`.\n properties:\n type:\n type: string\n description: Output format type.\n enum: [\"text\", \"json_schema\", \"blob\"]\n x-enumDescriptions:\n text: Free-form text response from the model.\n json_schema: Structured JSON output conforming to a defined schema.\n blob: Binary data output for non-text content.\n schemaMode:\n type: string\n enum: [\"manual\", \"json\"]\n x-enumDescriptions:\n manual: The schema editor opens `jsonSchema` in the visual field-by-field view\n json: The schema editor opens `jsonSchema` in the raw JSON view\n description: |-\n Which view a schema editor opens `jsonSchema` in; the schema sent to the provider\n is the same either way. Stored and echoed on guardrails for parity with the\n AI-agent format block — the guardrail form does not set it today.\n x-celigo-ai-guidance:\n - |-\n Resend the stored value on every update — a full PUT that omits the field clears\n it — and never invent one: a value outside the enum fails the save with 422 `enum`\n on `guardrail.aiAgent.litellm.responseFormat.schemaMode`.\n name:\n type: string\n description: Name for the output format (used with json_schema).\n x-celigo-ui-override: >-\n Required by the AI agent form when output format type is json_schema. Encoded\n here to mirror the form so builders produce connectable configurations.\n strict:\n type: boolean\n description: When true, enforces strict schema validation on output.\n jsonSchema:\n type: object\n description: JSON Schema for structured output. Required when `responseFormat.type` is \"json_schema\".\n x-celigo-ui-override: >-\n Required by the AI agent form when output format type is json_schema. Encoded\n here to mirror the form so builders produce connectable configurations.\n properties:\n type:\n type: string\n description: Root JSON Schema type of the structured output; use \"object\" for record-shaped results.\n enum: [\"object\", \"array\", \"string\", \"number\", \"integer\", \"boolean\"]\n x-enumDescriptions:\n object: JSON object with named properties.\n array: Ordered list of values.\n string: Text string value.\n number: Numeric value including decimals.\n integer: Whole number value without decimals.\n boolean: True or false value.\n properties:\n type: object\n additionalProperties: true\n description: JSON Schema definitions for each field the structured output may contain.\n required:\n type: array\n description: Property names the model must include in the structured output.\n items:\n type: string\n additionalProperties:\n type: boolean\n description: When true, the structured output may include properties beyond those defined in `properties`.\n if:\n properties:\n type:\n const: json_schema\n required: [type]\n then:\n required: [name, jsonSchema]\n _overrides:\n type: object\n description: Provider-specific overrides\n properties:\n gemini:\n type: object\n description: |\n Gemini-specific configuration overrides.\n required: [systemInstruction]\n properties:\n systemInstruction:\n type: string\n maxLength: 1000000\n description: |\n System instruction for Gemini models. Equivalent to OpenAI's `instructions`.\n Maximum 1,000,000 characters.\n x-celigo-ui-override: >-\n Required by the AI agent configuration form (the model's system prompt, the\n Gemini equivalent of `instructions`). Encoded here to mirror the form so\n builders produce connectable configurations.\n tools:\n type: array\n deprecated: true\n description: |-\n Not available on guardrails: a non-empty array fails the save with 422\n `guardrail_tools_not_allowed`.\n items:\n type: object\n additionalProperties: true\n prompts:\n type: array\n description: |-\n MCP prompt entries saved with a Gemini guardrail. Each item references one\n MCP connection and prompt names; a guardrail's run never fetches them.\n items:\n type: object\n properties:\n type:\n type: string\n enum: [\"mcp\"]\n x-enumDescriptions:\n mcp: Connect to an MCP server to use its exposed prompts.\n description: Type of prompt entry. Always \"mcp\".\n mcp:\n type: object\n description: MCP server prompt configuration.\n properties:\n _mcpConnectionId:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Connection to the MCP server.\n allowedPrompts:\n type: array\n description: Prompt names saved with the entry; a guardrail's run never fetches them.\n items:\n type: string\n resources:\n $ref: '../schemas/aiagent.yml#/McpResources'\n responseModalities:\n type: array\n description: Response output modalities\n x-celigo-ai-guidance:\n - |-\n `[\"image\"]` requires an image-capable Gemini model (currently `gemini-2.5-flash-image`).\n Selecting it on a text-only model is rejected by the provider — pair image\n output with the right model or omit.\n items:\n type: string\n enum: [\"text\", \"image\"]\n x-enumDescriptions:\n text: Generate text content in the response.\n image: Generate image content in the response.\n topK:\n type: number\n description: Top-K sampling parameter for Gemini\n thinkingConfig:\n type: object\n description: Controls Gemini's extended thinking capabilities\n x-celigo-ai-guidance:\n - |-\n Supported on reasoning-capable Gemini models only (currently `gemini-2.5-pro`\n and `gemini-2.5-flash`).\n Setting `thinkingConfig` on a non-thinking model (e.g.\n `gemini-2.5-flash-lite`,\n `gemini-2.5-flash-image`) is rejected by the provider at runtime — omit the\n field on those models.\n properties:\n includeThoughts:\n type: boolean\n description: When true, includes the model's thinking steps in the response.\n x-celigo-ai-guidance:\n - Whether to include thinking steps in the response\n thinkingBudget:\n type: number\n minimum: 100\n maximum: 4000\n description: Maximum tokens allocated for thinking\n thinkingLevel:\n type: string\n description: Controls how much thinking effort the model applies; use higher levels for complex, multi-step tasks at the cost of latency and tokens.\n enum: [\"minimal\", \"low\", \"medium\", \"high\"]\n x-enumDescriptions:\n minimal: Least thinking effort for simple tasks.\n low: Light thinking for moderately simple tasks.\n medium: Balanced thinking effort for typical tasks.\n high: Maximum thinking effort for complex, multi-step tasks.\n imageConfig:\n type: object\n description: Gemini image generation configuration\n properties:\n aspectRatio:\n type: string\n description: Aspect ratio of generated images; choose a ratio matching the intended display format.\n enum: [\"1:1\", \"2:3\", \"3:2\", \"3:4\", \"4:3\", \"4:5\", \"5:4\", \"9:16\", \"16:9\", \"21:9\"]\n x-enumDescriptions:\n \"1:1\": Square aspect ratio.\n \"2:3\": Portrait aspect ratio (2 wide by 3 tall).\n \"3:2\": Landscape aspect ratio (3 wide by 2 tall).\n \"3:4\": Portrait aspect ratio (3 wide by 4 tall).\n \"4:3\": Standard landscape aspect ratio (4 wide by 3 tall).\n \"4:5\": Portrait aspect ratio (4 wide by 5 tall).\n \"5:4\": Landscape aspect ratio (5 wide by 4 tall).\n \"9:16\": Tall portrait aspect ratio for mobile screens.\n \"16:9\": Widescreen landscape aspect ratio.\n \"21:9\": Ultra-widescreen cinematic aspect ratio.\n imageSize:\n type: string\n description: Output resolution of generated images; higher resolutions increase detail and file size.\n enum: [\"1K\", \"2K\", \"4K\"]\n x-enumDescriptions:\n \"1K\": Standard resolution output around 1024 pixels.\n \"2K\": High resolution output around 2048 pixels.\n \"4K\": Ultra-high resolution output around 4096 pixels.\n mediaResolution:\n type: string\n enum: [\"low\", \"medium\", \"high\"]\n x-enumDescriptions:\n low: Low resolution for faster processing and reduced token usage.\n medium: Balanced resolution for typical media processing.\n high: High resolution for maximum detail in media inputs.\n description: Resolution for media inputs (images, video)\n anthropic:\n type: object\n description: |\n Claude-specific configuration overrides. Used when `provider` is \"anthropic\".\n required: [systemInstruction]\n properties:\n systemInstruction:\n type: string\n maxLength: 1000000\n description: |\n System instruction for Claude models. Equivalent to OpenAI's `instructions`.\n Maximum 1,000,000 characters.\n x-celigo-ui-override: >-\n Required by the AI agent configuration form (the model's system prompt, the\n Claude equivalent of `instructions`). Encoded here to mirror the form so\n builders produce connectable configurations.\n topK:\n type: number\n minimum: 0\n description: |-\n Top-K sampling parameter. Deprecated on Claude models released after Claude Opus\n 4.6, which reject any value at runtime; set it only on older models.\n thinkingConfig:\n type: object\n description: Controls Claude's extended thinking capabilities.\n properties:\n type:\n type: string\n enum: [\"enabled\", \"disabled\", \"adaptive\"]\n x-enumDescriptions:\n enabled: Turn on extended thinking with an explicit token budget.\n disabled: Turn off extended thinking.\n adaptive: Let the model decide how much to think based on task complexity.\n description: Extended thinking mode.\n budgetTokens:\n type: number\n minimum: 1024\n description: |-\n Maximum tokens allocated for thinking. Required when `type` is \"enabled\" —\n omitting it fails the save with 422 `invalid_thinking_config`.\n x-celigo-ui-override: >-\n Required by the AI agent form when thinkingConfig.type is \"enabled\". Encoded\n here to mirror the form so builders produce connectable configurations.\n display:\n type: string\n enum: [\"summarized\", \"omitted\"]\n x-enumDescriptions:\n summarized: Include a summarized view of the model's thinking in the response.\n omitted: Exclude the model's thinking from the response.\n description: How thinking output is surfaced in the response.\n effort:\n type: string\n enum: [\"low\", \"medium\", \"high\", \"xhigh\", \"max\"]\n x-enumDescriptions:\n low: Light thinking effort for moderately simple tasks.\n medium: Balanced thinking effort for typical tasks.\n high: Maximum standard thinking effort for complex tasks.\n xhigh: Extended thinking effort beyond high for demanding tasks.\n max: Highest available thinking effort.\n description: |-\n How much thinking effort the model applies; use with `type` \"adaptive\". Higher\n values (`xhigh`, `max`) may be gated to specific Claude models by the provider.\n if:\n properties:\n type:\n const: enabled\n required: [type]\n then:\n required: [budgetTokens]\n serviceTier:\n type: string\n enum: [\"auto\", \"standard_only\"]\n x-enumDescriptions:\n auto: Let Anthropic select the appropriate service tier.\n standard_only: Restrict processing to the standard service tier.\n description: Anthropic service tier for the request.\n tools:\n type: array\n deprecated: true\n description: |-\n Not available on guardrails: a non-empty array fails the save with 422\n `guardrail_tools_not_allowed`.\n items:\n type: object\n additionalProperties: true\n prompts:\n type: array\n description: |-\n MCP prompt entries saved with a Claude guardrail. Each item references one\n MCP connection and prompt names; a guardrail's run never fetches them.\n items:\n type: object\n properties:\n type:\n type: string\n enum: [\"mcp\"]\n x-enumDescriptions:\n mcp: Connect to an MCP server to use its exposed prompts.\n description: Type of prompt entry. Always \"mcp\".\n mcp:\n type: object\n description: MCP server prompt configuration.\n properties:\n _mcpConnectionId:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Connection to the MCP server.\n allowedPrompts:\n type: array\n description: Prompt names saved with the entry; a guardrail's run never fetches them.\n items:\n type: string\n resources:\n $ref: '../schemas/aiagent.yml#/McpResources'\n # Same provider→block chain as the AI agent import's AiAgentConfig: the instance-level\n # `required` is what forces the provider's block to exist.\n if:\n properties:\n provider:\n const: openai\n required: [provider]\n then:\n required: [openai]\n properties:\n openai:\n required: [model, instructions]\n else:\n if:\n properties:\n provider:\n enum: [gemini, anthropic]\n required: [provider]\n then:\n required: [litellm]\n properties:\n litellm:\n required: [model]\n else:\n if:\n properties:\n provider:\n enum: [mistral, xai, huggingface, deepseek, cohere, groq]\n required: [provider]\n then:\n required: [model]\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":64907,"content_sha256":"1e452865ad24e26122944c90695aea5562b5a446bd5a4ffc73119d82667f5cba"},{"filename":"references/schemas/aiagent.yml","content":"AiAgentConfig:\n type: object\n description: |\n AI agent configuration for an AiAgentImport. An `ai_agent` guardrail is configured by\n the guardrail spec's `GuardrailAiAgentConfig` instead, which has no `tools` arrays.\n\n Configures which AI provider and model to use, along with instructions, parameter\n tuning, output format, and available tools. Providers come in two families with\n two configuration formats:\n\n - **Built-in providers** — each has its own configuration block.\n - **openai**: OpenAI models (GPT-4.1, GPT-5, etc.). Configure via the `openai` object.\n - **gemini**: Google Gemini models via the LiteLLM proxy. Configure via `litellm` with overrides in `litellm._overrides.gemini`.\n - **anthropic**: Anthropic Claude models via the LiteLLM proxy. Configure via `litellm` with overrides in `litellm._overrides.anthropic`.\n - **Catalog providers** — `mistral`, `xai`, `huggingface`, `deepseek`, `cohere`, `groq`. Added\n as catalog data rather than platform code, they share one generic flat format:\n `model`, `modelOptions`, `instructions`, `output`, and the `tools` / `prompts` /\n `resources` arrays directly on this object.\n\n A `_connectionId` on the parent import selects bring-your-own-key (BYOK)\n credentials. Without one, built-in providers run on platform-managed\n credentials; catalog providers have no platform-managed credentials, so the\n save requires one — without it the agent fails to save with 422 on\n `aiAgent.model`: `ai_model_not_available` for a catalog-listed model,\n `ai_model_not_in_catalog` for one the catalog does not list (the model\n catalog check answers before any `connection_required`).\n\n **Model lifecycle.** Every model the platform offers — built-in and catalog\n providers alike — carries a lifecycle in the platform's model catalog:\n `active`, `deprecated` (still usable; a retirement date and suggested\n replacement models may be published), or `retired`, plus an availability\n switch. The save and every run check the configured model against it: a\n model that is not in the catalog, switched off, not offered for the agent's\n credential type, or retired (by status, or once its retirement date has\n passed) is refused — the save with 422 `ai_model_not_in_catalog`,\n `ai_model_disabled`, `ai_model_not_available`, or `ai_model_retired` on the\n model field, the run with a `Run blocked:` error carrying the same code before\n any model call (`Run blocked: model \"gpt-4.1-nano\" is retired on \"Oct 23,\n 2026\". Please choose a supported model and test your agent configuration.` —\n the run-time message names the date only; the chat route streams it as an\n `error` event). A deprecated model saves and runs until its retirement date. The\n platform never substitutes a model: once a model is retired, every save of\n the agent is refused until the model changes. Agents and guardrails on\n affected models are listed by `GET /v1/aiagents/aiModelHealth`. With a\n `_connectionId`, a model the catalog does not list is passed through to the\n vendor as-is; a catalog-listed model remains governed.\n required: [provider]\n properties:\n provider:\n type: string\n enum: [\"openai\", \"gemini\", \"anthropic\", \"mistral\", \"xai\", \"huggingface\", \"deepseek\", \"cohere\", \"groq\"]\n x-enumDescriptions:\n openai: Use OpenAI models via the OpenAI Responses API (built-in; `openai` block).\n gemini: Use Google Gemini models via the LiteLLM proxy (built-in; `litellm` block).\n anthropic: Use Anthropic Claude models via the LiteLLM proxy (built-in; `litellm` block).\n mistral: Use Mistral models (catalog provider; generic flat format, BYOK).\n xai: Use xAI Grok models (catalog provider; generic flat format, BYOK).\n huggingface: Use Hugging Face inference endpoint models (catalog provider; generic flat format, BYOK).\n deepseek: Use DeepSeek models (catalog provider; generic flat format, BYOK).\n cohere: Use Cohere models (catalog provider; generic flat format, BYOK).\n groq: Use Groq-hosted models (catalog provider; generic flat format, BYOK).\n description: AI provider to use.\n x-celigo-ai-guidance:\n - |-\n The provider decides the configuration format, and the server rejects a\n mismatch with 422 `invalid_provider_config_format`: `openai` needs the\n `openai` block, `gemini` and `anthropic` need the `litellm` block (Gemini- or\n Claude-specific settings under `litellm._overrides`), and the catalog\n providers (`mistral`, `xai`, `huggingface`, `deepseek`, `cohere`, `groq`) need the\n generic flat format — `model` and `modelOptions` directly on this object,\n never an `openai` or `litellm` block.\n - |-\n Catalog providers are BYOK-only, and the save enforces it through the model\n catalog: without a `_connectionId` the agent fails to save with 422\n `ai_model_not_available` on `aiAgent.model` (\"Model \\\"grok-4.6\\\" is not\n supported. Please choose a supported model and test your agent\n configuration.\") — the model is a catalog row that is not offered on\n platform-managed credentials. Attach an HTTP connection to the vendor's API\n (token auth) before saving a catalog-provider agent.\n model:\n type: string\n description: |-\n Model identifier for a catalog provider (generic flat format) — for example\n `grok-4.6`. Built-in providers set the model inside their own block\n (`openai.model`, `litellm.model`) instead.\n x-celigo-ai-guidance:\n - |-\n Required for every catalog provider — omitting it or sending an empty string\n fails the save with 422 `missing_required_field`.\n - |-\n Without a `_connectionId` on the parent import the model check answers first:\n a catalog-provider agent fails to save with 422 on `aiAgent.model` —\n `ai_model_not_available` for a catalog-listed model (catalog providers are not\n offered on platform-managed credentials), `ai_model_not_in_catalog` for an id\n the catalog does not list.\n With a `_connectionId` (BYOK) a model the catalog does not list saves as-is —\n the server does not check it against the catalog or the vendor; a\n catalog-listed model is still governed by its lifecycle row (see the\n schema description — a retired or switched-off model is refused on save).\n modelOptions:\n type: object\n description: |-\n Model tuning for a catalog provider (generic flat format). Which keys, values, and\n ranges are accepted is defined per catalog model and validated on save when the\n model is a catalog entry.\n properties:\n maxOutputTokens:\n type: number\n description: Maximum number of tokens the model may generate.\n temperature:\n type: number\n description: Sampling temperature.\n topP:\n type: number\n description: Nucleus sampling threshold.\n reasoning_effort:\n type: string\n enum: [\"low\", \"medium\", \"high\", \"xhigh\"]\n x-enumDescriptions:\n low: Minimal reasoning before answering.\n medium: Moderate reasoning.\n high: Extensive reasoning.\n xhigh: Maximum reasoning.\n description: Reasoning depth for models that expose it.\n x-celigo-ai-guidance:\n - |-\n When the model is a catalog entry, the server validates this object against\n that model's schema and fails the save with 422 `invalid_model_options`\n naming the key: unknown keys are rejected, `reasoning_effort` must be one of\n the enum values where the model accepts it at all (xAI's `grok-4.6` does;\n DeepSeek's `deepseek-chat` rejects the key), and numeric bounds apply —\n on `grok-4.6`, `temperature` ≤ 2, `topP` ≤ 1, `maxOutputTokens` ≥ 100.\n - |-\n With a `_connectionId` and a model the catalog does not list, the object is\n passed through unvalidated — send only options the vendor's API accepts.\n instructions:\n type: string\n description: |-\n System prompt for a catalog provider (generic flat format). Built-in providers\n set it inside their own block (`openai.instructions`,\n `litellm._overrides.anthropic.systemInstruction`).\n output:\n type: object\n description: |-\n Output format for a catalog provider (generic flat format). Built-in providers\n configure it inside their own block (`openai.output`, `litellm.responseFormat`).\n properties:\n format:\n type: object\n description: Controls the structure of the model's output.\n properties:\n type:\n type: string\n enum: [\"text\", \"json_schema\"]\n x-enumDescriptions:\n text: Free-form text.\n json_schema: Structured JSON conforming to `jsonSchema`.\n description: Output type.\n x-celigo-ai-guidance:\n - |-\n `blob` is not accepted here — the save fails with 422\n `unsupported_output_format` naming the model (verified on `grok-4.6`).\n name:\n type: string\n description: Name of the JSON schema, for `json_schema` output.\n strict:\n type: boolean\n description: When true, the model must conform exactly to `jsonSchema`.\n jsonSchema:\n type: object\n additionalProperties: true\n description: JSON Schema the structured output must conform to, for `json_schema` output.\n verbose:\n type: string\n description: Level of detail in the model's response, for models that expose it.\n tools:\n type: array\n description: |-\n Tools available to a catalog-provider agent (generic flat format). Entries mirror\n the built-in providers' tool entries: a Celigo Tool by `_toolId`, an MCP server by\n `_mcpConnectionId`, or a vendor web-search tool.\n items:\n type: object\n properties:\n type:\n type: string\n enum: [\"tool\", \"mcp\", \"web_search\"]\n x-enumDescriptions:\n tool: Reference a reusable Celigo Tool resource.\n mcp: Connect to an MCP server for additional external tools.\n web_search: Search the web for real-time information, where the vendor's model supports it.\n description: Type of tool entry.\n tool:\n type: object\n description: Reference to a Celigo Tool resource, used when type is \"tool\".\n properties:\n _toolId:\n type: string\n format: objectId\n x-celigo-refModel: tools\n description: The Celigo Tool to call.\n mcp:\n type: object\n description: MCP server connection, used when type is \"mcp\".\n properties:\n _mcpConnectionId:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Connection to the MCP server.\n allowedTools:\n type: array\n description: |-\n Specific tools to allow from the MCP server (all if omitted). Each entry\n is either a plain tool name or an object carrying display metadata.\n items:\n type: [string, object]\n config:\n type: object\n additionalProperties: true\n description: Tool-specific options for vendor-native tools (for example web-search settings).\n x-celigo-ai-guidance:\n - |-\n References are checked on save: an unknown `_toolId` or `_mcpConnectionId`\n fails with 422 `invalid_ref` naming the array index.\n prompts:\n type: array\n description: |-\n MCP prompt entries available to a catalog-provider agent (generic flat format).\n Each entry references one MCP connection and the prompt names allowed from it.\n items:\n type: object\n properties:\n type:\n type: string\n enum: [\"mcp\"]\n x-enumDescriptions:\n mcp: Reference prompts exposed by an MCP server.\n description: Type of prompt entry. Always \"mcp\".\n mcp:\n type: object\n properties:\n _mcpConnectionId:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Connection to the MCP server that exposes the prompts.\n allowedPrompts:\n type: array\n description: Prompt names the agent may fetch from the server.\n items:\n type: string\n x-celigo-ai-guidance:\n - |-\n An unknown `_mcpConnectionId` fails the save with 422 `invalid_ref` naming\n the array index.\n resources:\n $ref: '#/McpResources'\n openai:\n type: object\n description: |\n OpenAI-specific configuration. Used when `provider` is \"openai\".\n required: [model, instructions]\n properties:\n instructions:\n type: string\n maxLength: 1000000\n description: |\n System prompt that defines the AI agent's behavior, goals, and constraints.\n examples: [\"You are a data validation agent. Check each record for completeness.\"]\n x-celigo-ui-override: >-\n Required by the AI agent configuration form (the model's system prompt). Encoded\n here to mirror the form so builders produce connectable configurations.\n model:\n type: string\n description: |-\n OpenAI model identifier. Open string (not an enum) — model names change frequently.\n examples: [\"gpt-5.4\", \"gpt-5.4-pro\", \"gpt-5-mini\", \"gpt-5-nano\", \"gpt-4.1-mini\"]\n x-celigo-ui-override: >-\n Required by the AI agent configuration form. Encoded here to mirror the form so\n builders produce connectable configurations.\n x-celigo-ai-guidance:\n - |-\n With platform-managed credentials (no `_connectionId` on the parent\n import), the model must be a catalog row the platform offers on its own\n credentials. The save checks this — a model that is not in the catalog\n fails with 422 `ai_model_not_in_catalog` on `aiAgent.openai.model`\n (\"Model \\\"gpt-4\\\" is not supported. Please choose a supported model and\n test your agent configuration.\"), and a dry-run answers the same\n `errors` with `valid: false`. A retired model is refused the same way with\n `ai_model_retired` — the message names the retirement date and, when any\n remain eligible, the suggested next models (\"Model \\\"gpt-4.1-nano\\\" is\n retired on \\\"Oct 23, 2026\\\" and cannot be used. Suggested next models:\n gpt-5.6-luna. Please test and update your agent configuration.\") — and so\n is a switched-off one with `ai_model_disabled`; a deprecated model saves.\n Never swap the model on the user's behalf — relay the message and let\n them choose.\n - |-\n With a `_connectionId` (BYOK) a model the catalog does not list is passed\n through to the provider as-is; a catalog-listed model is still governed by\n its lifecycle row.\n reasoning:\n type: object\n description: |-\n Controls depth of reasoning for complex tasks.\n x-celigo-ai-guidance:\n - |-\n Supported on reasoning-capable OpenAI models only (currently the GPT-5 family and the o-series).\n Setting `reasoning` on a non-reasoning model (e.g.\n `gpt-4.1-mini`) is rejected by the provider at runtime — omit the field on those models.\n properties:\n effort:\n type: string\n enum: [\"none\", \"minimal\", \"low\", \"medium\", \"high\", \"xhigh\", \"max\"]\n x-enumDescriptions:\n none: No additional reasoning effort.\n minimal: Least reasoning effort for simple, straightforward tasks.\n low: Light reasoning for moderately simple tasks.\n medium: Balanced reasoning effort for typical tasks.\n high: Heavy reasoning effort for complex, multi-step tasks.\n xhigh: Very high reasoning effort for the most demanding, multi-step tasks.\n max: Highest available reasoning effort, above xhigh.\n description: How much reasoning effort the model should invest\n x-celigo-ai-guidance:\n - |-\n How much reasoning effort the model should invest.\n Reasoning- capable models only — see `reasoning` for the gating rule.\n - |-\n Any other value fails the save with 422 `enum` on `aiAgent.openai.reasoning.effort`.\n summary:\n type: string\n enum: [\"concise\", \"auto\", \"detailed\"]\n x-enumDescriptions:\n concise: Brief, high-level reasoning summary.\n auto: Let the model decide the appropriate level of detail.\n detailed: Comprehensive reasoning summary with full explanations.\n description: Level of detail in reasoning summaries\n x-celigo-ai-guidance:\n - |-\n Level of detail in reasoning summaries.\n Reasoning-capable models only — see `reasoning` for the gating rule.\n temperature:\n type: number\n minimum: 0\n maximum: 2\n description: |\n Sampling temperature. Higher values (e.g. 1.5) produce more creative output,\n lower values (e.g. 0.2) produce more focused and deterministic output.\n topP:\n type: number\n minimum: 0.1\n maximum: 1\n description: Nucleus sampling parameter\n topLogprobs:\n type: number\n minimum: 0\n maximum: 20\n description: Number of most likely tokens to return log probabilities for at each output position.\n maxOutputTokens:\n type: number\n minimum: 100\n maximum: 128000\n default: 5000\n description: |-\n Caps the length of the model's response, in tokens. An agent created without it is\n saved with 5000; raise it for long outputs such as large extractions.\n serviceTier:\n type: string\n enum: [\"auto\", \"default\", \"priority\"]\n x-enumDescriptions:\n auto: Let OpenAI automatically select the appropriate service tier.\n default: Standard service tier with normal rate limits and latency.\n priority: Premium tier with higher rate limits and lower latency at increased cost.\n default: \"default\"\n description: |-\n OpenAI service tier. \"priority\" provides higher rate limits and\n lower latency at increased cost.\n x-celigo-ai-guidance:\n - |-\n BYOK only.\n Platform-managed credentials (no `_connectionId` on the parent import) always\n run at `default` regardless of what's set here.\n output:\n type: object\n description: Output format configuration\n properties:\n format:\n type: object\n description: |\n Controls the structure of the model's output.\n properties:\n type:\n type: string\n enum: [\"text\", \"json_schema\", \"blob\"]\n x-enumDescriptions:\n text: Free-form text response from the model.\n json_schema: Structured JSON output conforming to a defined schema.\n blob: Image generation — the run calls the image generation tool and returns the stored image's `blobKey`, not `_text` or `_json`.\n default: \"text\"\n description: Output format type.\n schemaMode:\n type: string\n enum: [\"manual\", \"json\"]\n x-enumDescriptions:\n manual: The UI opens `jsonSchema` in the visual field-by-field editor\n json: The UI opens `jsonSchema` in the raw JSON editor\n description: |-\n Which editor the UI opens `jsonSchema` in; the schema sent to the provider\n is the same either way, and the field is only meaningful when `type` is\n \"json_schema\". Unset opens the visual editor, and an edit there rebuilds\n `jsonSchema` from the field list, keeping only `type`, `properties`,\n `required`, `items`, `description`, `enum`, `pattern`, length and numeric\n bounds, `multipleOf`, `minItems`/`maxItems` and a `\"null\"` type — other\n keywords are dropped without a prompt, and `integer` becomes `number`.\n x-celigo-ai-guidance:\n - |-\n Set `json` on a schema written as raw JSON; keep a user's `manual` and stay\n inside the keywords the visual editor keeps. A full PUT that omits the field\n clears it, and a value outside the enum fails the save with 422 `enum` on\n `aiAgent.openai.output.format.schemaMode`.\n name:\n type: string\n description: Name for the output format (used with json_schema)\n x-celigo-ui-override: >-\n Required by the AI agent form when output format type is json_schema. Encoded\n here to mirror the form so builders produce connectable configurations.\n strict:\n type: boolean\n default: false\n description: When true, enforces strict schema validation on output.\n x-celigo-ai-guidance:\n - Whether to enforce strict schema validation on output\n jsonSchema:\n type: object\n description: |\n JSON Schema for structured output. Required when `format.type` is \"json_schema\".\n x-celigo-ui-override: >-\n Required by the AI agent form when output format type is json_schema. Encoded\n here to mirror the form so builders produce connectable configurations.\n properties:\n type:\n type: string\n description: Root JSON Schema type of the structured output; use \"object\" for record-shaped results.\n enum: [\"object\", \"array\", \"string\", \"number\", \"integer\", \"boolean\"]\n x-enumDescriptions:\n object: JSON object with named properties.\n array: Ordered list of values.\n string: Text string value.\n number: Numeric value including decimals.\n integer: Whole number value without decimals.\n boolean: True or false value.\n properties:\n type: object\n additionalProperties: true\n description: JSON Schema definitions for each field the structured output may contain.\n required:\n type: array\n description: Property names the model must include in the structured output.\n items:\n type: string\n additionalProperties:\n type: boolean\n description: When true, the structured output may include properties beyond those defined in `properties`.\n schema:\n type: object\n readOnly: true\n additionalProperties: true\n description: |-\n Server-written copy of `jsonSchema`, returned whenever `jsonSchema` is set — on\n AI agent imports and on guardrail agents alike. Send `jsonSchema`, not this field.\n if:\n properties:\n type:\n const: json_schema\n required: [type]\n then:\n required: [name, jsonSchema]\n verbose:\n type: string\n enum: [\"low\", \"medium\", \"high\"]\n x-enumDescriptions:\n low: Minimal detail in the model's response.\n medium: Moderate detail in the model's response, the default level.\n high: Maximum detail and verbosity in the model's response.\n default: \"medium\"\n description: Level of detail in the model's response\n x-celigo-ai-guidance:\n - |-\n Supported on the GPT-5 family only. Other OpenAI models (e.g.\n `gpt-4.1`, `gpt-4.1-mini`,\n `gpt-4.1-nano`) accept only `medium` (the no-op default) — `low` and `high` are\n rejected by the provider at runtime.\n Omit the field entirely on non-GPT-5 models rather than relying on the default.\n tools:\n type: array\n description: |\n Tools available to the AI agent during processing.\n items:\n type: object\n properties:\n type:\n type: string\n enum: [\"web_search\", \"mcp\", \"image_generation\", \"code_execution\", \"tool\"]\n x-enumDescriptions:\n web_search: Search the web for real-time information.\n mcp: Connect to an MCP server for additional external tools.\n image_generation: Generate images using an AI image model.\n code_execution: \"Enable OpenAI's built-in code-execution tool; carries no configuration block, send `{type: code_execution}` only\"\n tool: Reference a reusable Celigo Tool resource.\n description: |-\n Type of tool. Every type except `code_execution` is configured through the\n sibling object of the same name; OpenAI spells this one `code_execution`, while\n the Gemini and Anthropic overrides spell theirs `codeExecution` — the other spelling\n fails with 422 `enum`.\n webSearch:\n type: object\n description: Web search configuration (empty object to enable)\n imageGeneration:\n type: object\n description: Image generation configuration\n properties:\n background:\n type: string\n description: Controls whether generated images have a transparent or opaque background; use transparent only with output formats that support it (png, webp).\n enum: [\"transparent\", \"opaque\"]\n x-enumDescriptions:\n transparent: Generate an image with a transparent background.\n opaque: Generate an image with a solid, non-transparent background.\n quality:\n type: string\n description: Rendering quality of generated images, trading detail for generation speed and file size.\n enum: [\"low\", \"medium\", \"high\"]\n x-enumDescriptions:\n low: Lower quality for faster generation and smaller file size.\n medium: Balanced quality and generation speed.\n high: Highest quality output with more detail.\n size:\n type: string\n description: Pixel dimensions of generated images; choose square, portrait, or landscape to match the intended use.\n enum: [\"1024x1024\", \"1024x1536\", \"1536x1024\"]\n x-enumDescriptions:\n 1024x1024: Square image at 1024 by 1024 pixels.\n 1024x1536: Portrait image at 1024 by 1536 pixels.\n 1536x1024: Landscape image at 1536 by 1024 pixels.\n outputFormat:\n type: string\n description: File format of generated images; use png or webp when transparency is needed.\n enum: [\"png\", \"webp\", \"jpeg\"]\n x-enumDescriptions:\n png: PNG format with lossless compression, supports transparency.\n webp: WebP format with efficient compression for web use.\n jpeg: JPEG format with lossy compression for smaller file sizes.\n mcp:\n type: object\n description: MCP server tool configuration\n properties:\n _mcpConnectionId:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Connection to the MCP server\n allowedTools:\n type: array\n description: |-\n Specific tools to allow from the MCP server (all if\n omitted). Each entry is either a plain tool name (legacy\n form) or an object carrying display metadata.\n items:\n type: [string, object]\n properties:\n name:\n type: string\n maxLength: 256\n description: Tool name as exposed by the MCP server.\n title:\n type: string\n maxLength: 300\n description: Display title shown for the tool.\n description:\n type: string\n maxLength: 1000\n description: Display description shown for the tool.\n required: [name]\n x-celigo-ai-guidance:\n - |-\n Object entries accept only name/title/description — any\n other key fails validation with \"unknown field(s)\", and\n an object without a non-empty name fails with a\n missing-required-field error. String entries are the\n legacy form and stay valid.\n allowedPrompts:\n type: array\n deprecated: true\n description: Not stored on a tool entry — a save drops it. Prompt names go in a `prompts[]` entry's own `mcp.allowedPrompts`.\n items:\n type: string\n tool:\n type: object\n description: |\n Reference to a Celigo Tool resource.\n properties:\n _toolId:\n type: string\n format: objectId\n x-celigo-refModel: tools\n description: Reference to the Tool resource\n overrides:\n type: object\n description: Per-agent overrides for the tool's internal resources\n properties:\n connections:\n type: array\n description: |\n Remaps the tool's abstract connections for this agent. Each entry pairs\n the tool's abstract connection placeholder (`_abstractId`) with the\n concrete connection (`_id`) to use for this agent; entries without\n `_id` keep the tool's own default connection.\n items:\n type: [object, 'null']\n required: [_abstractId]\n properties:\n _abstractId:\n type: string\n format: objectId\n description: The tool's abstract connection placeholder being overridden.\n _id:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Concrete connection to use in place of the abstract placeholder.\n prompts:\n type: array\n description: |\n MCP prompt entries available to the agent. Each item references one MCP connection\n and the prompt names allowed from it. Configured alongside `tools` in the form but\n stored separately; an entry's `allowedPrompts` is what distinguishes a prompt entry\n from an MCP tool entry (which carries `allowedTools`).\n items:\n type: object\n properties:\n type:\n type: string\n enum: [\"mcp\"]\n x-enumDescriptions:\n mcp: Connect to an MCP server to use its exposed prompts.\n description: Type of prompt entry. Always \"mcp\".\n mcp:\n type: object\n description: MCP server prompt configuration.\n properties:\n _mcpConnectionId:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Connection to the MCP server.\n allowedPrompts:\n type: array\n description: Prompt names to allow from the MCP server.\n items:\n type: string\n resources:\n $ref: '#/McpResources'\n litellm:\n type: object\n description: |\n LiteLLM proxy configuration. Used when `provider` is \"gemini\" or \"anthropic\".\n\n LiteLLM provides a unified interface to multiple AI providers. Gemini-specific\n settings are in `_overrides.gemini`; Claude-specific settings are in\n `_overrides.anthropic`.\n\n `model` is required when litellm is the active provider path.\n properties:\n model:\n type: string\n description: |-\n LiteLLM model identifier. For Gemini, models are stored without the `gemini/`\n prefix; for Anthropic, use the Claude model id (e.g. `claude-sonnet-4-6`).\n examples: [\"gemini-2.5-flash\", \"gemini-2.5-pro\", \"gemini-2.5-flash-lite\", \"claude-sonnet-4-6\", \"claude-opus-4-7\"]\n x-celigo-ui-override: >-\n Required by the AI agent configuration form. Encoded here to mirror the form so\n builders produce connectable configurations.\n x-celigo-ai-guidance:\n - |-\n With platform-managed credentials (no `_connectionId` on the parent\n import), the model must be a catalog row the platform offers for the\n provider on its own credentials — Gemini and Anthropic Claude each have\n their own rows. The save checks this: a model that is not in the catalog\n fails with 422 `ai_model_not_in_catalog` on `aiAgent.litellm.model`, a\n retired one with `ai_model_retired` (the message names the retirement\n date and, when any remain eligible, the suggested next models), a\n switched-off one with `ai_model_disabled`; a deprecated model saves. Never\n swap the model on the user's behalf — relay the message and let them choose.\n - |-\n With a `_connectionId` (BYOK) a model the catalog does not list is passed\n through to the provider as-is; a catalog-listed model is still governed by\n its lifecycle row.\n temperature:\n type: number\n minimum: 0\n maximum: 2\n description: Sampling temperature\n x-celigo-ai-guidance:\n - |-\n When `provider` is \"anthropic\", the value must be between 0 and 1 — a higher value\n fails the save with 422 `invalid_field_value`. Gemini accepts the full 0-2 range.\n maxCompletionTokens:\n type: number\n minimum: 100\n maximum: 128000\n default: 5000\n description: Maximum number of tokens in the response\n topP:\n type: number\n minimum: 0.1\n maximum: 1\n description: Nucleus sampling parameter\n seed:\n type: number\n description: Random seed for reproducible outputs\n responseFormat:\n type: object\n description: Output format configuration\n x-celigo-ai-guidance:\n - |-\n When `provider` is \"anthropic\", `type` must be \"text\" or \"json_schema\" — \"blob\" is\n rejected with 422 `invalid_response_format`. As with the other providers, \"json_schema\"\n requires `name` and `jsonSchema`.\n properties:\n type:\n type: string\n description: Output format type.\n enum: [\"text\", \"json_schema\", \"blob\"]\n x-enumDescriptions:\n text: Free-form text response from the model.\n json_schema: Structured JSON output conforming to a defined schema.\n blob: Image generation — the run returns the stored image's `blobKey`, not `_text` or `_json`.\n default: \"text\"\n schemaMode:\n type: string\n enum: [\"manual\", \"json\"]\n x-enumDescriptions:\n manual: The UI opens `jsonSchema` in the visual field-by-field editor\n json: The UI opens `jsonSchema` in the raw JSON editor\n description: |-\n Which editor the UI opens `jsonSchema` in; the schema sent to the provider is\n the same either way, and the field is only meaningful when `type` is\n \"json_schema\". Unset opens the visual editor, and an edit there rebuilds\n `jsonSchema` from the field list, keeping only `type`, `properties`, `required`,\n `items`, `description`, `enum`, `pattern`, length and numeric bounds,\n `multipleOf`, `minItems`/`maxItems` and a `\"null\"` type — other keywords are\n dropped without a prompt, and `integer` becomes `number`.\n x-celigo-ai-guidance:\n - |-\n Set `json` on a schema written as raw JSON; keep a user's `manual` and stay inside\n the keywords the visual editor keeps. A full PUT that omits the field clears it,\n and a value outside the enum fails the save with 422 `enum` on\n `aiAgent.litellm.responseFormat.schemaMode`.\n name:\n type: string\n description: Name for the output format (used with json_schema).\n x-celigo-ui-override: >-\n Required by the AI agent form when output format type is json_schema. Encoded\n here to mirror the form so builders produce connectable configurations.\n strict:\n type: boolean\n description: When true, enforces strict schema validation on output.\n default: false\n jsonSchema:\n type: object\n description: JSON Schema for structured output. Required when `responseFormat.type` is \"json_schema\".\n x-celigo-ui-override: >-\n Required by the AI agent form when output format type is json_schema. Encoded\n here to mirror the form so builders produce connectable configurations.\n properties:\n type:\n type: string\n description: Root JSON Schema type of the structured output; use \"object\" for record-shaped results.\n enum: [\"object\", \"array\", \"string\", \"number\", \"integer\", \"boolean\"]\n x-enumDescriptions:\n object: JSON object with named properties.\n array: Ordered list of values.\n string: Text string value.\n number: Numeric value including decimals.\n integer: Whole number value without decimals.\n boolean: True or false value.\n properties:\n type: object\n additionalProperties: true\n description: JSON Schema definitions for each field the structured output may contain.\n required:\n type: array\n description: Property names the model must include in the structured output.\n items:\n type: string\n additionalProperties:\n type: boolean\n description: When true, the structured output may include properties beyond those defined in `properties`.\n if:\n properties:\n type:\n const: json_schema\n required: [type]\n then:\n required: [name, jsonSchema]\n _overrides:\n type: object\n description: Provider-specific overrides\n properties:\n gemini:\n type: object\n description: |\n Gemini-specific configuration overrides.\n required: [systemInstruction]\n properties:\n systemInstruction:\n type: string\n maxLength: 1000000\n description: |\n System instruction for Gemini models. Equivalent to OpenAI's `instructions`.\n Maximum 1,000,000 characters.\n x-celigo-ui-override: >-\n Required by the AI agent configuration form (the model's system prompt, the\n Gemini equivalent of `instructions`). Encoded here to mirror the form so\n builders produce connectable configurations.\n tools:\n type: array\n description: Gemini-specific tools\n items:\n type: object\n properties:\n type:\n type: string\n enum: [\"googleSearch\", \"urlContext\", \"fileSearch\", \"codeExecution\", \"mcp\", \"tool\"]\n x-enumDescriptions:\n googleSearch: Use Google Search for real-time information grounding.\n urlContext: Retrieve and use content from specified URLs.\n fileSearch: Search through previously uploaded files.\n codeExecution: \"Enable Gemini's built-in code-execution tool; carries no configuration block, send `{type: codeExecution}` only\"\n mcp: Connect to an MCP server for additional external tools.\n tool: Reference a reusable Celigo Tool resource.\n description: |-\n Type of Gemini tool. `codeExecution` is the Gemini spelling — `code_execution`\n fails with 422 `enum`.\n googleSearch:\n type: object\n description: Google Search configuration (empty object to enable)\n urlContext:\n type: object\n description: URL context configuration (empty object to enable)\n fileSearch:\n type: object\n description: File search configuration, used when type is \"fileSearch\".\n properties:\n fileSearchStoreNames:\n type: array\n description: Names of the file search stores the model can query.\n items:\n type: string\n mcp:\n type: object\n description: MCP server tool configuration, used when type is \"mcp\".\n properties:\n _mcpConnectionId:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Connection to the MCP server.\n allowedTools:\n type: array\n description: |-\n Specific tools to allow from the MCP server (all if\n omitted). Each entry is either a plain tool name\n (legacy form) or an object carrying display metadata\n — same contract as the OpenAI `allowedTools`.\n items:\n type: [string, object]\n properties:\n name:\n type: string\n maxLength: 256\n description: Tool name as exposed by the MCP server.\n title:\n type: string\n maxLength: 300\n description: Display title shown for the tool.\n description:\n type: string\n maxLength: 1000\n description: Display description shown for the tool.\n required: [name]\n allowedPrompts:\n type: array\n deprecated: true\n description: Not stored on a tool entry — a save drops it. Prompt names go in a `prompts[]` entry's own `mcp.allowedPrompts`.\n items:\n type: string\n tool:\n type: object\n description: Reference to a Celigo Tool resource, used when type is \"tool\".\n properties:\n _toolId:\n type: string\n format: objectId\n x-celigo-refModel: tools\n description: Reference to the Tool resource.\n overrides:\n type: object\n description: Per-agent overrides for the tool's internal resources.\n properties:\n connections:\n type: array\n description: |\n Remaps the tool's abstract connections for this agent. Each entry\n pairs the tool's abstract connection placeholder (`_abstractId`)\n with the concrete connection (`_id`) to use for this agent;\n entries without `_id` keep the tool's own default connection.\n items:\n type: [object, 'null']\n required: [_abstractId]\n properties:\n _abstractId:\n type: string\n format: objectId\n description: The tool's abstract connection placeholder being overridden.\n _id:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Concrete connection to use in place of the abstract placeholder.\n prompts:\n type: array\n description: |\n MCP prompt entries available to the Gemini agent. Each item references one\n MCP connection and the prompt names allowed from it. The presence of\n `allowedPrompts` distinguishes a prompt entry from an MCP tool entry.\n items:\n type: object\n properties:\n type:\n type: string\n enum: [\"mcp\"]\n x-enumDescriptions:\n mcp: Connect to an MCP server to use its exposed prompts.\n description: Type of prompt entry. Always \"mcp\".\n mcp:\n type: object\n description: MCP server prompt configuration.\n properties:\n _mcpConnectionId:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Connection to the MCP server.\n allowedPrompts:\n type: array\n description: Prompt names to allow from the MCP server.\n items:\n type: string\n resources:\n $ref: '#/McpResources'\n responseModalities:\n type: array\n description: Response output modalities\n x-celigo-ai-guidance:\n - |-\n `[\"image\"]` requires an image-capable Gemini model (currently `gemini-2.5-flash-image`).\n Selecting it on a text-only model is rejected by the provider — pair image\n output with the right model or omit.\n items:\n type: string\n enum: [\"text\", \"image\"]\n x-enumDescriptions:\n text: Generate text content in the response.\n image: Generate image content in the response.\n default: [\"text\"]\n topK:\n type: number\n description: Top-K sampling parameter for Gemini\n thinkingConfig:\n type: object\n description: Controls Gemini's extended thinking capabilities\n x-celigo-ai-guidance:\n - |-\n Supported on reasoning-capable Gemini models only (currently `gemini-2.5-pro`\n and `gemini-2.5-flash`).\n Setting `thinkingConfig` on a non-thinking model (e.g.\n `gemini-2.5-flash-lite`,\n `gemini-2.5-flash-image`) is rejected by the provider at runtime — omit the\n field on those models.\n properties:\n includeThoughts:\n type: boolean\n description: When true, includes the model's thinking steps in the response.\n x-celigo-ai-guidance:\n - Whether to include thinking steps in the response\n thinkingBudget:\n type: number\n minimum: 100\n maximum: 4000\n description: Maximum tokens allocated for thinking\n thinkingLevel:\n type: string\n description: Controls how much thinking effort the model applies; use higher levels for complex, multi-step tasks at the cost of latency and tokens.\n enum: [\"minimal\", \"low\", \"medium\", \"high\"]\n x-enumDescriptions:\n minimal: Least thinking effort for simple tasks.\n low: Light thinking for moderately simple tasks.\n medium: Balanced thinking effort for typical tasks.\n high: Maximum thinking effort for complex, multi-step tasks.\n imageConfig:\n type: object\n description: Gemini image generation configuration\n properties:\n aspectRatio:\n type: string\n description: Aspect ratio of generated images; choose a ratio matching the intended display format.\n enum: [\"1:1\", \"2:3\", \"3:2\", \"3:4\", \"4:3\", \"4:5\", \"5:4\", \"9:16\", \"16:9\", \"21:9\"]\n x-enumDescriptions:\n \"1:1\": Square aspect ratio.\n \"2:3\": Portrait aspect ratio (2 wide by 3 tall).\n \"3:2\": Landscape aspect ratio (3 wide by 2 tall).\n \"3:4\": Portrait aspect ratio (3 wide by 4 tall).\n \"4:3\": Standard landscape aspect ratio (4 wide by 3 tall).\n \"4:5\": Portrait aspect ratio (4 wide by 5 tall).\n \"5:4\": Landscape aspect ratio (5 wide by 4 tall).\n \"9:16\": Tall portrait aspect ratio for mobile screens.\n \"16:9\": Widescreen landscape aspect ratio.\n \"21:9\": Ultra-widescreen cinematic aspect ratio.\n imageSize:\n type: string\n description: Output resolution of generated images; higher resolutions increase detail and file size.\n enum: [\"1K\", \"2K\", \"4K\"]\n x-enumDescriptions:\n \"1K\": Standard resolution output around 1024 pixels.\n \"2K\": High resolution output around 2048 pixels.\n \"4K\": Ultra-high resolution output around 4096 pixels.\n mediaResolution:\n type: string\n enum: [\"low\", \"medium\", \"high\"]\n x-enumDescriptions:\n low: Low resolution for faster processing and reduced token usage.\n medium: Balanced resolution for typical media processing.\n high: High resolution for maximum detail in media inputs.\n description: Resolution for media inputs (images, video)\n anthropic:\n type: object\n description: |\n Claude-specific configuration overrides. Used when `provider` is \"anthropic\".\n required: [systemInstruction]\n properties:\n systemInstruction:\n type: string\n maxLength: 1000000\n description: |\n System instruction for Claude models. Equivalent to OpenAI's `instructions`.\n Maximum 1,000,000 characters.\n x-celigo-ui-override: >-\n Required by the AI agent configuration form (the model's system prompt, the\n Claude equivalent of `instructions`). Encoded here to mirror the form so\n builders produce connectable configurations.\n topK:\n type: number\n minimum: 0\n description: |-\n Top-K sampling parameter. Deprecated on Claude models released after Claude Opus\n 4.6, which reject any value at runtime; set it only on older models.\n thinkingConfig:\n type: object\n description: Controls Claude's extended thinking capabilities.\n properties:\n type:\n type: string\n enum: [\"enabled\", \"disabled\", \"adaptive\"]\n x-enumDescriptions:\n enabled: Turn on extended thinking with an explicit token budget.\n disabled: Turn off extended thinking.\n adaptive: Let the model decide how much to think based on task complexity.\n default: \"disabled\"\n description: Extended thinking mode.\n budgetTokens:\n type: number\n minimum: 1024\n description: |-\n Maximum tokens allocated for thinking. Required when `type` is \"enabled\" —\n omitting it fails the save with 422 `invalid_thinking_config`.\n x-celigo-ui-override: >-\n Required by the AI agent form when thinkingConfig.type is \"enabled\". Encoded\n here to mirror the form so builders produce connectable configurations.\n display:\n type: string\n enum: [\"summarized\", \"omitted\"]\n x-enumDescriptions:\n summarized: Include a summarized view of the model's thinking in the response.\n omitted: Exclude the model's thinking from the response.\n default: \"summarized\"\n description: How thinking output is surfaced in the response.\n effort:\n type: string\n enum: [\"low\", \"medium\", \"high\", \"xhigh\", \"max\"]\n x-enumDescriptions:\n low: Light thinking effort for moderately simple tasks.\n medium: Balanced thinking effort for typical tasks.\n high: Maximum standard thinking effort for complex tasks.\n xhigh: Extended thinking effort beyond high for demanding tasks.\n max: Highest available thinking effort.\n description: |-\n How much thinking effort the model applies; use with `type` \"adaptive\". Higher\n values (`xhigh`, `max`) may be gated to specific Claude models by the provider.\n if:\n properties:\n type:\n const: enabled\n required: [type]\n then:\n required: [budgetTokens]\n serviceTier:\n type: string\n enum: [\"auto\", \"standard_only\"]\n x-enumDescriptions:\n auto: Let Anthropic select the appropriate service tier.\n standard_only: Restrict processing to the standard service tier.\n default: \"auto\"\n description: Anthropic service tier for the request.\n tools:\n type: array\n description: Claude-specific tools.\n items:\n type: object\n properties:\n type:\n type: string\n enum: [\"tool\", \"mcp\", \"webSearch\", \"codeExecution\"]\n x-enumDescriptions:\n tool: Reference a reusable Celigo Tool resource.\n mcp: Connect to an MCP server for additional external tools.\n webSearch: Search the web for real-time information.\n codeExecution: \"Enable Anthropic's built-in code-execution tool; carries no configuration block, send `{type: codeExecution}` only\"\n description: |-\n Type of Claude tool. `codeExecution` is the Anthropic spelling — `code_execution`\n fails with 422 `enum`.\n tool:\n type: object\n description: Reference to a Celigo Tool resource, used when type is \"tool\".\n properties:\n _toolId:\n type: string\n format: objectId\n x-celigo-refModel: tools\n description: Reference to the Tool resource.\n overrides:\n type: object\n description: Per-agent overrides for the tool's internal resources.\n properties:\n connections:\n type: array\n description: |\n Remaps the tool's abstract connections for this agent. Each entry\n pairs the tool's abstract connection placeholder (`_abstractId`)\n with the concrete connection (`_id`) to use for this agent;\n entries without `_id` keep the tool's own default connection.\n items:\n type: [object, 'null']\n required: [_abstractId]\n properties:\n _abstractId:\n type: string\n format: objectId\n description: The tool's abstract connection placeholder being overridden.\n _id:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Concrete connection to use in place of the abstract placeholder.\n mcp:\n type: object\n description: MCP server tool configuration, used when type is \"mcp\".\n properties:\n _mcpConnectionId:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Connection to the MCP server.\n allowedTools:\n type: array\n description: |-\n Specific tools to allow from the MCP server (all if omitted). Each\n entry is either a plain tool name or an object carrying display\n metadata — same contract as the OpenAI and Gemini `allowedTools`.\n items:\n oneOf:\n - title: name\n type: string\n description: Tool name as exposed by the MCP server.\n - title: tool\n type: object\n required: [name]\n properties:\n name:\n type: string\n maxLength: 256\n description: Tool name as exposed by the MCP server.\n title:\n type: string\n maxLength: 300\n description: Display title shown for the tool.\n description:\n type: string\n maxLength: 1000\n description: Display description shown for the tool.\n x-celigo-ai-guidance:\n - |-\n Object entries accept only name/title/description — any other key\n fails the save with 422 `invalid_field_value` (\"unknown field(s): …\"),\n and an object without `name` fails with 422 `missing_required_field`\n (\"Expected field: name to be present.\").\n examples: [[\"hub_repo_search\", {\"name\": \"hub_repo_info\", \"title\": \"Repository info\", \"description\": \"Look up a repository by name\"}]]\n webSearch:\n type: object\n description: Web search configuration, used when type is \"webSearch\".\n x-celigo-ai-guidance:\n - |-\n `allowedDomains` and `blockedDomains` are mutually exclusive — set at most\n one. Sending both fails with 422 `invalid_hosted_tool`.\n - |-\n When `userLocation` is present, set at least one of `city`, `country`,\n `region`, or `timezone`; an empty `userLocation` fails with 422\n `missing_required_field`.\n properties:\n version:\n type: string\n pattern: \"^\\\\d{8}$\"\n description: |-\n Anthropic web search tool version (YYYYMMDD). Selects the tool version sent\n on the wire; unsupported versions surface as an Anthropic 400.\n examples: [\"20250305\"]\n allowedDomains:\n type: array\n description: Domains the search may return results from. Mutually exclusive with `blockedDomains`.\n items:\n type: string\n blockedDomains:\n type: array\n description: Domains to exclude from search results. Mutually exclusive with `allowedDomains`.\n items:\n type: string\n userLocation:\n type: object\n description: |-\n Approximate user location used to localize search results. When present, at\n least one of `city`, `country`, `region`, or `timezone` must be set.\n properties:\n type:\n type: string\n enum: [\"approximate\"]\n x-enumDescriptions:\n approximate: Localize results to an approximate user location.\n description: Location type. Always \"approximate\".\n city:\n type: string\n maxLength: 256\n description: City name for localizing search results.\n country:\n type: string\n maxLength: 8\n description: ISO 3166-1 alpha-2 country code for localizing search results.\n region:\n type: string\n maxLength: 256\n description: Region or state for localizing search results.\n timezone:\n type: string\n maxLength: 64\n description: IANA timezone for localizing search results.\n prompts:\n type: array\n description: |\n MCP prompt entries available to the Claude agent. Each item references one MCP\n connection and the prompt names allowed from it. The presence of `allowedPrompts`\n distinguishes a prompt entry from an MCP tool entry.\n items:\n type: object\n properties:\n type:\n type: string\n enum: [\"mcp\"]\n x-enumDescriptions:\n mcp: Connect to an MCP server to use its exposed prompts.\n description: Type of prompt entry. Always \"mcp\".\n mcp:\n type: object\n description: MCP server prompt configuration.\n properties:\n _mcpConnectionId:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Connection to the MCP server.\n allowedPrompts:\n type: array\n description: Prompt names to allow from the MCP server.\n items:\n type: string\n resources:\n $ref: '#/McpResources'\n # Instance-level `required: [openai|litellm|model]` is what actually forces the\n # provider's format to exist — nesting requireds under `properties` alone does not (a\n # bare `{provider: openai}` would validate). The server 422s\n # (\"`model` is required for AI Agent import\") when the block — or, for a catalog\n # provider, the top-level `model` — is missing. `model` is server-required;\n # `instructions` is a form-mirror (server accepts its omission — see the\n # x-celigo-ui-override on that field).\n if:\n properties:\n provider:\n const: openai\n required: [provider]\n then:\n required: [openai]\n properties:\n openai:\n required: [model, instructions]\n else:\n if:\n properties:\n provider:\n enum: [gemini, anthropic]\n required: [provider]\n then:\n required: [litellm]\n properties:\n litellm:\n required: [model]\n else:\n if:\n properties:\n provider:\n enum: [mistral, xai, huggingface, deepseek, cohere, groq]\n required: [provider]\n then:\n required: [model]\n\nMcpResources:\n type: array\n description: |-\n Governed MCP resources — read-only reference content (policies, schemas, documentation)\n pulled from connected MCP servers and made available to the agent as a consistent source\n of truth. Each entry references one MCP connection and the specific resources allowed from it.\n items:\n type: object\n required: [type, mcp]\n properties:\n type:\n type: string\n enum: [\"mcp\"]\n x-enumDescriptions:\n mcp: Reference read-only resources exposed by an MCP server.\n description: Type of resource entry. Always \"mcp\".\n mcp:\n type: object\n required: [_mcpConnectionId, allowedResources]\n description: MCP server resource configuration.\n properties:\n _mcpConnectionId:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Connection to the MCP server that exposes the resources.\n allowedResources:\n type: array\n minItems: 1\n description: |-\n Resources to expose to the agent from the MCP server. Must contain at least one\n entry; each entry identifies one resource by name and URI.\n items:\n type: object\n required: [name, uri]\n properties:\n name:\n type: string\n description: Display name of the MCP resource.\n examples: [\"Data Retention Policy\"]\n uri:\n type: string\n description: URI that identifies the resource on the MCP server.\n examples: [\"resource://policies/data-retention\"]\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":72955,"content_sha256":"3c54c942d2dba35deb3c36017fee57ff80db8b3358980b9ffcae1ce759f229c0"},{"filename":"references/schemas/as2.yml","content":"As2:\n type: object\n description: |-\n Configures how outbound AS2 messages are built for this import. Required when the\n _connectionId field references an AS2 connection. AS2 (Applicability Statement 2) transmits\n EDI and other data securely over HTTP/S using S/MIME encryption and digital signatures; this\n object names the payload file, the message identifier, retry behavior, and any extra headers.\n required:\n - fileNameTemplate\n properties:\n _tpConnectorId:\n type: string\n format: objectId\n x-celigo-refModel: tradingpartnerconnectors\n description: |-\n Trading Partner Connector that supplies the partner-specific EDI and AS2 configuration.\n When set, the import inherits the connector's pre-configured settings; omit to use only\n the AS2 connection details.\n examples: [\"60a2c4e6f321d800129a1a3c\"]\n fileNameTemplate:\n type: string\n x-celigo-agent:\n - handlebars\n x-celigo-ui-override: >-\n Required by the AS2 import form (fieldDefinitions/resources/imports/as2.js:\n as2.fileNameTemplate is required:true, default 'file-{{timestamp}}'). Encoded to mirror\n the form so builders produce connectable configurations.\n description: |-\n Handlebars template that names the file carrying each AS2 message payload. Include a\n uniqueness token such as {{timestamp}} so concurrent or repeated sends do not collide;\n defaults to file-{{timestamp}} in the form.\n examples: [\"file-{{timestamp}}\", \"orders-{{timestamp}}.edi\", \"{{record.documentNumber}}.edi\"]\n messageIdTemplate:\n type: string\n x-celigo-agent:\n - handlebars\n description: |-\n Handlebars template that generates the Message-ID header for each AS2 message. Placeholders\n are replaced at runtime; the resulting value must be globally unique and follow RFC 5322\n header formatting. Leave unset to let the platform generate the Message-ID.\n examples: [\"{{timestamp}}-{{record.id}}@celigo.com\", \"\u003c{{timestamp}}@example.com>\"]\n maxRetries:\n type: number\n default: 0\n description: |-\n Number of times a failed transmission is retried after transient errors such as network\n failures or timeouts; does not affect the initial send. The form offers 1–5; set to 0\n (the default) to report failure immediately without retrying.\n examples: [0, 3, 5]\n headers:\n type: array\n items:\n type: object\n properties:\n name:\n type: string\n description: Name of the HTTP header sent with the AS2 message (for example, `content-disposition`). Header names are case-insensitive.\n value:\n type: string\n description: Value sent for the header. Supports handlebars expressions rendered against the record.\n x-celigo-agent:\n - handlebars\n description: |-\n HTTP headers sent with the AS2 message transmission.\n Supports both standard AS2 headers and custom headers required by trading partner agreements.\n examples:\n - [{ \"name\": \"content-disposition\", \"value\": \"attachment; filename=order-{{record.id}}.edi\" }]\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":3221,"content_sha256":"81f266b7406a913018335384ce211f70b229ecc5775bd731942672a8aed086ad"},{"filename":"references/schemas/base.yml","content":"ImportBase:\n type: object\n description: Writable import fields shared by the request and response schemas.\n properties:\n _connectionId:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: |-\n Connection this import uses to reach the destination system. The connection's type must\n be compatible with the import's `adaptorType` (e.g. an `HTTPImport` needs an `http`\n connection; `NetSuiteDistributedImport` and `NetSuiteHTTPImport` both need a `netsuite`\n connection). Server-required — POST without it fails with 422 \"Expected field:\n _connectionId to be present\" — except for the connection-less flavors (`ToolImport`,\n `AiAgentImport`, `GuardrailImport`, `TopicImport`), which the server creates without\n one. An Event Streams publisher (`TopicImport`) must omit it: with a connection set the\n server validates the document as that connection's adaptor instead.\n examples: [\"60a2c4e6f321d800129a1a3c\"]\n _integrationId:\n type: [string, 'null']\n format: objectId\n x-celigo-refModel: integrations\n description: |-\n Not stored on an import: a value sent on create or on a full update is dropped, and a GET\n doesn't return the field. An import isn't owned by an integration; to find the flows and\n other resources that use it, call `GET /v1/imports/{_id}/dependencies`.\n _connectorId:\n type: string\n format: objectId\n x-celigo-refModel: connectors\n description: |-\n Connector this import was created from, set when the import is part of an installed\n integration app. The app's install sets it: a create that sends it is refused with 403\n (\"Cannot create an Integration App resource\").\n examples: [\"60a2c4e6f321d800129a1a3c\"]\n adaptorType:\n type: string\n description: |-\n Selects the adaptor technology that executes this import, which determines the compatible\n connection types and which adaptor-specific configuration object must also be supplied\n (e.g. set `salesforce` when using `SalesforceImport`).\n x-celigo-ai-guidance:\n - |-\n Use HTTPImport for all REST/SOAP APIs unless a specialized adaptor exists for the\n target system; always pair the adaptorType with its matching configuration object.\n - |-\n For NetSuite, use NetSuiteDistributedImport (with the netsuite_da object) when the\n Celigo SuiteApp or bundle is installed in the account — check with\n `GET /v1/connections/{_id}/distributedApps` (`suiteapp.success` / `bundle.success`).\n Use NetSuiteHTTPImport (with `nsDomainType` and the http object) when neither is\n installed or the user asks for NetSuite's REST APIs; the platform's SuiteTalk connector\n catalogs `POST /services/rest/record/v1/\u003ctype>` for inserts and\n `PATCH /services/rest/record/v1/\u003ctype>/{id}` for updates, and saving links the step to\n them. NetSuiteImport is a legacy adaptor for accounts without the Celigo SuiteApp 2.0 —\n do not use it unless explicitly requested.\n - |-\n Use RDBMSImport (with the rdbms object) for built-in database types such as SQL Server,\n MySQL, PostgreSQL, Snowflake, Oracle, and MariaDB. Use JDBCImport (with the jdbc\n object) only for generic JDBC connections that are not built-in RDBMS types.\n enum:\n - HTTPImport\n - FTPImport\n - AS2Import\n - S3Import\n - NetSuiteImport\n - NetSuiteDistributedImport\n - NetSuiteHTTPImport\n - SalesforceImport\n - JDBCImport\n - RDBMSImport\n - MongodbImport\n - DynamodbImport\n - WrapperImport\n - AiAgentImport\n - GuardrailImport\n - FileSystemImport\n - ToolImport\n - TopicImport\n - RESTImport\n x-enumDescriptions:\n HTTPImport: Generic REST or SOAP API import via HTTP connections.\n FTPImport: File import via FTP or SFTP connections.\n AS2Import: File import via AS2 (Applicability Statement 2) protocol.\n S3Import: File import to Amazon S3 storage.\n NetSuiteImport: Legacy NetSuite import for accounts without the Celigo SuiteApp 2.0.\n NetSuiteDistributedImport: NetSuite import using the distributed SuiteApp 2.0 adaptor (default for NetSuite when the SuiteApp or bundle is installed).\n NetSuiteHTTPImport: NetSuite import over its REST APIs — SuiteTalk REST record endpoints or a custom RESTlet — on a netsuite connection, without the Celigo SuiteApp or bundle.\n SalesforceImport: Salesforce-specific import using SOAP, REST, Metadata, or Composite APIs.\n JDBCImport: Database import via a generic JDBC connection (not for built-in RDBMS types).\n RDBMSImport: Database import for built-in RDBMS types such as SQL Server, MySQL, PostgreSQL, Snowflake, and Oracle.\n MongodbImport: MongoDB database import for document-based operations.\n DynamodbImport: Amazon DynamoDB import for NoSQL table operations.\n WrapperImport: Custom adaptor import built outside of Celigo (rarely used).\n AiAgentImport: AI agent import using OpenAI, Gemini, Anthropic Claude, or a BYOK catalog provider (Mistral, xAI, Hugging Face, DeepSeek, Cohere, Groq) for data processing.\n GuardrailImport: Safety and compliance check import for PII detection, moderation, or AI evaluation.\n FileSystemImport: Local or network file system import.\n ToolImport: Import that executes a reusable Celigo Tool resource.\n TopicImport: Publishes each record as a message to an Event Streams topic, configured through `topic`; needs no connection.\n RESTImport: Legacy REST adaptor, superseded by HTTPImport — still accepted and present on older imports.\n examples: [\"HTTPImport\", \"NetSuiteDistributedImport\", \"NetSuiteHTTPImport\", \"SalesforceImport\", \"FTPImport\", \"AiAgentImport\", \"GuardrailImport\", \"TopicImport\"]\n nsDomainType:\n type: string\n enum:\n - suitetalk\n - restlet\n x-lowercase: true\n x-enumDescriptions:\n suitetalk: NetSuite SuiteTalk REST web services — record endpoints under /services/rest/record/v1\n restlet: A custom RESTlet script deployed in the NetSuite account, called at /app/site/hosting/restlet.nl?script=\u003cid>&deploy=\u003cn>\n description: |-\n Selects which NetSuite REST host a `NetSuiteHTTPImport` calls. The server derives the\n full base URL from the connection's NetSuite account\n (`https://\u003caccount>.suitetalk.api.netsuite.com` or `https://\u003caccount>.restlets.api.netsuite.com`)\n and signs each request with the connection's token-based credentials, so each\n `http.relativeURI` entry carries the complete path starting at `/services/rest/...` or\n `/app/site/hosting/restlet.nl`. Ignored on other adaptor types.\n x-celigo-ai-guidance:\n - |-\n Required on `NetSuiteHTTPImport`: omitting it fails the save with 422 \"Path `nsDomainType`\n is required.\" The import's `_connectionId` must be a netsuite connection — an http\n connection fails with 422 invalid_connection_type.\n - |-\n With `suitetalk`, saving links `http._httpConnectorVersionId`, `_httpConnectorResourceId`\n and `_httpConnectorEndpointIds` to the platform's SuiteTalk connector from each\n `http.method` + `http.relativeURI` pair (for example `POST /services/rest/record/v1/customer`\n links to that record type's insert endpoint), so the UI's Simple view renders an\n API-authored step; `restlet` steps are not linked.\n externalId:\n type: [string, 'null']\n description: External identifier for correlating the import with a record in another system.\n examples: [\"shopify-customer-import\"]\n as2:\n $ref: './as2.yml#/As2'\n dynamodb:\n $ref: './dynamodb.yml#/Dynamodb'\n http:\n $ref: './http.yml#/Http'\n ftp:\n $ref: './ftp.yml#/Ftp'\n jdbc:\n $ref: './jdbc.yml#/Jdbc'\n mongodb:\n $ref: './mongodb.yml#/Mongodb'\n netsuite:\n $ref: './netsuite.yml#/NetSuite'\n netsuite_da:\n $ref: './netsuitedistributed.yml#/NetsuiteDistributed'\n rdbms:\n $ref: './rdbms.yml#/Rdbms'\n s3:\n $ref: './s3.yml#/S3'\n wrapper:\n $ref: './wrapper.yml#/Wrapper'\n salesforce:\n $ref: './salesforce.yml#/Salesforce'\n tool:\n $ref: './tool.yml#/Tool'\n topic:\n $ref: './topic.yml#/Topic'\n file:\n $ref: './file.yml#/File'\n filesystem:\n $ref: './filesystem.yml#/FileSystem'\n aiAgent:\n $ref: './aiagent.yml#/AiAgentConfig'\n guardrail:\n $ref: './guardrail.yml#/GuardrailConfig'\n name:\n type: string\n minLength: 1\n maxLength: 100\n description: |-\n Display name for the import, shown in the flow builder, job history, and error logs.\n Descriptive, unique names indicating the destination system and purpose make large\n accounts easier to manage.\n examples: [\"Shopify Customer Import\", \"NetSuite Sales Order Import\"]\n description:\n type: [string, 'null']\n description: |-\n Free-text summary of what the import writes and why. Shown in the UI and available to\n AI agents for context; has no effect on execution.\n maxLength: 5120\n unencrypted:\n type: object\n description: Custom configuration values stored without encryption and returned in API responses.\n sampleData:\n type: [object, array, string]\n description: Sample input record used to preview and build the import's mappings.\n distributed:\n type: boolean\n description: |-\n When true, the import uses a distributed adaptor (such as `NetSuiteDistributedImport`)\n that executes inside the target application rather than on Celigo's servers.\n maxAttempts:\n type: number\n description: Maximum number of attempts made to deliver a record before it is marked as failed.\n ignoreExisting:\n type: boolean\n description: |-\n When true, records that already exist in the destination system are silently skipped\n instead of being created or updated — used for create-only operations that must avoid\n duplicates. Existing records are identified by the import's lookup configuration or by\n a populated `ignoreExtract` field on the incoming record.\n x-celigo-ai-guidance:\n - |-\n Set true for create/insert-only operations when the user asks to skip, ignore,\n or avoid duplicating existing records.\n Leave false for update, sync, or upsert operations.\n Requires a configured lookup (or ignoreExtract) to identify existing records.\n - |-\n ## EXAMPLES:\n\n ### Should set ignoreExisting: true\n - \"Create customer records in Shopify while ignoring existing customers\" → ignoreExisting: true\n - \"Create vendor records while ignoring existing vendors\" → ignoreExisting: true\n - \"Import new products only, skip existing ones\" → ignoreExisting: true\n - \"Add orders to the system, ignore any that already exist\" → ignoreExisting: true\n - \"Create accounts but don't update existing ones\" → ignoreExisting: true\n - \"Insert new contacts, skip duplicates\" → ignoreExisting: true\n - \"Create records, skipping any that match existing data\" → ignoreExisting: true\n\n ### Should NOT set ignoreExisting: true\n - \"Update existing customer records\" → ignoreExisting: false (update operation)\n - \"Create or update records\" → ignoreExisting: false (upsert operation)\n - \"Sync all records\" → ignoreExisting: false (sync/upsert operation)\n - |-\n ## WHEN TO SET THIS TO TRUE (MUST DETECT THESE PATTERNS):\n\n **ALWAYS set ignoreExisting to true** when the user's prompt includes ANY of these patterns:\n\n ### Primary Patterns (very common):\n - \"ignore existing\" / \"ignoring existing\" / \"ignore any existing\"\n - \"skip existing\" / \"skipping existing\" / \"skip any existing\"\n - \"while ignoring existing\" / \"while skipping existing\"\n - \"skip duplicates\" / \"avoid duplicates\" / \"ignore duplicates\"\n\n ### Secondary Patterns:\n - \"only create new\" / \"only add new\" / \"create new only\"\n - \"don't update existing\" / \"avoid updating existing\" / \"without updating\"\n - \"create if doesn't exist\" / \"add if doesn't exist\"\n - \"insert only\" / \"create only\"\n - \"skip if already exists\" / \"ignore if exists\"\n - \"don't process existing\" / \"bypass existing\"\n\n ### Context Clues:\n - User says \"create\" or \"insert\" AND mentions checking/matching against existing data\n - User wants to add records but NOT modify what's already there\n - User is concerned about duplicate prevention during creation\n - |-\n ## WHEN TO LEAVE THIS FALSE/UNDEFINED:\n\n Leave this false or undefined when:\n - The prompt doesn't mention skipping or ignoring existing records\n - The import is only performing updates (no inserts)\n - The prompt says \"sync\", \"update\", or \"upsert\"\n - |-\n ## IMPORTANT:\n - This field is typically used with INSERT operations\n - Common pattern: \"Create X while ignoring existing X\" → MUST set to true\n - |-\n ## IMPORTANT NOTES:\n - This flag requires a properly configured lookup to identify existing records, or have an ignoreExtract field configured to identify the field that is used to determine if the record already exists.\n - The lookup typically checks a unique identifier field (id, email, external_id, etc.)\n - When true, existing records are silently skipped (not updated, not errored)\n - Use this for insert-only or create-only operations\n - Do NOT use this if the user wants to update existing records\n - |-\n ## NOT FOR COMPOSITE UPSERTS\n\n ``ignoreExisting`` is for single-operation imports (CREATE-only)\n that want to **skip** records that already exist. It is NOT\n the field for composite upsert imports (those with both CREATE\n and UPDATE in ``requestType``) — those route records between two\n write paths instead of skipping, and use\n ``http.existingExtract`` / ``http.existingLookupName`` as their\n decider field. ``ignoreExisting`` and ``existingExtract`` /\n ``existingLookupName`` are mutually exclusive — never set both\n on the same import.\n - |-\n ## ADAPTOR-LEVEL FIELDS YOU PAIR WITH THIS\n\n Once ``ignoreExisting`` is enabled you must also set the field\n that identifies the existence-check key. Two patterns,\n mutually exclusive — pick exactly one based on whether the\n incoming record already carries the destination id:\n\n - ``http.ignoreExtract`` (or the adaptor's equivalent) — a\n field name on the incoming record. Zero per-record HTTP\n overhead. Use when an upstream step or source-system field\n already populates the id on each record.\n - ``http.ignoreLookupName`` (or the adaptor's equivalent) —\n the name of a lookup in ``http.lookups`` that probes the\n destination per record. Adds one HTTP request per record.\n Use when the record has no field to inspect.\n ignoreMissing:\n type: boolean\n description: |-\n When true, records that do not already exist in the destination system are silently\n skipped instead of producing errors — used for update-only operations. Existing records\n are identified the same way as for `ignoreExisting`: the adaptor's ignore lookup\n (`ignoreLookupName`) or a populated `ignoreExtract` field on the incoming record.\n idLockTemplate:\n type: [string, 'null']\n description: |-\n Handlebars template that generates a concurrency lock key for each record: records that\n render the same key are never submitted at the same time, while records with different\n keys are processed in parallel. When omitted, the record's trace key is the lock key (a\n blank trace key means no lock; a custom `traceKeyTemplate` upstream arrives on the record\n as `io_custom_trace_key`); a template replaces that default rather than adding to it.\n The template is rendered from the record as it arrives at the import — before the import's\n own lookups and mapping run — so a key built from one of the import's lookup results\n (`{{lookup.\u003cname>}}`) or from a mapped field renders empty, and an empty key is no lock at\n all; a field added upstream by a lookup step's response mapping is on the record and works.\n x-celigo-ai-guidance:\n - |-\n The lock is scoped to this import resource across every page and every concurrent run\n that submits through it — including other flows sharing the import — not to one page,\n and not to the connection: two different imports rendering the same key on the same\n connection still run in parallel. Records within a single page are\n sent one at a time regardless, so the lock matters when same-key records sit in different\n pages or runs.\n - |-\n With `ignoreExisting` and an ignore lookup, the existence lookup runs inside the lock:\n lookup-then-write for one key does not interleave with another record's lookup-then-write\n for the same key. That pairing is the duplicate-prevention pattern for destinations that\n can be queried but do not reject duplicates — key the lock on the business identifier the\n write is keyed on. With the check but no lock, concurrent same-key pages all read a miss\n before any write lands and all create. Connection `concurrencyLevel: 1` also closes that\n race for the import (measured on a single import: its pages ran one at a time and the\n same-key pages created once) — the lock is the per-key tool that leaves unrelated records\n parallel.\n - |-\n Key it on a field the record carries when it reaches this import (`{{{record.externalId}}}-GR`,\n or a field an upstream lookup step added through its response mapping), never on one of\n this import's own lookup results; the key must render non-empty for the records that\n matter, or those records have no lock.\n examples: [\"{{record.customerId}}\", \"{{{record.externalId}}}-GR\"]\n dataURITemplate:\n type: [string, 'null']\n x-celigo-agent:\n - handlebars\n description: |-\n Handlebars template that builds a link back to each record in the destination\n application's UI. The resolved URL is stored with error records in job history so users\n can jump straight to the record.\n examples: [\"https://admin.shopify.com/customers/{{record.id}}\"]\n oneToMany:\n $ref: '../common-schemas/one-to-many.yml#/OneToMany'\n pathToMany:\n $ref: '../common-schemas/path-to-many.yml#/PathToMany'\n blobKeyPath:\n type: string\n x-celigo-excludeAdaptorTypes:\n - AiAgentImport\n - GuardrailImport\n - ToolImport\n description: |-\n Path in the input record that holds the blob key identifying the file content to import.\n At send time the platform follows this path, retrieves the referenced file from integrator.io\n storage, and streams its bytes into the outgoing request. Not accepted on `AiAgentImport`,\n `GuardrailImport`, or `ToolImport` — those adaptors process records only and reject the\n field with 422 `not_supported`.\n x-celigo-ai-guidance:\n - |-\n On AiAgentImport, GuardrailImport and ToolImport the save fails with 422 `not_supported`\n (\"Field \"blobKeyPath\" is not supported for this import type.\"); route file content\n through a blob-capable import (HTTP, FTP, S3) instead.\n - |-\n Optional on a `blob: true` file-provider import: when omitted the platform reads the\n record's root `blobKey` — the field a blob export emits on its `{blobKey, fileMeta}`\n record — so a file-transfer flow needs no `blobKeyPath`. Observed on an FTP import\n (recorded runs writing files with no `blobKeyPath`) and on a Google Drive folder\n upload (`http.type: file`, the marketplace file-transfer template's shape); other\n file providers were not probed. Set it only when a response mapping placed the key\n elsewhere; on those two providers its absence is not a cause of upload failures.\n - |-\n The value at this path must be a blobKey produced by an upstream blob\n export or blob lookup and carried onto the record via that step's\n response mapping (idiom: extract ``data.0.blobKey`` → generate\n ``blobKey``). A record field holding base64 file *content* is not a\n blobKey — inline content is record data, not a storage reference, so\n it is typically staged into blob storage first (blob steps support a\n base64 ``blobFormat``) or sent as ordinary record data instead.\n - |-\n For multipart/form-data uploads, pair this field with an ``http.body``\n parts array whose attachment part has ``\"value\": \"{{blob}}\"`` — the\n platform substitutes the file bytes at that part. (APIs expecting\n ``multipart/related`` instead — e.g. Google Drive\n ``uploadType=multipart`` — take a hand-authored MIME body that\n references ``{{blob}}`` directly; see the ``http.body`` guidance.)\n The assembled multipart body is not available in audit logs, so\n verify delivery from job results or the destination side rather\n than log output.\n examples: [\"attachment.blobKey\"]\n blob:\n type: boolean\n x-celigo-excludeAdaptorTypes:\n - AiAgentImport\n - GuardrailImport\n - ToolImport\n description: |-\n When true, this import transfers raw file content (blobs) to the destination rather than\n structured records, reading each file through `blobKeyPath` (optional on FTP and Google\n Drive file-provider imports, which fall back to the record's root `blobKey`). Not accepted on\n `AiAgentImport`, `GuardrailImport`, or `ToolImport` — those adaptors process records only\n and reject the flag with 422 `not_supported`.\n x-celigo-ai-guidance:\n - |-\n On AiAgentImport, GuardrailImport and ToolImport the save fails with 422 `not_supported`\n (\"Field \"blob\" is not supported for this import type.\"); never set it on those adaptors.\n examples: [true, false]\n assistant:\n type: string\n description: |-\n Identifier for the connector assistant used to configure this import. For the cloud\n file providers it also selects the provider-specific location rules —\n `azuredatalakestoragegen2` (container in `file.directory.name`, file path split into\n `http.relativeURI` and `file.fileName`) and `googlecloudstorage` (bucket in\n `file.directory.storageRootId`).\n examples: [\"shopify\", \"azuredatalakestoragegen2\", \"googlecloudstorage\"]\n deleteAfterImport:\n type: boolean\n description: When true, the source file is deleted after it is successfully imported.\n assistantMetadata:\n type: object\n description: Metadata associated with the connector assistant configuration.\n useTechAdaptorForm:\n type: boolean\n description: |-\n When true, the UI presents the full technical adaptor form for this import instead of\n the simplified assistant form.\n distributedAdaptorData:\n type: object\n description: Internal state stored by distributed adaptors (such as the NetSuite SuiteApp) for this import.\n filter:\n x-celigo-agent:\n - filter\n description: |-\n Filter applied to incoming records before they are sent to the destination system.\n Records that match continue through the import; records that don't skip this import,\n counted as ignored, and continue to the next step. Filter expressions reference the\n incoming record's fields.\n allOf:\n - $ref: '../common-schemas/filter.yml#/Filter'\n x-celigo-ai-guidance:\n - |-\n ## IMPORT-SPECIFIC BEHAVIOR\n\n **Pre-Import Filtering**: Filters are applied to incoming records before they are sent to the destination system. A record that doesn't match is not sent, but it is not removed from the flow: it continues to the next step without this import's response-mapped fields, so a later step that writes those fields needs its own filter.\n traceKeyTemplate:\n type: string\n x-celigo-agent:\n - handlebars\n description: |-\n Handlebars template that overrides how each record's unique trace key is generated,\n used to track records through the flow and match errors to records. Trace keys are\n capped at 256 characters; a longer key is stored truncated from the middle, keeping\n the beginning and end of the value. The trace key is also the import's default\n concurrency lock key when `idLockTemplate` is not set; it never de-duplicates — a record\n that arrives twice with the same key is processed twice.\n x-celigo-ai-guidance:\n - |-\n ## CRITICAL: ONLY VALID WHEN ``oneToMany: true``\n On import resources, the platform REJECTS this field unless\n ``oneToMany`` is also ``true``. Validation error code:\n ``[cannot_set_traceKeyTemplate] The traceKeyTemplate field on\n the import document can only be set in case of oneToMany.``\n Default behavior (one-to-one imports) does NOT need a trace\n key template -- omit the field entirely. Only emit it when\n the import is genuinely fan-out (one source record produces\n many destination calls) and you are explicitly setting\n ``oneToMany: true`` in the same payload.\n examples: [\"{{record.orderNumber}}\"]\n mockResponse:\n type: array\n description: |-\n Predefined response records used in place of calling the destination system when\n testing the import, so flows can run without writing real data. Records must be in\n integrator.io canonical format — the server rejects any other shape with a 422.\n x-celigo-ai-guidance:\n - |-\n Flow test runs and flow-builder previews NEVER execute imports —\n they substitute this mockResponse, or the platform placeholder\n ``{id: \"1234567890\", mockResponse: \"replace me\"}`` when none is\n saved. With the placeholder, downstream response mappings extract\n nothing real and downstream steps' sample panels go empty — the\n fix for \"no data reaches the next step\" in the builder is saving\n a realistic mockResponse here, not editing downstream mappings.\n - |-\n ``POST /v1/imports/preview`` is the exception: it executes the\n composed destination request for real — mockResponse does not\n protect that surface.\n items:\n type: object\n properties:\n statusCode:\n type: integer\n description: HTTP status code the mock returns (e.g. 200, 403).\n id:\n type: [string, integer]\n description: Identifier echoed for the mocked record.\n ignored:\n type: boolean\n description: When true, the mocked record reports as ignored rather than imported.\n dataURI:\n type: string\n description: Data URI echoed for the mocked record.\n errors:\n type: array\n description: Mock error entries returned with the record.\n _json:\n type: [object, array]\n description: Mock response body.\n _headers:\n type: object\n description: Mock response headers.\n _ediProfileId:\n type: string\n format: objectId\n x-celigo-refModel: ediprofiles\n description: |-\n EDI profile this import uses to generate outbound X12 or EDIFACT documents — it supplies\n the envelope qualifiers, delimiters, version, and validation rules. Set it when the\n import produces EDI output; omit it otherwise. Accepted on the file-writing imports —\n FTP, AS2, S3 and HTTP file mode (`http.type: \"file\"`).\n x-celigo-ai-guidance:\n - >-\n On an HTTP import set `http.type: \"file\"` (the folder mode of file-provider connectors)\n and pair this field with `file.type: \"filedefinition\"` and `file.skipAggregation: true`\n or `file.aggregation.type: \"key\"` — the platform saves EDI files per page or per key,\n never aggregated per run.\n parsers:\n type: array\n description: |-\n Legacy parser configuration slot. The server initializes this field to an empty array\n and current API writes never populate it; treat it as server bookkeeping rather than a\n setting to configure.\n hooks:\n type: object\n description: |-\n Custom JavaScript hooks that run at fixed points in the import lifecycle for\n transformations, validation, and custom result handling beyond what configuration alone\n can express. A hook name not listed here (an export's `preSavePage`, for example) is\n dropped on save without an error, so read `hooks` back after saving.\n x-celigo-ai-guidance:\n - |-\n JavaScript hook references attached to lifecycle stages of the import.\n Engage hooks ONLY when the user explicitly wants scripting OR a requirement\n cannot be expressed through declarative configuration (mappings / filters /\n responseTransform).\n Hooks bypass the platform's declarative validation, so use them sparingly.\n properties:\n preMap:\n type: object\n x-celigo-agent:\n - script\n description: |-\n Hook that runs on each page of records before the import's mappings are applied.\n Commonly used to reshape or filter records ahead of mapping. Its changes shape only\n the request this import sends: the next step receives the record as it arrived here,\n plus whatever this step's response mapping merges back. A value later steps need\n belongs in the source export's `preSavePage`, in this step's `postResponseMap` hook on\n the flow, or in the later import's own `preMap`.\n\n The function returns one element per record in `options.data`, in order:\n `{ data: \u003crecord> }` sends the record and `{ errors: [{ code, message }] }` fails it.\n An element without `data` — `{}`, or the record itself returned unwrapped — skips that\n record with no error: nothing is sent and the record is reported `ignored: true` at\n stage `preMapHook`. A return with more or fewer elements fails every record on the page\n with `invalid_extension_response` (\"Extension result doesn't contain the same number\n of elements as the request object. Expected 2, got 1. Extension returned statusCode\n 200\"), source `pre_map_hook`.\n x-celigo-ai-guidance:\n - |-\n Pre-map JavaScript hook — runs once per page BEFORE the\n mapping step, on the raw upstream records. Engage when the\n user needs to reshape, validate, filter, or annotate records\n before the mapping sees them (drop archived records,\n normalize date strings). Don't\n engage for static value substitution or simple field\n renames — mappings handle those.\n\n Runtime contract: ``options.data[]`` (the page of upstream\n records) → array of the same length, each element\n ``{data: record}`` or ``{errors: [...]}``. A bare record in\n the returned array is skipped without an error, so\n ``options.data.map(r => { r.total = t; return r })`` sends\n nothing.\n - |-\n Verified live 2026-10-22 by invoking an HTTP import with a\n Mapper 2.0 number mapping: one ``{data: record}`` per record\n sent both records; the bare records sent nothing and both\n came back ``ignored: true``, ``stage: preMapHook``, with no\n error; one element for two records failed both with 422\n ``invalid_extension_response``.\n allOf:\n - $ref: '../common-schemas/hooks.yml#/Hook'\n postMap:\n type: object\n x-celigo-agent:\n - script\n description: |-\n Hook that runs after the import's mappings are applied but before records are sent\n to the destination system. Commonly used for adjustments that need the mapped\n (destination-shaped) record. Like `preMap`, its changes shape only the request this\n import sends; the next step never sees them.\n x-celigo-ai-guidance:\n - |-\n Post-map JavaScript hook — runs once per page AFTER the\n mapping step but BEFORE the destination call. Engage when\n the user needs to adjust the mapped output's shape in ways\n the mapping schema can't express (XML/SOAP envelope\n wrapping, conditional payload composition, collapsing\n multiple mapped records into one batch payload). Engage\n rarely — declarative mappings cover most cases.\n\n Runtime contract: ``options.preMapData[]`` (the page as it\n entered the mapping step) + ``options.postMapData[]`` (the\n mapping output) → array same length as ``postMapData``.\n allOf:\n - $ref: '../common-schemas/hooks.yml#/Hook'\n postSubmit:\n type: object\n x-celigo-agent:\n - script\n description: |-\n Hook that runs after the destination system responds, with access to both the\n submitted records and the destination's responses. Commonly used to inspect results\n or adjust the response data passed downstream. Each `options.responseData` entry is\n one submitted record's result, in order: `statusCode`, `errors` (`{code, message,\n source}`, empty on success), `ignored`, `id`, and `_json` — the destination's parsed\n response body, where its own error detail sits (`_headers` holds the response\n headers).\n x-celigo-ai-guidance:\n - |-\n Post-submit JavaScript hook — runs AFTER each batch of\n destination writes, on the destination's response.\n\n Runtime contract: ``options.responseData[]`` (one per\n submitted record; ``{statusCode, errors, _json, ...}`` —\n ``id`` is the destination record's on real runs; flow test\n runs substitute the saved ``mockResponse`` or the placeholder\n response, whose envelope carries the mock's id, such as\n ``1234567890``) → array of the same length, typically mutated\n to annotate or augment per-record errors.\n allOf:\n - $ref: '../common-schemas/hooks.yml#/Hook'\n postAggregate:\n type: object\n x-celigo-agent:\n - script\n description: |-\n Hook that runs after an aggregated file has been submitted to the destination, for\n file-based imports that combine records into a single file.\n x-celigo-ai-guidance:\n - |-\n Post-aggregate JavaScript hook — runs ONCE after a\n file-aggregating import (FTP/S3-style) submits its\n aggregated file. Only legal on those imports: the API\n rejects it on HTTP records imports with a 422\n ``invalid_hook`` (\"unsupported postAggregate hook\") —\n never emit it for non-file adaptors.\n\n Runtime contract: ``options.postAggregateData``\n (``{success, _json, code, message, source}``) → no return\n value; side effects only (logging, notifying, downstream\n triggers).\n allOf:\n - $ref: '../common-schemas/hooks.yml#/Hook'\n sampleResponseData:\n type: [object, array, string]\n description: |-\n Sample destination response kept with the import for authoring its response mapping.\n Flow test runs and previews don't read it; they substitute `mockResponse`, or a\n placeholder when none is saved.\n responseTransform:\n x-celigo-agent:\n - transform\n description: |-\n Transformation that reshapes the destination system's response after records are\n imported: it runs after the postSubmit hook, on each record's response body (`_json`),\n and its output replaces that body, so response mapping can't read a field the\n transform drops. Expression rules use version `\"1\"` (`rules`); version `\"2\"` is refused\n on save. Commonly used to extract relevant fields from verbose API responses.\n allOf:\n - $ref: '../common-schemas/transform.yml#/Transform'\n - type: object\n properties:\n expression:\n type: object\n properties:\n version:\n type: string\n enum: [\"1\"]\n x-enumDescriptions:\n \"1\": Transformation 1.0 rules in `rules`, the only version an import's response transform accepts.\n description: Must be `\"1\"` on an import's response transform; `\"2\"` is refused with 422.\n x-celigo-ai-guidance:\n - |-\n Verified live 2026-10-21 on an HTTP import in a flow run: the postSubmit hook's changes\n to `responseData[i]._json` were in the transform's input, the transform's output replaced\n `_json` (only the mapped fields were kept), and response mapping then found nothing at a\n dropped field's path. The execution-log stage is `responseTransformation`, and\n `POST /v1/imports/{_id}/invoke` returns the transformed body.\n - |-\n A save with `expression.version: \"2\"` (`rulesTwoDotZero`) fails with 422 \"`2` is not a\n valid enum value for path `responseTransform.expression.version`.\" Write `version: \"1\"`\n rules (`[[{extract, generate}, ...]]`).\n - |-\n ## IMPORT-SPECIFIC BEHAVIOR\n\n **Response Transformation**: Transforms the response data returned by the destination system after records have been imported. This allows you to reshape, filter, or enrich the response before response mapping reads it; the record passed downstream is the incoming record plus what response mapping copies out.\n\n **Common Use Cases**:\n - Extracting relevant fields from verbose API responses\n - Normalizing response formats across different destination systems\n - Adding computed fields based on response data\n - Filtering out sensitive information before logging or further processing\n - |-\n ## Transformation 1.0 rules only, and what response mapping reads\n\n Expression rules here are Transformation 1.0 only (`expression.version: \"1\"`, the\n `rules` array) or a script: a version \"2\" expression is refused with 422\n (`` `2` is not a valid enum value for path `responseTransform.expression.version`. ``).\n The rules read the parsed response body, and their output replaces `_json`. v1 rules\n keep only the fields they generate, so the step's response mapping reads those\n names (at the envelope root or under `_json`) and nothing the rules dropped; `id`\n still resolves.\n modelMetadata:\n type: object\n description: Metadata about the destination data model captured for this import. Rarely set.\n mapping:\n type: object\n x-celigo-agent:\n - mapping\n description: |-\n Mapper 1.0 mapping configuration. `fields` maps individual fields; `lists` maps\n record lists, each with its own `fields` array. Superseded by Mapper 2.0\n (`mappings`) but still widely used.\n x-celigo-ai-guidance:\n - |-\n ## Adaptor routing\n\n REQUIRED for ``SalesforceImport`` and ``WrapperImport``: these\n adaptors must use Mapper 1.0; Mapper 2.0 is not supported for\n them. ``NetSuiteDistributedImport`` is also V1-required\n but uses a NetSuite-specific variant at its adaptor-scoped copy\n ``netsuite_da.mapping`` (richer sublist / subRecord vocabulary) —\n not this field.\n\n Optional on other import adaptors (``HTTPImport``, ``RDBMSImport``,\n ``FTPImport``, ``JDBCImport``, etc.) when the user explicitly asks\n for Mapper 1.0 — e.g. when the import wraps a legacy v1 connector\n or is being migrated from an older integration. Those adaptors\n default to Mapper 2.0 at the top-level ``mappings`` (plural) field.\n Do not set this field unsolicited; the v2 default produces better\n output for modern connectors.\n\n An import that carries both this field and ``mappings`` runs only\n ``mappings``: the Mapper 1.0 rows are ignored, with no error. A\n conversion to Mapper 2.0 therefore translates every row and removes\n this field in the same save.\n - |-\n Allowed (but not the default) on other import adaptors such as\n ``HTTPImport``, ``RDBMSImport``, ``FTPImport``, and\n ``JDBCImport`` when the user explicitly asks for Mapper 1.0 —\n for example when wrapping a legacy v1 connector or migrating\n from an older integration. Those adaptors default to Mapper 2.0\n at ``mappings.yml#/Mappings``; emit this shape only when V1 is\n intended.\n\n Shape is a flat ``{fields, lists}`` pair: ``fields`` maps body-level\n destination columns; ``lists`` maps source arrays into destination\n line-item / sublist arrays. Unlike Mapper 2.0 there is no recursive\n ``buildArrayHelper`` — arrays are expressed only through ``lists``.\n\n Lookups referenced by ``lookupName`` live on the top-level\n ``resource.lookups`` array (mirrored under the adaptor key for\n adaptors that maintain their own copy; on an HTTP import that\n copy, ``http.lookups``, is the one a save keeps — see the mirror\n invariant on ``lookups``). Lookups are independent of this\n mapping: they are neither created nor cleared automatically when\n this mapping changes.\n properties:\n fields:\n type: array\n description: Field-level mapping entries applied to each record.\n items:\n $ref: '#/MappingField'\n x-celigo-ai-guidance:\n - |-\n Body-level field mappings. Each entry writes one destination\n column from one source expression. Order has no semantic\n meaning; entries are applied independently per record.\n lists:\n type: array\n description: List-level mappings, each generating a sublist on the target record.\n items:\n type: object\n properties:\n generate:\n type: string\n description: Target sublist or array path to generate.\n x-celigo-ai-guidance:\n - |-\n Destination array / line-item id (e.g. Salesforce child\n relationship name, HTTP request sublist key, NetSuite\n sublist name like ``item`` or ``addressbook``).\n\n This identifies the array itself — do NOT add bracket\n notation here. Per-element semantics belong on the inner\n ``fields[].extract`` paths (which path-walk from the source\n record root and use ``[*]`` to drive iteration). Dot\n notation IS allowed here when the destination places the\n array inside a parent object (e.g. ``order.lines``).\n fields:\n type: array\n description: Field mappings applied within each generated list item.\n items:\n $ref: '#/MappingField'\n x-celigo-ai-guidance:\n - |-\n Per-element field mappings applied to each item in the extracted\n array. Uses the same shape as top-level ``fields`` — same\n extract / generate / lookupName / conditional semantics.\n x-celigo-ai-guidance:\n - |-\n One source-array → destination-array projection. The destination is\n identified by ``generate`` (e.g. a line-item collection name on the\n target record). Each element of the source array is walked through\n ``fields`` to build the corresponding destination element.\n - |-\n Line-item / sublist mappings. Each entry projects an array from\n the source record into a destination array, applying its own\n ``fields`` sub-mapping to each element.\n mappings:\n x-celigo-agent:\n - mapping\n description: |-\n Field mappings that transform incoming records into the destination system's field\n structure — renaming fields, converting types, building nested objects, and applying\n formulas or lookups to derive values. The mapped output builds only the request this\n import sends: the next step receives the record as it arrived here, plus whatever this\n step's response mapping merges back.\n x-celigo-ai-guidance:\n - |-\n ## Extract evaluation root\n\n ``mappings[].extract`` JSONPath values are interpreted against\n the root selected by the sibling ``inputContext`` field on this\n import — ``record`` mode (the default) evaluates JSONPath\n against the flat record itself (``\"$.field\"``), and ``envelope``\n mode evaluates against the wrapped runtime envelope\n (``\"$.record.field\"``). See ``inputContext`` for the full\n per-mode contract. Handlebars expressions in ``extract`` always\n reference ``{{record.field}}`` regardless — only JSONPath\n behavior flips with ``inputContext``.\n - |-\n ## JSON structure — record { } or rows [ ]\n\n The shape of this array is the mapper's JSON structure. Named\n top-level items produce an object record; ``http.body`` and the\n other templates bind it as ``record``. Exactly one top-level item\n with ``dataType: objectarray`` and no ``generate`` is the\n ``rows [ ]`` structure: the mapped record is an array — one\n element per element the item's ``buildArrayHelper`` iterates —\n and templates bind it as ``rows`` (``{{#each rows}}``), with\n ``record.*`` undefined. The mapping editor shows that row's\n destination disabled; its save check accepts the unnamed root\n only in the ``rows [ ]`` structure and fails any other row that\n has a source and no ``generate`` with \"One or more destination\n field values not entered\". Keep the root unnamed when editing a\n ``rows [ ]`` mapping — naming it turns the output into an object\n and a ``{{#each rows}}`` body renders nothing.\n - |-\n ## IMPORT-SPECIFIC BEHAVIOR\n\n **Data Transformation**: Mappings define how fields from incoming records are transformed and mapped to the destination system's field structure. This enables data normalization, field renaming, value transformation, and complex nested object construction.\n\n **Common Use Cases**:\n - Mapping source field names to destination field names\n - Transforming data types and formats\n - Building nested object structures required by the destination\n - Applying formulas and lookups to derive field values\n allOf:\n - $ref: '../common-schemas/mappings.yml#/Mappings'\n lookups:\n x-celigo-agent:\n - lookup\n description: |-\n Top-level mirror of the adaptor-specific ``lookups`` array. Celigo persists lookups\n in TWO locations on every import resource: ``\u003cadaptor_key>.lookups`` (the typed,\n adaptor-specific definition) and this top-level ``lookups`` array (a permissive copy).\n x-celigo-ai-guidance:\n - |-\n ## Mirror invariant\n\n Both arrays MUST hold byte-for-byte identical content: write\n both on every save and treat them as a single logical field.\n On an HTTP import the adaptor copy decides: a save keeps\n ``http.lookups`` and mirrors it here, overwriting a copy here\n that differs, and an array sent only here is dropped on create\n and on PUT with no error. A mapping row that names one of its\n lookups then fails every record with ``Unable to find lookup in\n mapping configs - \u003cname>``.\n - |-\n ## Shape\n\n The shape of each item depends on the adaptor. See the\n ``lookups`` property on the adaptor sub-schema for the canonical\n definition:\n\n - ``salesforce.lookups`` — SOQL-based (sObjectType, whereClause,\n resultField, ...)\n - ``netsuite_da.lookups`` — NetSuite search (recordType,\n searchField, operator, ...)\n - ``http.lookups`` — HTTP request (method, relativeURI, body,\n extract, ...)\n - ``jdbc.lookups`` / ``rdbms.lookups`` — SQL query (query,\n extract, ...)\n - ``wrapper.lookups`` — generic (map, _lookupCacheId, extract,\n default)\n\n Agents SHOULD generate and validate lookups against the adaptor\n schema, then mirror the result here unchanged. The platform\n ignores extraneous fields on this top-level array.\n allOf:\n - $ref: '../common-schemas/lookups.yml#/Lookups'\n inputContext:\n type: string\n enum: [record, envelope]\n x-enumDescriptions:\n record: The import receives each record directly (default behavior).\n envelope: The import receives records wrapped in an envelope with metadata.\n description: Controls the shape of the input passed to the import's processing pipeline.\n x-celigo-ai-guidance:\n - |-\n ## v1 mappings ignore this field\n\n Mapper 1.0 (``mapping`` field) and the NetSuite DA variant\n (``netsuite_da.mapping``) use dot-paths instead of JSONPath,\n and the v1 runtime always evaluates those dot-paths against\n the flat record regardless of ``inputContext``. The v1\n mapperProcessor does not traverse wrapped-envelope samples,\n so envelope-mode framing is meaningless for v1 — agents\n authoring v1 mappings emit bare dot-paths (``email``,\n ``customer.first_name``, ``line_items[*].sku``) in both\n modes.\n - |-\n ## Allowed values\n\n - ``record`` (default — emitted when the field is absent /\n empty / undefined): the mapper sees the FLAT record itself as\n its evaluation root. ``mappings[].extract`` JSONPath values\n MUST anchor directly at `` configuring-imports — Skillopedia ` with NO ``record.`` segment —\n for example ``\"$.orderId\"`` or ``\"$.customer.email\"``.\n PLACEHOLDER extracts use ``\"$.PLACEHOLDER_\u003cfield>\"``. This is\n the dominant Celigo idiom and the right choice for new\n imports.\n\n - ``envelope``: the mapper sees a wrapped object of the shape\n ``{record: {...}, settings: ..., job: ...}`` — the same\n runtime-context envelope Handlebars / SQL / GraphQL\n templates render against. ``mappings[].extract`` JSONPath values\n MUST include the ``record.`` segment — for example\n ``\"$.record.orderId\"`` or ``\"$.record.customer.email\"``.\n PLACEHOLDER extracts use ``\"$.record.PLACEHOLDER_\u003cfield>\"``.\n Pick this mode only when the import was authored against the\n envelope shape on purpose (typically because an upstream step\n intentionally feeds the wrapped runtime context through to\n the import, or the customer is migrating a legacy config that\n assumes the envelope).\n - |-\n ## Handlebars expressions are NOT affected\n\n Handlebars expressions inside ``extract`` (``{{record.field}}``,\n ``{{dateFormat ... record.field}}``, etc.) ALWAYS reference the\n runtime context's ``record`` root regardless of\n ``inputContext`` — the Handlebars runtime always sees the\n wrapped envelope. Only JSONPath behavior flips with this field.\n - |-\n ## Agent guidance\n\n Set this field explicitly on every new import the agent\n authors. Leaving it unset is interpreted as ``record`` by the\n platform, but emitting ``inputContext: record`` makes the\n contract self-documenting and prevents the mapper from drifting\n into envelope mode if an upstream change ever changes the\n platform default. Read this value before writing\n ``mappings[].extract`` so the JSONPath form matches the runtime\n root.\n rest:\n type: object\n deprecated: true\n additionalProperties: true\n description: |-\n Legacy REST adaptor configuration, present on imports created before the REST adaptor\n was folded into `http`. Superseded by the `http` object.\n settingsForm:\n x-celigo-agent:\n - settings_form\n $ref: '../common-schemas/form.yml#/Form'\n preSave:\n $ref: '../common-schemas/pre-save.yml#/PreSave'\n settings:\n $ref: '../common-schemas/settings.yml#/Settings'\n\nMappingField:\n type: object\n description: One Mapper 1.0 field-mapping entry.\n properties:\n generate:\n type: string\n description: Target field path to write on the destination record.\n examples: [\"billingAddress.city\"]\n x-celigo-ai-guidance:\n - |-\n Destination field path on the target record. Use the target\n system's raw field id — for Salesforce the API name (e.g.\n ``AccountId``, ``CustomField__c``); for HTTP/REST the JSON\n key the endpoint expects; for NetSuite the internal field id\n (lowercase, no spaces).\n\n Mapper 1.0 expands two notations into the destination\n structure at write time:\n\n - **Dot notation** — ``customer.email`` builds nested\n objects, writing ``{customer: {email: value}}``. Use it\n freely for any depth of nested object on adaptors whose\n payloads are JSON (HTTP, REST, Shopify-style, etc.).\n NetSuite body fields are flat — do not use dot notation\n there.\n - **Bracket-star notation** — ``addresses[*].company`` is the\n idiomatic V1 expression for \"this scalar belongs to every\n element of the destination array as it is being\n constructed.\" Two or more ``[*]``-suffixed fields with the\n same array prefix collaborate to populate a single element\n per source record, e.g. pairing\n ``customer.addresses[*].company`` with\n ``customer.addresses[*].city`` produces one address object\n carrying both fields.\n - **Numeric bracket notation** — ``addresses[0].company``\n writes to a specific index. It works, but hardcodes the\n slot and is only appropriate when the destination genuinely\n targets a fixed position (rare). Prefer ``[*]`` whenever\n the user describes \"an address\" / \"a line\" / \"an entry\";\n reserve ``[N]`` for \"the second address\" and similar\n explicit-index intents.\n\n customer.email → {\"customer\": {\"email\": ...}}\n customer.addresses[*].company → {\"customer\": {\"addresses\": [{\"company\": ...}]}}\n customer.addresses[*].city ↑ adds \"city\" to the same address element\n extract:\n type: string\n description: |\n Source expression. Accepts:\n - A dot path from the record root (e.g. `email`, `address.city`), read from the\n incoming record. A path that matches no field writes nothing.\n - Handlebars template (contains `{{`) for concatenation /\n formatting / conditional logic (e.g. `{{firstName}} {{lastName}}`,\n `{{#if mobile}}{{mobile}}{{else}}{{phone}}{{/if}}`). The template reads the\n record's fields by bare name: `{{record.firstName}}` and `{{data.firstName}}`\n render nothing, so the field is left out even when the record has the value.\n\n A `$.` JSONPath (the Mapper 2.0 `mappings` syntax) reads nothing here, and a bare\n string is read as a field path, not a constant: set constants in `hardCodedValue`.\n Omit when using `hardCodedValue`.\n examples: [\"address.city\"]\n hardCodedValue:\n type: [string, 'null']\n description: Static value written to `generate` instead of extracting from the source record.\n x-celigo-ai-guidance:\n - |-\n Static value written verbatim. Use it for every constant: a bare\n string in ``extract`` is read as a field path. Mutually exclusive\n with ``extract``.\n dataType:\n type: string\n enum: [string, number, boolean, numberarray, stringarray, json]\n x-lowercase: true\n x-enumDescriptions:\n string: Text string data type.\n number: Numeric data type including integers and decimals.\n boolean: True or false data type.\n numberarray: Array of numeric values.\n stringarray: Array of text string values.\n json: Arbitrary JSON object or structure.\n description: Data type coercion applied to the mapped value.\n discardIfEmpty:\n type: boolean\n description: When true, the field is omitted from the output when the extracted value is empty.\n x-celigo-ai-guidance:\n - |-\n When true, skip this field entirely when ``extract`` / lookup\n produces an empty string, null or nothing. Without this flag an\n empty string is written as ``\"\"`` and null as null, either of which\n may overwrite existing data; a missing source is left out either way.\n immutable:\n type: boolean\n description: When true, the mapped value cannot be overwritten by later mapping steps.\n lookupName:\n type: string\n description: Name of an entry in the import's `lookups` array (`http.lookups` on an HTTP import) used to translate the value.\n x-celigo-ai-guidance:\n - |-\n Name of a lookup defined on ``resource.lookups`` whose result\n replaces the extracted value. The looked-up record's\n ``resultField`` (or full record, depending on lookup shape)\n becomes the mapped value.\n default:\n type: [string, 'null']\n description: |-\n Fallback value written when the source value is an empty string, null or\n missing. Without a default, an empty string and null are written as they are\n and a missing field is left out.\n x-celigo-ai-guidance:\n - |-\n Use ``default``, not ``hardCodedValue``, for a fallback on a field that is\n sometimes empty: ``hardCodedValue`` is written on every record and replaces\n the values that exist.\n extractDateFormat:\n type: string\n description: Date format of the source value, used to parse it before conversion.\n x-celigo-ai-guidance:\n - |-\n Moment.js-style format describing how the source date string is\n structured (e.g. ``MM/DD/YYYY``, ``YYYY-MM-DDTHH:mm:ssZ``).\n extractDateTimezone:\n type: string\n description: Timezone applied when parsing the source date value.\n x-celigo-ai-guidance:\n - |-\n IANA timezone of the source value (e.g. ``America/New_York``).\n Set together with ``generateDateTimezone`` only when a real\n timezone conversion is required.\n generateDateFormat:\n type: string\n description: Date format applied to the value written to the destination.\n generateDateTimezone:\n type: string\n description: Timezone applied when formatting the destination date value.\n conditional:\n type: object\n description: |\n Only apply this mapping entry when the given condition is satisfied.\n properties:\n when:\n type: string\n enum:\n - record_created\n - record_updated\n - extract_not_empty\n - lookup_not_empty\n - lookup_empty\n - expression\n x-enumDescriptions:\n record_created: Apply only when creating a new record.\n record_updated: Apply only when updating an existing record.\n extract_not_empty: Apply only when `extract` produces a non-empty value.\n lookup_not_empty: Apply only when the paired lookup returns a value.\n lookup_empty: Apply only when the paired lookup returns no value.\n expression: Apply only when the accompanying `expression` evaluates truthy.\n description: |\n Condition that gates whether this mapping entry is applied.\n `lookup_not_empty` / `lookup_empty` evaluate the lookup named by the\n sibling `lookupName`; `expression` evaluates the sibling `expression`.\n lookupName:\n type: string\n description: |\n Lookup to evaluate for the `lookup_not_empty` / `lookup_empty`\n conditions.\n expression:\n type: string\n description: |\n Expression evaluated when `when` is `expression`.\n x-celigo-ai-guidance:\n - |-\n One Mapper 1.0 body field assignment. Exactly one of ``extract`` or\n ``hardCodedValue`` should carry the value; setting both is\n rejected by the mapper processor.\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":62792,"content_sha256":"6f03d8c7902e57b686b71502a9562d3e6b0e43ede596e1be2ec8b1061dd8cd14"},{"filename":"references/schemas/clone-request.yml","content":"CloneRequest:\n type: object\n description: Request body for cloning an import.\n properties:\n name:\n type: string\n description: Optional name for the cloned resource. If omitted, the server may generate a default clone name.\n examples: [\"Clone - Import Orders\"]\n connectionMap:\n type: object\n description: |\n Mapping of original connection ids to replacement connection ids.\n Keys are source connection ids on the original resource; values are target connection ids.\n For an import that uses a connection, an empty map fails with 400\n `invalid_connectionMap`, as it does for an export; map the connection to itself to reuse\n it in the clone.\n additionalProperties:\n type: string\n example:\n 5f7c579b6411271af4e7cefa: 64a1234567890abcdef12345\n 5f7c579b6411271af4e7cefb: 64a1234567890abcdef12346\n _stackId:\n type: string\n format: objectId\n x-celigo-refModel: stacks\n description: |-\n Stack that every stack reference in the cloned resources (hooks and wrapper\n connections running on a stack) points at. Required when the clone preview reports\n `stackRequired: true`: without it the clone fails with 400 `stack_not_found`. The\n stack must belong to the target environment or be shared with it (404\n `stack_not_found` for an unknown id, 400 `invalid_stack` otherwise).\n examples: [\"5f8d43a1b9e5a80011a35f2c\"]\n sandbox:\n type: boolean\n deprecated: true\n description: Optional flag copied from the original resource when applicable.\n additionalProperties: true\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":1635,"content_sha256":"0ab5faa8a10ffed16060f4384639dada0758e3b9e1cac75fa22aac45de52eb03"},{"filename":"references/schemas/clone-response.yml","content":"CloneResponse:\n description: Response body for a clone operation. Some clone endpoints return the cloned resource, while others may return a list of related created resources.\n oneOf:\n - $ref: \"./response.yml#/Import\"\n - type: array\n items:\n type: object\n properties:\n model:\n type: string\n description: Model name of the created resource (e.g., Flow, Export, Import).\n _id:\n type: string\n format: objectId\n description: Unique id of the created resource.\n name:\n type: string\n description: Optional name of the created resource.\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":661,"content_sha256":"6eeecb5a6a7ba49ef92aac37441c22c4e1ebedb0031269cdf8e69e7aa44137c5"},{"filename":"references/schemas/dynamodb.yml","content":"Dynamodb:\n type: object\n description: |-\n Configuration for DynamoDB imports. Required when the _connectionId field references a\n DynamoDB connection; must not be included for other connection types. putItem writes a full\n item document; updateItem modifies named attributes of an existing item identified by its\n key.\n required:\n - region\n - method\n - tableName\n properties:\n region:\n type: string\n x-lowercase: true\n enum: [\"us-east-1\", \"us-east-2\", \"us-west-1\", \"us-west-2\", \"af-south-1\", \"ap-east-1\",\n \"ap-south-1\", \"ap-northeast-1\", \"ap-northeast-2\", \"ap-northeast-3\", \"ap-southeast-1\",\n \"ap-southeast-2\", \"ca-central-1\", \"eu-central-1\", \"eu-west-1\", \"eu-west-2\",\n \"eu-west-3\", \"eu-south-1\", \"eu-north-1\", \"me-south-1\", \"sa-east-1\"]\n x-enumDescriptions:\n us-east-1: US East (N. Virginia).\n us-east-2: US East (Ohio).\n us-west-1: US West (N. California).\n us-west-2: US West (Oregon).\n af-south-1: Africa (Cape Town).\n ap-east-1: Asia Pacific (Hong Kong).\n ap-south-1: Asia Pacific (Mumbai).\n ap-northeast-1: Asia Pacific (Tokyo).\n ap-northeast-2: Asia Pacific (Seoul).\n ap-northeast-3: Asia Pacific (Osaka).\n ap-southeast-1: Asia Pacific (Singapore).\n ap-southeast-2: Asia Pacific (Sydney).\n ca-central-1: Canada (Central).\n eu-central-1: Europe (Frankfurt).\n eu-west-1: Europe (Ireland).\n eu-west-2: Europe (London).\n eu-west-3: Europe (Paris).\n eu-south-1: Europe (Milan).\n eu-north-1: Europe (Stockholm).\n me-south-1: Middle East (Bahrain).\n sa-east-1: South America (Sao Paulo).\n default: \"us-east-1\"\n x-celigo-ui-override: >-\n Required by the DynamoDB import form (dynamodb.region is required:true, default\n us-east-1).\n Encoded to mirror the form so builders produce connectable configurations.\n description: |-\n AWS region hosting the DynamoDB table. Determines the service endpoint used for all\n requests and must match the region where the table is deployed.\n x-celigo-ai-guidance:\n - |-\n Specifies the AWS region where the DynamoDB service is hosted and where all\n database operations will be executed.\n This property defines the precise geographical location of the DynamoDB instance,\n which is essential for optimizing application latency,\n ensuring compliance with data residency and sovereignty regulations,\n and aligning with organizational infrastructure strategies.\n Selecting the appropriate region helps minimize network latency by placing data\n physically closer to end-users or application servers,\n thereby improving performance and responsiveness.\n Additionally, the chosen region influences service availability,\n pricing structures, feature sets, and legal compliance requirements,\n making it a critical configuration parameter.\n The region value must be a valid AWS region identifier (such as \"us-east-1\" or\n \"eu-west-1\") and is used to construct the correct service endpoint URL for all\n DynamoDB interactions.\n Proper configuration of this property is vital for seamless integration with AWS SDKs,\n authentication mechanisms, and IAM policies that may be region-specific,\n ensuring secure and efficient access to DynamoDB resources.\n - |-\n ## FIELD BEHAVIOR:\n - Determines the AWS data center location for all DynamoDB operations.\n - Directly impacts network latency, data residency, and compliance adherence.\n - Must be specified as a valid and supported AWS region code.\n - Influences the endpoint URL, authentication endpoints, and service routing.\n - Affects service availability, pricing, feature availability, and latency.\n - Remains consistent throughout the lifecycle of the application unless explicitly changed.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Validate the region against the current list of supported AWS regions to prevent misconfiguration and runtime errors.\n - Use the region value to dynamically construct the DynamoDB service endpoint URL and configure SDK clients accordingly.\n - Allow the region to be configurable and overrideable to support testing environments, multi-region deployments, disaster recovery, or failover scenarios.\n - Ensure consistency of the region setting across all AWS service configurations within the application to avoid cross-region conflicts and data inconsistency.\n - Consider the implications of region selection on data sovereignty laws, compliance requirements, and organizational policies.\n - Monitor AWS announcements for new regions or changes in service availability to keep configurations up to date.\n - |-\n ## EXAMPLES:\n - \"us-east-1\" (Northern Virginia, USA)\n - \"eu-west-1\" (Ireland, Europe)\n - \"ap-southeast-2\" (Sydney, Australia)\n - \"ca-central-1\" (Canada Central)\n - \"sa-east-1\" (São Paulo, Brazil)\n examples: [\"us-east-1\", \"eu-west-1\", \"ap-southeast-2\", \"ca-central-1\", \"sa-east-1\"]\n method:\n type: string\n enum:\n - putItem\n - updateItem\n x-enumDescriptions:\n putItem: Create a new item or replace an existing item entirely in the DynamoDB\n table.\n updateItem: Modify specific attributes of an existing item in the DynamoDB\n table without replacing it.\n x-celigo-ui-override: >-\n Required by the DynamoDB import form (dynamodb.method is required:true). Encoded\n to mirror\n the form so builders produce connectable configurations.\n description: Write operation performed against the DynamoDB table for each record.\n tableName:\n type: string\n x-celigo-ui-override: >-\n Required by the DynamoDB import form (dynamodb.tableName is required:true).\n Encoded to\n mirror the form so builders produce connectable configurations.\n description: |-\n DynamoDB table that receives the imported records.\n Must match an existing table in the target AWS account and region; table names are case-sensitive.\n x-celigo-ai-guidance:\n - |-\n The name of the DynamoDB table to be accessed or manipulated,\n serving as the primary identifier for routing API requests to the correct data store.\n This string must exactly match the name of an existing DynamoDB table within the\n specified AWS account and region,\n including case sensitivity.\n It is essential for performing operations such as reading, writing, updating,\n or deleting items within the table,\n ensuring that all data interactions target the intended resource accurately.\n - |-\n ## FIELD BEHAVIOR:\n - Specifies the exact DynamoDB table targeted for all data operations.\n - Must be unique within the AWS account and region to prevent ambiguity.\n - Directs API calls to the appropriate table for the requested action.\n - Case-sensitive and must strictly adhere to DynamoDB naming conventions.\n - Acts as a critical routing parameter in API requests to identify the data source.\n - |-\n ## EXAMPLES:\n - \"Users\"\n - \"Orders2024\"\n - \"Inventory_Table\"\n - \"CustomerData\"\n - \"Sales-Data.2024\"\n - |-\n ## IMPORTANT NOTES:\n - DynamoDB table names can be up to 255 characters in length.\n - Table names are case-sensitive and must be used consistently across all API calls.\n - The specified table must exist in the targeted AWS region before any operation is performed.\n - Changing the table name requires updating all dependent code and redeploying applications as necessary.\n - Incorrect table names will result in API errors such as ResourceNotFoundException.\n - |-\n ## DEPENDENCY CHAIN:\n - Dependent on the AWS account and region context where DynamoDB is accessed.\n - Requires appropriate IAM permissions to perform operations on the specified table.\n - Works in conjunction with other DynamoDB parameters such as key schema, attribute definitions, and provisioned throughput settings.\n - Relies on network connectivity and AWS service availability for successful API interactions.\n - |-\n ## TECHNICAL DETAILS:\n - Represented as a string value.\n examples: [\"Customers\", \"Orders\", \"Product-Catalog\", \"Users\", \"Orders2024\",\n \"Inventory_Table\", \"CustomerData\", \"Sales-Data.2024\"]\n partitionKey:\n type: string\n x-celigo-agent:\n - handlebars\n x-celigo-ui-override: >-\n Required by the DynamoDB import form when method is updateItem (dynamodb.partitionKey\n required, visibleWhen method=updateItem).\n description: |-\n Value of the target item's partition key for the updateItem operation; together with\n sortKey (when the table defines one) it identifies the item each record writes to.\n Supports handlebars to reference the incoming record's fields.\n examples: [\"{{record.id}}\", \"{{record.customerId}}\"]\n sortKey:\n type: string\n x-celigo-agent:\n - handlebars\n description: |-\n Value of the target item's sort key; supports handlebars. Omit when the table defines only\n a partition key.\n examples: [\"{{record.timestamp}}\", \"{{record.lineId}}\"]\n itemDocument:\n type: string\n x-celigo-agent:\n - handlebars\n description: |-\n JSON document written to the table for the putItem operation, containing all attribute\n names and values for the item, including its key attributes. Supports handlebars.\n x-celigo-ai-guidance:\n - |-\n ## EXAMPLES:\n - `{ \"UserId\": \"12345\", \"Name\": \"John Doe\", \"Age\": 30, \"Preferences\": { \"Language\": \"en\", \"Notifications\": true } }`\n examples: ['{\"id\": \"{{record.id}}\", \"name\": \"{{record.name}}\", \"total\": {{record.total}} }']\n updateExpression:\n type: string\n x-celigo-ui-override: >-\n Required by the DynamoDB import form when method is updateItem (dynamodb.updateExpression\n required, visibleWhen method=updateItem).\n description: |-\n DynamoDB update expression naming which attributes to set, add, or remove on the item for\n the updateItem operation. Only the named attributes change; reference attribute names and\n values through the expressionAttributeNames and expressionAttributeValues placeholders.\n x-celigo-ai-guidance:\n - |-\n A string that defines the specific attributes to be updated, added,\n or removed within a DynamoDB item using DynamoDB's UpdateExpression syntax.\n This expression provides precise control over how item attributes are modified\n by including one or more of the following clauses:\n SET (to assign or overwrite attribute values), REMOVE (to delete attributes),\n ADD (to increment numeric attributes or add elements to sets),\n and DELETE (to remove elements from sets).\n The updateExpression enables atomic, conditional,\n and efficient updates without replacing the entire item,\n ensuring data integrity and minimizing write costs.\n It must be carefully constructed using placeholders for attribute names and\n values—ExpressionAttributeNames and ExpressionAttributeValues—to avoid conflicts\n with reserved keywords and prevent injection vulnerabilities.\n This expression is essential for performing complex update operations in a single request,\n supporting both simple attribute changes and advanced manipulations like\n incrementing counters or modifying sets.\n - |-\n ## FIELD BEHAVIOR:\n - Specifies one or more atomic update operations on a DynamoDB item's attributes.\n - Supports combining multiple update actions (SET, REMOVE, ADD, DELETE) within a single expression, separated by spaces.\n - Requires strict adherence to DynamoDB's UpdateExpression syntax and semantics.\n - Works in conjunction with ExpressionAttributeNames and ExpressionAttributeValues to safely reference attribute names and values.\n - Only affects the attributes explicitly mentioned; all other attributes remain unchanged.\n - Enables conditional updates when combined with ConditionExpression for safe concurrent modifications.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Construct the updateExpression using valid DynamoDB syntax, incorporating clauses such as SET, REMOVE, ADD, and DELETE as needed.\n - Always use placeholders (e.g., '#attrName' for attribute names and ':value' for attribute values) to avoid conflicts with reserved words and enhance security.\n - Validate the expression syntax and ensure all referenced placeholders are defined before executing the update operation.\n - Combine multiple update actions by separating them with spaces within the same expression string.\n - Test expressions thoroughly to prevent runtime errors caused by syntax issues or missing placeholders.\n - Use SET for assigning or overwriting attribute values, REMOVE for deleting attributes, ADD for incrementing numbers or adding set elements, and DELETE for removing set elements.\n - Avoid using reserved keywords directly; always substitute them with ExpressionAttributeNames placeholders.\n - |-\n ## EXAMPLES:\n - \"SET #name = :newName, #age = :newAge REMOVE #obsoleteAttribute\"\n - \"ADD #score :incrementValue DELETE #tags :tagToRemove\"\n - \"SET #status = :statusValue\"\n examples: [\"SET #name = :name, #total = :total\", \"SET #status = :status REMOVE\n #temp\", \"SET #name = :newName, #age = :newAge REMOVE #obsoleteAttribute\",\n \"ADD #score :incrementValue DELETE #tags :tagToRemove\", \"SET #status = :statusValue\"]\n conditionExpression:\n type: string\n description: |-\n Condition evaluated against the existing item before the write executes.\n When the condition evaluates to false, the operation is aborted and no changes are made.\n x-celigo-ai-guidance:\n - |-\n A string that defines the conditional logic to be applied to a DynamoDB operation,\n specifying the precise criteria that must be met for the operation to proceed.\n This expression leverages DynamoDB's condition expression syntax to evaluate\n attributes of the targeted item,\n enabling fine-grained control over operations such as PutItem, UpdateItem,\n DeleteItem, and transactional writes.\n It supports a comprehensive set of logical operators (AND, OR, NOT),\n comparison operators (=, \u003c, >, BETWEEN, IN), functions (attribute_exists,\n attribute_not_exists, begins_with, contains, size),\n and attribute path notations to construct complex, nested conditions.\n This ensures data integrity by preventing unintended modifications and enforcing\n business rules at the database level.\n - |-\n ## FIELD BEHAVIOR:\n - Determines whether the DynamoDB operation executes based on the evaluation of the condition against the item's attributes.\n - Evaluated atomically before performing the operation; if the condition evaluates to false, the operation is aborted and no changes are made.\n - Supports combining multiple conditions using logical operators for complex, multi-attribute criteria.\n - Utilizes built-in functions to inspect attribute existence, value patterns, and sizes, enabling sophisticated conditional checks.\n - Applies exclusively to the specific item targeted by the operation, including nested attributes accessed via attribute path notation.\n - Influences transactional operations by ensuring all conditions in a transaction are met before committing.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Always use ExpressionAttributeNames and ExpressionAttributeValues to safely reference attribute names and values, avoiding conflicts with reserved keywords and improving readability.\n - Validate the syntax of the condition expression prior to request submission to prevent runtime errors and failed operations.\n - Carefully construct expressions to handle edge cases, such as attributes that may be missing or have null values.\n - Combine multiple conditions logically to enforce precise constraints and business logic on the operation.\n - Test condition expressions thoroughly across diverse data scenarios to ensure expected behavior and robustness.\n - Keep expressions as simple and efficient as possible to optimize performance and reduce request complexity.\n - |-\n ## EXAMPLES:\n - \"attribute_exists(#name) AND #age >= :minAge\"\n - \"#status = :activeStatus\"\n - \"attribute_not_exists(#id)\"\n - \"#score BETWEEN :low AND :high\"\n - \"begins_with(#title, :prefix)\"\n - \"contains(#tags, :tagValue) OR size(#comments) > :minComments\"\n - |-\n ## IMPORTANT NOTES:\n - The conditionExpression is optional but highly recommended to safeguard against unintended overwrites, deletions, or updates.\n examples: [\"attribute_not_exists(#pk)\", \"#version = :expectedVersion\", \"attribute_exists(#name)\n AND #age >= :minAge\", \"#status = :activeStatus\", \"attribute_not_exists(#id)\",\n \"#score BETWEEN :low AND :high\", \"begins_with(#title, :prefix)\", \"contains(#tags,\n :tagValue) OR size(#comments) > :minComments\"]\n expressionAttributeNames:\n type: string\n description: |-\n JSON placeholder-to-name map for attribute names referenced in expressions, for example {\"#N\": \"Name\"}.\n Use it when an attribute name is a reserved word or contains special characters; placeholder keys start with #.\n x-celigo-ai-guidance:\n - |-\n expressionAttributeNames is a map of substitution tokens used to safely\n reference attribute names within DynamoDB expressions.\n This property allows you to define placeholder tokens for attribute names that\n might otherwise cause conflicts due to being reserved words,\n containing special characters, or having names that are dynamically generated.\n By using these placeholders,\n you can avoid syntax errors and ambiguities in expressions such as ProjectionExpression,\n FilterExpression, UpdateExpression, and ConditionExpression,\n ensuring your queries and updates are both valid and maintainable.\n - |-\n ## FIELD BEHAVIOR:\n - Functions as a dictionary where each key is a placeholder token starting with a '#' character, and each value is the actual attribute name it represents.\n - Enables the use of reserved words, special characters, or otherwise problematic attribute names within DynamoDB expressions.\n - Supports complex expressions by allowing multiple attribute name substitutions within a single expression.\n - Prevents syntax errors and improves clarity by explicitly mapping placeholders to attribute names.\n - Applies only within the scope of the expression where it is defined, ensuring localized substitution without affecting other parts of the request.\n - |-\n ## EXAMPLES:\n - `{\"#N\": \"Name\", \"#A\": \"Age\"}` to substitute attribute names \"Name\" and \"Age\" in an expression.\n - `{\"#status\": \"Status\"}` when \"Status\" is a reserved word or needs escaping in an expression.\n - `{\"#yr\": \"Year\", \"#mn\": \"Month\"}` for expressions involving multiple date-related attributes.\n - `{\"#addr\": \"Address\", \"#zip\": \"ZipCode\"}` to handle attribute names with special characters or spaces.\n - `{\"#P\": \"Price\", \"#Q\": \"Quantity\"}` used in an UpdateExpression to increment numeric attributes.\n examples: ['{\"#name\": \"Name\", \"#status\": \"status\"}']\n expressionAttributeValues:\n type: string\n x-celigo-agent:\n - handlebars\n description: |-\n JSON placeholder-to-value map for attribute values referenced in expressions; placeholder\n keys start with a colon (e.g. \":val\") and each value is a DynamoDB attribute-value object\n such as {\"S\": \"text\"}. Supports handlebars.\n x-celigo-ai-guidance:\n - |-\n expressionAttributeValues is a map of substitution tokens for attribute values\n used within DynamoDB expressions.\n These tokens serve as placeholders that safely inject values into expressions\n such as ConditionExpression,\n FilterExpression, UpdateExpression, and KeyConditionExpression.\n By separating data from code,\n this mechanism prevents injection attacks and syntax errors,\n ensuring secure and reliable query and update operations.\n Each key in this map is a placeholder string that begins with a colon (\":\"),\n and each corresponding value is a DynamoDB attribute value object representing\n the actual data to be used in the expression.\n This design enforces that user input is never directly embedded into expressions,\n enhancing both security and maintainability.\n - |-\n ## FIELD BEHAVIOR:\n - Contains key-value pairs where keys are placeholders prefixed with a colon (e.g., \":val1\") used within DynamoDB expressions.\n - Each key maps to a DynamoDB attribute value object specifying the data type and value, supporting all DynamoDB data types including strings, numbers, binaries, lists, maps, sets, and booleans.\n - Enables safe and dynamic substitution of attribute values in expressions, avoiding direct string concatenation and reducing the risk of injection vulnerabilities.\n - Mandatory when expressions reference attribute values to ensure correct parsing and execution of queries or updates.\n - Placeholders are case-sensitive and must exactly match those used in the corresponding expressions.\n - Supports complex data structures, allowing nested maps and lists as attribute values for advanced querying scenarios.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Always prefix placeholder keys with a colon (\":\") to clearly identify them as substitution tokens within expressions.\n - Ensure every placeholder referenced in an expression has a corresponding entry in expressionAttributeValues to avoid runtime errors.\n - Format each value according to DynamoDB's attribute value specification, such as { \"S\": \"string\" }, { \"N\": \"123\" }, { \"BOOL\": true }, or complex nested structures.\n - Avoid using reserved words, spaces, or invalid characters in placeholder names to prevent syntax errors and conflicts.\n - Validate that the data types of values align with the expected attribute types defined in the table schema to maintain data integrity.\n - When multiple expressions are used simultaneously, maintain unique and consistent placeholder names to prevent collisions and ambiguity.\n - Use expressionAttributeValues in conjunction with expressionAttributeNames when attribute names are reserved words or contain special characters to ensure proper expression parsing.\n - Consider the size limitations of expressionAttributeValues to avoid exceeding DynamoDB's request size limits.\n examples: ['{\":name\": {\"S\": \"{{record.name}}\"}, \":total\": {\"N\": \"{{record.total}}\"}}']\n ignoreExtract:\n type: string\n x-celigo-agent:\n - handlebars\n description: |-\n Path to the field in the source record used to determine whether the record already exists.\n Only used when ignoreExisting is set on the import.\n examples: [\"{{record.id}}\"]\n if:\n properties:\n method:\n const: updateItem\n required:\n - method\n then:\n required:\n - partitionKey\n - updateExpression\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":24013,"content_sha256":"0abf2afc2d2c6528499172a18c9062baecc4c71eb2888a7edd65a7f9968ec615"},{"filename":"references/schemas/file.yml","content":"File:\n type: object\n x-celigo-excludeAdaptorTypes:\n - RDBMSImport\n - JDBCImport\n - SalesforceImport\n - NetSuiteImport\n - MongodbImport\n x-celigo-ai-guidance:\n - |-\n Never author or patch `file` on an `RDBMSImport`. On a bulk load the API echoes a\n platform-derived `file` object whose values a save cannot change: the write is accepted,\n the change is discarded, and a read-back shows the original values. Shape the records\n before the write (a preMap/postMap hook, the mapping, a transform) or correct the source\n data instead.\n description: |-\n **CRITICAL: This object is REQUIRED for all file-based import adaptor types.**\n\n **When to include this object**\n\n ✅ **MUST SET** when `adaptorType` is one of:\n - `S3Import`\n - `FTPImport`\n - `AS2Import`\n - `HTTPImport` in file mode (`http.type: \"file\"` — the folder mode of the cloud file\n providers such as Google Drive, Box, Dropbox, Azure Blob, Google Cloud Storage and Celigo\n Storage); the same `file` object, including `type: filedefinition` with `_ediProfileId`\n for EDI, applies as on S3 / FTP\n\n ❌ **DO NOT SET** for non-file-based imports like:\n - `SalesforceImport`\n - `NetSuiteImport`\n - `HTTPImport` in records mode (`http.type` unset — a REST endpoint receiving records)\n - `MongodbImport`\n - `RDBMSImport`\n - `JDBCImport`\n\n **Read-only on database bulk loads**\n\n An `RDBMSImport` whose `rdbms.queryType` is `bulk_load` returns a `file` object anyway: the\n platform's description of the staging file it generates for the load (delimiter, quoting,\n newline handling, and how the load is batched into files). It is derived on every read,\n never stored — a save that changes it is accepted and the change discarded (the audit log\n still records the request), and the import editor has no control for it. Change what\n reaches the destination with a preMap/postMap hook, the mapping, or a transform, or correct\n the source data.\n\n **Minimum required fields**\n\n For most file imports, you need at minimum:\n - `fileName`: The output file name (supports Handlebars like `{{timestamp}}`)\n - `type`: The file format (json, csv, xml, xlsx)\n - `aggregation.type`: How the run's records are batched into files (`all` for one file per run)\n\n **Example (S3 Import)**\n\n ```json\n {\n \"file\": {\n \"fileName\": \"customers-{{timestamp}}.json\",\n \"aggregation\": { \"type\": \"all\" },\n \"type\": \"json\"\n },\n \"s3\": {\n \"region\": \"us-east-1\",\n \"bucket\": \"my-bucket\",\n \"fileKey\": \"customers-{{timestamp}}.json\"\n },\n \"adaptorType\": \"S3Import\"\n }\n ```\n properties:\n fileName:\n type: string\n description: |-\n **REQUIRED for file-based imports.**\n\n The name of the file to be created/written. Supports Handlebars expressions for dynamic naming,\n rendered against the incoming record — a file-transfer import keeps the source file's name from\n the record's `fileMeta`, whose field names are the exporting provider's\n (`{{record.fileMeta.fileName}}` from an FTP source, `fileMeta.name` from Google Drive).\n\n **Default behavior**\n - When the user does not specify a file naming convention, **always default to timestamped filenames** using `{{timestamp}}` (e.g., `\"items-{{timestamp}}.csv\"`). This ensures each run produces a unique file and avoids overwriting previous exports.\n - Only use a fixed filename (without timestamp) if the user explicitly requests overwriting or a fixed name.\n\n **Common patterns**\n - `\"data-{{timestamp}}.json\"` - Timestamped JSON file (DEFAULT — use this pattern when not specified)\n - `\"export-{{timestamp 'YYYY-MM-DD'}}.csv\"` - Date-stamped CSV file\n - `\"{{recordType}}-backup.xml\"` - Dynamic record type naming\n\n **Examples**\n - `\"customers-{{timestamp}}.json\"`\n - `\"orders-export.csv\"`\n - `\"inventory-{{timestamp 'YYYY-MM-DD'}}.xlsx\"`\n\n **Important**\n - For S3 imports, this should typically match the `s3.fileKey` value\n - For FTP imports, this should typically match the `ftp.fileName` value; the one blob file-transfer run on record set only this field and left `ftp.fileName` unset\n x-celigo-ai-guidance:\n - |-\n `fileMeta` is not a root key of this template's context (it is one only on\n file-provider export and lookup steps): `{{fileMeta.fileName}}` renders empty and the\n file has no name; on an FTP import the record then fails with `missing_s3_key` \"Can not\n process upload file without computedObject.fileName.\", labeled source `internal`\n although the template is the cause. The incoming record carries the source file's\n metadata — on an FTP blob import `{{record.fileMeta.fileName}}` keeps the source file's\n name at run time (the only spelling proven by a run). The field's editor preview\n (data version 1, its default) exposes the record as `data` — the `{blobKey, fileMeta}`\n record on a `blob: true` import, the page's record array otherwise — so `record.`\n previews empty there; judge by a run. The metadata field names are the exporting\n provider's — read them off the incoming sample record.\n examples: [\"customers-{{timestamp}}.json\", \"orders-export.csv\"]\n skipAggregation:\n type: boolean\n description: |-\n Legacy companion of `aggregation.type`: `true` when the run's records are written one\n file per page, `false` when they are combined across the run or grouped by key.\n Validation reads the value sent — an EDI file definition needs `true` unless\n `aggregation.type` is `key`, and `batchSize` needs `true` — then the save rewrites it\n from `aggregation.type`, so send the two as a consistent pair.\n default: false\n x-celigo-ai-guidance:\n - |-\n Send `skipAggregation` and `aggregation.type` as a consistent pair, exactly as the UI\n does: `page` with `true`; `all` or `key` with `false`. Validation reads the SENT flag —\n a `file.type: filedefinition` import saved with `page` and no `skipAggregation: true`\n fails with 422 `not_supported` (\"Aggregation is not supported for EDI files, please set\n skipAggregation to true in Imports\"), and `batchSize` fails with 422 `not_supported`\n (\"File imports do not support batching with aggregation\") unless the sent flag is\n `true`. The SAVED flag is then derived from `aggregation.type`, so this flag alone never\n says which mode is active — read `aggregation.type`. It does not mean one file per record.\n type:\n type: string\n enum:\n - json\n - csv\n - xml\n - xlsx\n - filedefinition\n x-enumDescriptions:\n json: JSON format, best for structured or nested data and API payloads; most common for API data\n csv: Comma-separated values format for flat tabular data.\n xml: XML format for XML-based integrations and SOAP services.\n xlsx: Microsoft Excel spreadsheet format.\n filedefinition: Custom file definition format for complex, proprietary, or EDI file structures.\n description: |-\n **REQUIRED for file-based imports.**\n\n The format of the output file.\n\n Select the format that matches the target system's requirements.\n x-celigo-ai-guidance:\n - |-\n ## SELECTION GUIDANCE:\n - Use `\"json\"` for structured/nested data, API payloads\n - Use `\"csv\"` for flat tabular data, spreadsheet-like records\n - Use `\"xml\"` for XML-based integrations, SOAP services\n - Use `\"xlsx\"` for Excel-compatible outputs\n encoding:\n type: string\n enum:\n - utf8\n - win1252\n - utf-16le\n - gb18030\n - macroman\n - iso88591\n - shiftjis\n x-enumDescriptions:\n utf8: UTF-8 Unicode encoding, the universal default for most systems.\n win1252: Windows-1252 Western European encoding for legacy Windows systems.\n \"utf-16le\": UTF-16 Little Endian encoding with byte order mark support.\n gb18030: Chinese National Standard encoding for Chinese character sets.\n macroman: Mac OS Roman encoding for legacy Macintosh systems.\n iso88591: ISO 8859-1 Latin-1 encoding for Western European languages.\n shiftjis: Shift JIS encoding for Japanese character sets.\n description: |-\n Character encoding for the output file.\n\n **Default:** `\"utf8\"`\n\n Change from the default only when the target system requires a specific encoding.\n default: utf8\n delete:\n type: boolean\n description: |-\n Whether to delete the source file after successful import.\n\n **Values**\n - `true`: Delete source file after processing\n - `false`: Keep source file\n\n **Default:** `false`\n default: false\n compressionFormat:\n type: string\n enum:\n - gzip\n - zip\n x-enumDescriptions:\n gzip: GZIP compression producing a single compressed .gz file.\n zip: ZIP compression producing a .zip archive that can contain multiple files.\n description: |-\n Compression format for the output file.\n\n **When to use**\n - Large files that benefit from compression\n - When target system expects compressed files\n backupPath:\n type: string\n description: |-\n Path where backup copies of files should be stored.\n\n **Examples**\n - `\"backup/\"` - Relative backup folder\n - `\"/archive/2024/\"` - Absolute backup path\n examples: [\"backup/\", \"/archive/2024/\"]\n purgeInternalBackup:\n type: boolean\n description: |-\n Whether to purge internal backup copies after successful processing.\n\n **Default:** `false`\n default: false\n batchSize:\n type: integer\n description: |-\n Number of records to include per batch/file when processing large datasets.\n\n **When to use**\n - Large imports that need to be split into multiple files\n - When target system has file size limitations\n\n **Note**\n - Offered when records are written one file per page (`aggregation.type: page` with\n `skipAggregation: true`)\n x-celigo-ai-guidance:\n - |-\n Probed for `all` and `page`: the save rejects `batchSize` with 422 `not_supported`\n (\"File imports do not support batching with aggregation\") when the sent\n `skipAggregation` is not `true`, and accepts it with `page` + `skipAggregation: true`.\n The `key` + `batchSize` combination has not been probed (it needs a B2B Manager\n account) — treat it as unverified, neither allowed nor forbidden.\n encrypt:\n type: boolean\n description: |-\n Whether to encrypt the output file.\n\n **Values**\n - `true`: Encrypt the file (requires PGP configuration)\n - `false`: No encryption\n\n **Default:** `false`\n default: false\n csv:\n type: object\n description: |-\n CSV-specific configuration. Only used when `type` is `\"csv\"`.\n properties:\n rowDelimiter:\n type: string\n description: Character(s) used to separate rows. Default is newline.\n default: \"\\n\"\n columnDelimiter:\n type: string\n description: Character(s) used to separate columns. Default is comma.\n default: \",\"\n includeHeader:\n type: boolean\n description: When true, includes a header row with column names in the output.\n default: true\n wrapWithQuotes:\n type: boolean\n description: When true, wraps field values in quotes.\n x-celigo-ai-guidance:\n - Whether to wrap field values in quotes.\n default: false\n replaceTabWithSpace:\n type: boolean\n description: Replace tab characters with spaces.\n default: false\n replaceNewlineWithSpace:\n type: boolean\n description: Replace newline characters with spaces within fields.\n default: false\n truncateLastRowDelimiter:\n type: boolean\n description: Remove trailing row delimiter from the file.\n default: false\n json:\n type: object\n description: |-\n JSON-specific configuration. Only used when `type` is `\"json\"`.\n properties:\n resourcePath:\n type: string\n description: JSONPath expression to locate records within the JSON structure.\n xml:\n type: object\n description: |-\n XML-specific configuration. Only used when `type` is `\"xml\"`.\n properties:\n resourcePath:\n type: string\n description: XPath expression to locate records within the XML structure.\n fileDefinition:\n type: object\n description: |\n Configuration settings for parsing files using a predefined file definition. This object enables processing of complex, non-standard, or proprietary file formats that require specialized parsing logic beyond what the standard parsers (CSV, JSON, XML, etc.) can handle.\n\n **When to use**\n\n Configure this object when the `type` field is set to \"filedefinition\". This approach is required for properly handling:\n - Legacy or proprietary file formats with complex structures\n - Fixed-width text files where field positions are defined by character positions\n - Electronic Data Interchange (EDI) documents (X12, EDIFACT, etc.)\n - Multi-record type files where different lines have different formats\n - Files requiring complex preprocessing or custom parsing logic\n\n **File definition characteristics**\n\n - **Custom Parsing Rules**: Applies predefined parsing logic to complex file formats\n - **Reusable Configurations**: References externally defined parsing rules that can be reused\n - **Complex Format Support**: Handles formats that standard parsers cannot process\n - **Specialized Processing**: Often used for industry-specific or legacy formats\n\n **Implementation strategy for ai agents**\n\n 1. **Format Analysis**:\n - Determine if the file format is standard (CSV, JSON, XML) or requires custom parsing\n - Check if the format follows industry standards like EDI, SWIFT, or fixed-width\n - Assess if there are multiple record types within the same file\n - Identify if specialized logic is needed to interpret the file structure\n\n 2. **File Definition Selection**:\n - Verify that a suitable file definition has already been created in the system\n - Check if existing file definitions match the format requirements\n - Confirm the file definition ID from system administrators if needed\n - Ensure the file definition is compatible with the export's needs\n properties:\n _fileDefinitionId:\n type: string\n format: objectId\n x-celigo-refModel: filedefinitions\n description: Reference to the file definition resource.\n examples: [\"66a1f2c3b4d5e6f7a8b9c0d1\"]\n aggregation:\n type: object\n description: |-\n Controls how a flow run's records are batched into output files — the transfer form's\n \"How many files would you like to generate?\" setting. The legacy `skipAggregation` flag\n mirrors `type` and is maintained by the platform.\n properties:\n type:\n type: string\n enum: [\"all\", \"page\", \"key\"]\n x-enumDescriptions:\n all: One file per flow run — every record from the run (per export, or per request on real-time flows) in a single file\n page: One file per page of records\n key: One file per aggregation key — EDI file definitions only; records sharing the key derived from the file definition's document type are combined into one document\n description: |-\n Which batches of a run's records become separate output files. EDI file definitions\n (`file.type: filedefinition`) are written per page or per key, never per run; `key`\n is accepted only for them and only when the account has B2B Manager. The grouping\n key and the line-item loop it merges come from the EDI file definition's document\n type, so there is no separate key field to configure.\n examples: [\"all\", \"page\", \"key\"]\n x-celigo-ai-guidance:\n - |-\n Validated on save: an unknown value fails with 422 `enum`; `key` fails with 422\n `not_supported` unless `file.type` is `filedefinition` (\"Aggregation type key is\n only supported for file type filedefinition\") and the account has B2B Manager\n (\"Aggregation type key is only supported for B2B Manager\").\n - |-\n For `file.type: filedefinition` send `page` (with `skipAggregation: true`) or `key`\n (with `false`); `all` is not an EDI option — the validator enforces this through the\n `skipAggregation` rule (see that field). For every other file type the choices are\n `all` and `page`. The UI offers exactly these splits.\n - |-\n A user asking where the \"aggregation key\" is configured is asking about this\n field: the key itself is derived from the file definition's document type, never\n entered.\n lookups:\n type: array\n description: |-\n Named value-substitution maps available to this import's field mappings; each lookup\n translates source values to target values, with an optional default for unmatched values.\n Referenced by name from mapping rules.\n items:\n type: object\n properties:\n name:\n type: string\n description: Identifier used to reference this lookup from mapping rules.\n examples: [\"status_Lookup\"]\n map:\n type: object\n description: Source-value to target-value pairs applied during mapping.\n additionalProperties:\n type: string\n default:\n type: [string, 'null']\n description: Value substituted when a source value is not found in map; empty string\n when omitted. May be stored as null (no default).\n allowFailures:\n type: boolean\n description: When true, a value missing from map does not fail the record and the default (or original value) is used.\n examples: [true, false]\n pgp:\n type: object\n description: |-\n PGP encryption settings applied when encrypt is true; selects the symmetric cipher and the\n signing hash. The connection must hold the recipient's public key.\n properties:\n symmetricKeyAlgorithm:\n type: string\n enum: [\"twofish\", \"cast5\", \"3des\", \"aes128\", \"aes192\", \"aes256\"]\n x-enumDescriptions:\n twofish: Twofish block cipher.\n cast5: CAST5 (CAST-128) block cipher.\n 3des: Triple DES block cipher (legacy).\n aes128: AES with a 128-bit key.\n aes192: AES with a 192-bit key.\n aes256: AES with a 256-bit key (default, recommended).\n default: aes256\n description: Symmetric cipher used to encrypt the file body; aes256 unless the recipient requires another.\n examples: [\"aes256\", \"3des\"]\n hashAlgorithm:\n type: string\n enum: [\"sha256\", \"sha384\", \"sha512\", \"sha224\"]\n x-enumDescriptions:\n sha256: SHA-256 signing hash.\n sha384: SHA-384 signing hash.\n sha512: SHA-512 signing hash.\n sha224: SHA-224 signing hash.\n description: Hash algorithm used when signing the encrypted file; set only when the recipient requires a specific one.\n examples: [\"sha256\", \"sha512\"]\n skipRename:\n type: boolean\n description: |-\n When true, the import writes directly to the final file name instead of writing to a\n temporary name and renaming on completion. Use only when the destination does not support\n atomic rename; the default (false) uses the safe write-then-rename behavior.\n examples: [true, false]\n appendToExistingFile:\n type: boolean\n default: false\n x-celigo-ui-override: >-\n Defaulted by the Azure Data Lake Storage Gen2 import form\n (fieldDefinitions/resources/imports/azuredatalakestoragegen2.js: 'file.appendToExistingFile'\n defaultValue r?.file?.appendToExistingFile === true, shown only for file.type csv in\n records mode). When it is on, preSave clears file.compressionFormat, file.encrypt,\n file.pgp and the Content-Type header — append mode and compress/encrypt are exclusive\n on the form. The API stores the flag on any file import.\n description: |-\n When true, an Azure Data Lake Storage Gen2 import (`assistant: azuredatalakestoragegen2`)\n appends each run's CSV output to the file named by `file.fileName` instead of replacing\n it. Appended content cannot be compressed or encrypted, so leave `compressionFormat`,\n `encrypt` and `pgp` unset when this is on.\n examples: [true, false]\n contentType:\n type: string\n x-celigo-agent:\n - handlebars\n description: |-\n MIME type recorded on the file written to a cloud file provider such as Azure Data Lake\n Storage Gen2. Accepts a Handlebars expression so the type can follow the record (e.g.\n `{{record.mimeType}}`). Omit to let the destination assign its default.\n examples: [\"text/csv\", \"application/json\", \"{{record.mimeType}}\"]\n directory:\n type: object\n required:\n - pathMode\n description: |-\n Structured destination location for cloud file-provider imports (Google Drive shared\n drives, Box, Dropbox, Google Cloud Storage, Azure Data Lake Storage Gen2), replacing the\n flat path used by classic FTP/S3 destinations. Select the location by folder ID or by a\n path relative to a configured storage root via pathMode. For the drive-style providers,\n directoryId mode addresses the folder by id alone — the flat destination-path field is\n not consulted at runtime. Google Cloud Storage puts the bucket in `storageRootId`;\n Azure Data Lake Storage Gen2 puts the container in `name` and the directory prefix of the\n file path in `http.relativeURI`.\n x-celigo-ai-guidance:\n - >-\n Save-time validation accepts any string for id without checking it against the\n provider — set it only from a folder ID the user supplied or the UI folder browser\n stored; never fabricate one. To address a folder by path instead, set pathMode to\n relativePath and put the path in the adaptor's flat path field.\n properties:\n pathMode:\n type: string\n enum: [\"relativePath\", \"directoryId\"]\n x-enumDescriptions:\n relativePath: Address the location by a path relative to the storage root identified by storageRootId.\n directoryId: Address the location directly by the provider's folder ID set in id.\n description: How the directory is addressed — by provider folder ID, or by a path relative to a configured storage root.\n examples: [\"relativePath\", \"directoryId\"]\n id:\n type: string\n description: |-\n Provider-native folder ID of the destination directory (e.g. a Google Drive folder ID);\n set when pathMode is directoryId. This is the file provider's own ID, not a Celigo resource ID.\n examples: [\"1KppvTTr4jKhJqwPAhGWN6SRwk5m45DL2\"]\n name:\n type: string\n description: |-\n Display name of the folder identified by id, kept for readability in the UI; the\n folder browser fills it when the location is picked visually. For Azure Data Lake\n Storage Gen2 it is the authoritative container name instead — the form saves the\n container here and no id.\n examples: [\"Invoices - Outbound\", \"invoices-outbound\"]\n storageRootId:\n type: string\n description: |-\n Provider-native ID of the storage root the relative path resolves against; set when\n pathMode is relativePath. The provider's own ID, not a Celigo resource ID. For Google\n Cloud Storage it is the bucket name.\n examples: [\"1qU1hvFarBFMb4-twqKm1rR5o_6j9vs9g\", \"erp-exports\"]\n storageRootName:\n type: string\n description: Display name of the storage root identified by storageRootId, retained for reference in the UI.\n examples: [\"Shared drive - Finance\"]\n backupDirectory:\n type: object\n required:\n - pathMode\n description: |-\n Structured location for backup copies on cloud file providers — the file-provider\n counterpart to backupPath. Addressed the same way as directory.\n properties:\n pathMode:\n type: string\n enum: [\"relativePath\", \"directoryId\"]\n x-enumDescriptions:\n relativePath: Address the backup location by a path relative to the storage root identified by storageRootId.\n directoryId: Address the backup location directly by the provider's folder ID set in id.\n description: How the backup location is addressed — by provider folder ID, or by a path relative to a configured storage root.\n examples: [\"relativePath\", \"directoryId\"]\n id:\n type: string\n description: Provider-native folder ID of the backup directory; set when pathMode is directoryId.\n examples: [\"1KppvTTr4jKhJqwPAhGWN6SRwk5m45DL2\"]\n name:\n type: string\n description: |-\n Display name of the backup folder identified by id, kept for readability in the UI;\n the folder browser fills it when the location is picked visually.\n examples: [\"Processed archive\"]\n storageRootId:\n type: string\n description: Provider-native ID of the storage root the backup relative path resolves against; set when pathMode is relativePath.\n examples: [\"1qU1hvFarBFMb4-twqKm1rR5o_6j9vs9g\"]\n\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":26617,"content_sha256":"f2c9a4cee979a34eae9a595223377dbeb81b190e4c3643cf70f03653e89025a6"},{"filename":"references/schemas/filesystem.yml","content":"FileSystem:\n type: object\n description: |-\n Defines where files are written to a local or mounted folder on the host running the\n on-premise agent. Required when the _connectionId field references a file-system (on-premise)\n connection; must not be included for other connection types.\n required:\n - directoryPath\n properties:\n directoryPath:\n type: string\n x-celigo-agent:\n - handlebars\n description: |-\n Folder on the on-premise agent's host where generated files are written; the agent's OS\n account must have write permission on it. Accepts a local OS path or a UNC network share,\n and supports handlebars templates for dynamic folders.\n examples: [\"C:\\\\Celigo\\\\Outbound\", \"D:\\\\imports\\\\incoming\", \"\\\\\\\\NAS\\\\Shared\\\\Drop\"]\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":794,"content_sha256":"88ef8fb58455f338adfc40e589119936556f1e236b4ff17b0017d74e405873a6"},{"filename":"references/schemas/ftp.yml","content":"Ftp:\n type: object\n description: |-\n Defines where files are written on an FTP, FTPS, or SFTP server. Required when the\n _connectionId field references an FTP/SFTP connection; must not be included for other\n connection types. directoryPath selects the target folder and the file naming comes from\n the import's file configuration.\n required:\n - directoryPath\n properties:\n _tpConnectorId:\n type: string\n format: objectId\n x-celigo-refModel: tradingpartnerconnectors\n description: |-\n Trading Partner Connector that supplies partner-specific B2B settings for this import.\n When set, the import inherits the connector's pre-configured settings; omit to use only\n the FTP connection details.\n x-celigo-ai-guidance:\n - |-\n Identifier for the third-party connector utilized in the FTP integration,\n serving as a unique and immutable reference that links the FTP configuration to\n an external connector service.\n This identifier is essential for accurately associating the FTP connection with\n the intended third-party system,\n enabling seamless and secure data transfer, synchronization,\n and management through the specified connector.\n It ensures that all FTP operations are correctly routed and managed via the external service,\n supporting reliable integration workflows and maintaining system integrity\n throughout the connection lifecycle.\n - |-\n ## FIELD BEHAVIOR:\n - Uniquely identifies the third-party connector associated with the FTP configuration.\n - Acts as a key reference to bind FTP settings to a specific external connector service.\n - Required when creating, updating, or managing FTP connections via third-party integrations.\n - Typically immutable after initial assignment to preserve connection consistency.\n - Used by system components to route FTP operations through the correct external connector.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Should be a string or numeric identifier that aligns with the third-party connector's identification scheme.\n - Must be validated against the system's registry of available connectors to ensure validity.\n - Should be set securely and protected from unauthorized modification.\n - Updates to this identifier should be handled cautiously, with appropriate validation and impact assessment.\n - Ensure the identifier complies with any formatting or naming conventions imposed by the third-party service.\n - Transmit and store this identifier securely, using encryption where applicable.\n - |-\n ## EXAMPLES:\n - \"connector_12345\"\n - \"tpConnector-67890\"\n - \"ftpThirdPartyConnector01\"\n - \"extConnector_abcde\"\n - \"thirdPartyFTP_001\"\n - |-\n ## IMPORTANT NOTES:\n - This field is critical for correctly mapping FTP configurations to their corresponding third-party services.\n - An incorrect or missing _tpConnectorId can lead to connection failures, data misrouting, or integration errors.\n - Synchronization between this identifier and the third-party connector records must be maintained to avoid inconsistencies.\n - Changes to this field may require re-authentication or reconfiguration of the FTP integration.\n - Proper error handling should be implemented to manage cases where the identifier does not match any registered connector.\n - |-\n ## DEPENDENCY CHAIN:\n - Depends on the existence and registration of third-party connectors within the system.\n - Referenced by FTP connection management modules, authentication services, and integration workflows.\n - Interacts with authorization mechanisms to ensure secure access to third-party services.\n - May influence logging, monitoring, and auditing processes related to FTP integrations.\n examples: [\"60a2c4e6f321d800129a1a3c\"]\n directoryPath:\n type: string\n x-celigo-agent:\n - handlebars\n x-celigo-ui-override: >-\n Required by the FTP import form (fieldDefinitions/resources/imports/ftp.js: ftp.directoryPath\n is required:true). Encoded to mirror the form so builders produce connectable configurations.\n description: |-\n Directory on the server where imported files are written, either absolute or relative to\n the login directory; the FTP user must have write permission on it. Use forward slashes\n regardless of server OS — paths are case-sensitive on UNIX/Linux servers. Supports\n handlebars templates.\n x-celigo-ai-guidance:\n - |-\n A handlebars template here is rendered against the same context as `fileName`\n (`fileMeta` is not a root key). An expression that renders empty fails the record\n with `missing_s3_key` \"Can not process upload file without\n computedObject.fileLocation.\" — labeled source `internal` although the template is\n the cause.\n - |-\n The path to the directory on the FTP server where files will be accessed,\n stored, or managed.\n This path can be specified either as an absolute path starting from the root\n directory of the FTP server or as a relative path based on the user's initial\n directory upon login,\n depending on the server's configuration and permissions.\n It is essential that the path adheres to the FTP server's directory structure,\n naming conventions,\n and access controls to ensure successful file operations such as uploading,\n downloading, listing, or deleting files.\n The directoryPath serves as the primary reference point for all file-related\n commands during an FTP session,\n determining the scope and context of file management activities.\n - |-\n ## FIELD BEHAVIOR:\n - Defines the target directory location on the FTP server for all file-related operations.\n - Supports both absolute and relative path formats, interpreted according to the FTP server's setup.\n - Must comply with the server's directory hierarchy, naming rules, and access permissions.\n - Used by the FTP client to navigate and perform operations within the specified directory.\n - Influences the scope of accessible files and subdirectories during FTP sessions.\n - Changes to this path affect subsequent file operations until updated or reset.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Validate the directory path format to ensure compatibility with the FTP server, typically using forward slashes (`/`) as separators.\n - Normalize the path to remove redundant elements such as `.` (current directory) and `..` (parent directory) to prevent errors and security vulnerabilities.\n - Handle scenarios where the specified directory does not exist by either creating it if permissions allow or returning a clear, descriptive error message.\n - Properly encode special characters and escape sequences in the path to conform with FTP protocol requirements and server expectations.\n - Provide informative and user-friendly error messages when the path is invalid, inaccessible, or lacks sufficient permissions.\n - Sanitize input to prevent directory traversal attacks or unauthorized access to restricted areas.\n - Consider server-specific behaviors such as case sensitivity, symbolic links, and virtual directories when interpreting the path.\n - Ensure that path changes are atomic and consistent to avoid race conditions during concurrent FTP operations.\n - |-\n ## EXAMPLES:\n - `/uploads/images`\n - `documents/reports/2024`\n - `/home/user/data`\n - `./backup`\n - `../shared/resources`\n - `/var/www/html`\n - `projects/current`\n - |-\n ## IMPORTANT NOTES:\n - Access to the specified directory depends on the credentials used during FTP authentication.\n examples: [\"users/dave\", \"incoming/orders\", \"/var/ftp/upload\", \"EDI/850\"]\n fileName:\n type: string\n x-celigo-agent:\n - handlebars\n description: |-\n Name of the file written to the server, including its extension; do not include directory\n separators — the location comes from directoryPath. Supports handlebars placeholders such\n as items-{{timestamp}}.csv to generate a unique name per run; `{{timestamp}}` alone renders\n the full ISO-8601 time, colons included (`2026-10-23T08:25:21.743Z`), and a format gives\n a date stamp instead: `orders-{{timestamp 'YYYYMMDD'}}.csv`. There is no `date` helper. The\n template is rendered\n against the incoming record: a file-transfer import from an FTP source kept the source\n file's name with `{{record.fileMeta.fileName}}` set on `file.fileName` and this field left\n unset — other source providers expose the name under their own `fileMeta` field.\n x-celigo-ai-guidance:\n - |-\n When the user does not specify a file naming convention,\n default to a timestamped filename such as \"items-{{timestamp}}.csv\" so each run\n produces a unique file and avoids overwriting previous files.\n Use a fixed filename only when the user explicitly requests overwriting or a fixed name.\n - |-\n `fileMeta` is not a root key of this template's context (it is one only on\n file-provider export and lookup steps): `{{fileMeta.fileName}}` renders empty and the\n upload fails with `missing_s3_key` \"Can not process upload file without\n computedObject.fileName.\" — an error the platform labels source `internal` although the\n template is the cause. The run-time-proven way to keep the source file's name on a\n `blob: true` import is `{{record.fileMeta.fileName}}` on `file.fileName` with this field\n omitted (recorded runs of an FTP blob import set exactly that); the same template\n on this field has not been run. The field's editor preview (data version 1, its default)\n exposes the record as `data` —\n the `{blobKey, fileMeta}` record on a blob import, the page's record array otherwise —\n so `record.` previews empty there; judge by a run.\n - |-\n The name of the file to be accessed, transferred,\n or manipulated via FTP (File Transfer Protocol).\n This property specifies the exact filename, including its extension,\n uniquely identifying the file within the FTP directory.\n It is critical for accurately targeting the file during operations such as upload,\n download, rename, or delete.\n The filename must be precise and comply with the naming conventions and\n restrictions of the FTP server's underlying file system to avoid errors.\n Case sensitivity depends on the server's operating system,\n and special attention should be given to character encoding,\n especially when handling non-ASCII or international characters.\n The filename should not include any directory path separators,\n as these are managed separately in directory or path properties.\n - |-\n ## FIELD BEHAVIOR:\n - Identifies the specific file within the FTP server directory for various file operations.\n - Must include the file extension to ensure precise identification.\n - Case sensitivity is determined by the FTP server's operating system.\n - Excludes directory or path information; only the filename itself is specified.\n - Used in conjunction with directory or path properties to locate the full file path.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Validate that the filename contains only characters allowed by the FTP server's file system.\n - Ensure the filename is a non-empty, well-formed string without path separators.\n - Normalize case if required to match server-specific case sensitivity rules.\n - Properly handle encoding for special or international characters to prevent transfer errors.\n - Combine with directory or path properties to construct the complete file location.\n - Avoid embedding directory paths or separators within the filename property.\n - |-\n ## EXAMPLES:\n - \"report.pdf\"\n - \"image_2024.png\"\n - \"backup_2023_12_31.zip\"\n - \"data.csv\"\n - \"items-{{timestamp}}.csv\"\n - |-\n ## IMPORTANT NOTES:\n - The filename alone may not be sufficient to locate the file without the corresponding directory or path.\n - Different FTP servers and operating systems may have varying rules regarding case sensitivity.\n - Path separators such as \"/\" or \"\\\" must not be included in the filename.\n - Adherence to FTP server naming conventions and restrictions is essential to prevent errors.\n - Be mindful of potential filename length limits imposed by the FTP server or protocol.\n - |-\n ## DEPENDENCY CHAIN:\n - Depends on FTP directory or path properties to specify the full file location.\n - Utilized alongside FTP server credentials and connection settings.\n - May be influenced by file transfer mode (binary or ASCII) based on the file type.\n examples: [\"items-{{timestamp}}.csv\", \"orders-{{timestamp 'YYYYMMDD'}}.xml\", \"export.json\"]\n inProgressFileName:\n type: string\n x-celigo-agent:\n - handlebars\n x-celigo-ui-override: >-\n The FTP import form requires this once its \"Use temp file\" toggle is on\n (fieldDefinitions/resources/imports/ftp.js: 'ftp.inProgressFileName' requiredWhen\n ftp.useTempFile is true; blob mode 'ftp.blobInProgressFileName' requiredWhen\n ftp.blobUseTempFile is true). The toggles are form-only and are not saved — the form\n derives them from whether this field is set — so the API has no field to condition on\n and saves the import with or without the name.\n description: |-\n Temporary name the file carries while its upload is in progress, preventing other systems\n from processing a partially transferred file; it is renamed to its final name once the\n upload completes. Its presence is what the UI shows as \"Use temp file while upload in\n progress\": set it to turn the temporary name on, omit it to turn it off. Include a\n handlebars uniqueness token (e.g. {{timestamp}}) when one file is written per flow run.\n x-celigo-ai-guidance:\n - |-\n inProgressFileName is the designated temporary filename assigned to a file\n during the FTP upload process to clearly indicate that the file transfer is\n currently underway and not yet complete.\n This naming convention serves as a safeguard to prevent other systems or\n processes from prematurely accessing,\n processing, or locking the file before the upload finishes successfully.\n Typically, once the upload is finalized,\n the file is renamed to its intended permanent filename,\n signaling that it is ready for further use or processing.\n Utilizing an in-progress filename helps maintain data integrity,\n avoid conflicts,\n and streamline workflow automation in environments where files are continuously\n transferred and monitored.\n It plays a critical role in managing file state visibility,\n ensuring that incomplete or partially transferred files are not mistakenly processed,\n which could lead to data corruption or inconsistent system behavior.\n - |-\n ## FIELD BEHAVIOR:\n - Acts as a temporary marker for files actively being uploaded via FTP or similar protocols.\n - Prevents other processes, systems, or automated workflows from accessing or processing the file until the upload is fully complete.\n - The file is renamed atomically from the inProgressFileName to the final intended filename upon successful upload completion.\n - Helps manage concurrency, avoid file corruption, partial reads, or race conditions during file transfer.\n - Indicates the current state of the file within the transfer lifecycle, providing clear visibility into upload progress.\n - Supports cleanup mechanisms by identifying orphaned or stale temporary files resulting from interrupted uploads.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Choose a unique, consistent, and easily identifiable naming pattern distinct from final filenames to clearly denote the in-progress status.\n - Commonly use suffixes or prefixes such as \".inprogress\", \".tmp\", \"_uploading\", or similar conventions that are recognized across systems.\n - Ensure the FTP server and client support atomic renaming operations to avoid partial file visibility or race conditions.\n - Verify that the chosen temporary filename does not collide with existing files in the target directory to prevent overwriting or confusion.\n - Maintain consistent naming conventions across all related systems, scripts, and automation tools for clarity and maintainability.\n - Implement robust cleanup routines to detect and remove orphaned or stale in-progress files from failed or abandoned uploads.\n - Coordinate synchronization between upload completion and renaming to prevent premature processing or file locking issues.\n - |-\n ## EXAMPLES:\n - \"datafile.csv.inprogress\"\n - \"upload.tmp\"\n - \"report_20240615.tmp\"\n - \"file_upload_inprogress\"\n - \"transaction_12345.uploading\"\n examples: [\"items-{{timestamp}}.csv.inprogress\", \"upload.tmp\"]\n backupDirectoryPath:\n type: string\n x-celigo-agent:\n - handlebars\n description: |-\n Directory on the same server where a copy of each written file is retained after a\n successful import; if omitted, no server-side backup is kept. Supports static paths or\n handlebars templates.\n x-celigo-ai-guidance:\n - |-\n backupDirectoryPath specifies the file system path to the directory where backup\n files are stored during FTP operations.\n This directory serves as the designated location for saving copies of files\n before they are overwritten or deleted,\n ensuring that original data can be recovered if necessary.\n The path can be either absolute or relative, depending on the system context,\n and must conform to the operating system's file path conventions.\n It is essential that this path is valid, accessible,\n and writable by the FTP service or application performing the backups to\n guarantee reliable backup creation and data integrity.\n Proper configuration of this path is critical to prevent data loss and to\n maintain a secure and organized backup environment.\n - |-\n ## FIELD BEHAVIOR:\n - Specifies the target directory for storing backup copies of files involved in FTP transactions.\n - Activated only when backup functionality is enabled within the FTP process.\n - Requires the path to be valid, accessible, and writable to ensure successful backup creation.\n - Supports both absolute and relative paths, with relative paths resolved against a defined base directory.\n - If unset or empty, backup operations may be disabled or fallback to a default directory if configured.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Validate the existence of the directory and verify write permissions before initiating backups.\n - Normalize paths to handle trailing slashes, redundant separators, and platform-specific path formats.\n - Provide clear and actionable error messages if the directory is invalid, inaccessible, or lacks necessary permissions.\n - Support environment variables or placeholders for dynamic path resolution where applicable.\n - Enforce security best practices by restricting access to the backup directory to authorized users or processes only.\n - Ensure sufficient disk space is available in the backup directory to accommodate backup files without interruption.\n - Implement concurrency controls such as file locking or synchronization if multiple backup operations may occur simultaneously.\n - |-\n ## EXAMPLES:\n - \"/var/ftp/backups\"\n - \"C:\\\\ftp\\\\backup\"\n - \"./backup_files\"\n - \"/home/user/ftp_backup\"\n - |-\n ## IMPORTANT NOTES:\n - The backup directory must have adequate storage capacity to prevent backup failures due to insufficient space.\n - Permissions must allow the FTP process or service to create and modify files within the directory.\n - Backup files may contain sensitive or confidential data; appropriate security measures should be enforced to protect them.\n - Path formatting should align with the conventions of the underlying operating system to avoid errors.\n examples: [\"processed\", \"archive\", \"backup/{{timestamp 'YYYY-MM'}}\"]\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":21621,"content_sha256":"f418788c226e4653ba0bbcb786dce6c1962e5c2ecd8256c6c1f9163ae22aa8c3"},{"filename":"references/schemas/guardrail.yml","content":"GuardrailConfig:\n type: object\n description: |-\n Configuration for GuardrailImport adaptor type.\n\n Guardrails evaluate data flowing through integrations for safety and\n compliance. The `type` field selects which check to apply, and the\n corresponding sub-object (`aiAgent`, `pii`, or `moderation`) provides\n the configuration.\n\n A `_connectionId` on the parent import is only needed for BYOK\n `ai_agent` guardrails. In responses the server echoes the active type's\n sub-object and applies the `confidenceThreshold` default; it also\n returns inactive sibling sub-objects (e.g. `moderation: {categories: []}`\n on a `pii` guardrail, or a populated `pii` left over from a type switch),\n but only the active type's sub-object is meaningful. Legacy documents\n may carry a server-written default `aiAgent` stub on `pii`/`moderation`\n guardrails; current servers strip the inactive `aiAgent` on write.\n x-celigo-ai-guidance:\n - |-\n Guardrails are safety and compliance checks that can be applied\n to data flowing through integrations. Three mutually-exclusive\n types are supported — see `type` below for the full decision\n rule on which to pick:\n\n - **pii**: Detect (and optionally mask) personally identifiable\n information. Local deterministic classifier; fixed enum of\n entity types (`pii.entities`).\n - **moderation**: Detect harmful / inappropriate / policy-\n violating CONTENT. Local deterministic classifier; fixed enum\n of categories (`moderation.categories` — `sexual`, `hate`,\n `harassment`, `self_harm`, `violence`, `illicit`, etc.). Use\n category names exactly as listed in the enum — e.g. `hate`\n (not \"hate speech\"), `violence` (not \"violent content\"). Map\n natural-language synonyms (\"explicit\" → `sexual`, \"toxic\" →\n `harassment`, etc.) to the closest enum value rather than\n falling back to `ai_agent`.\n - **ai_agent**: LLM-evaluated judgement against custom\n instructions. Catch-all for intents that don't fit `pii` or\n `moderation` — domain-specific compliance, business-policy\n checks, data-quality assertions, etc. Pick this LAST, not as\n the default.\n - |-\n Prefer the cheaper / more deterministic types (`pii` > `moderation` >\n `ai_agent`) whenever the brief's intent fits.\n - Guardrail imports do not require a `_connectionId` (unless using BYOK for `ai_agent` type).\n - |-\n Never send `tools` anywhere under `aiAgent` on an `ai_agent` guardrail (`aiAgent.tools`,\n `aiAgent.openai.tools`, `aiAgent.litellm._overrides.gemini.tools`, `….anthropic.tools`) —\n any non-empty array fails the save with 422 `guardrail_tools_not_allowed` (\"Tools aren't\n supported on a guardrail AI agent configuration…\"), on create and on update. `prompts` and\n `resources` are accepted on save but never used at run time, so put the policy in the\n instructions. The server still echoes\n `openai.tools: []` on OpenAI guardrails.\n properties:\n type:\n type: string\n enum: [\"ai_agent\", \"pii\", \"moderation\"]\n x-enumDescriptions:\n ai_agent: Evaluate data using an AI model with custom instructions.\n pii: Detect personally identifiable information in data fields.\n moderation: Check content for harmful or inappropriate categories.\n description: |-\n The type of guardrail to apply. Each type requires its corresponding\n sub-configuration object (`aiAgent`, `pii`, or `moderation`).\n x-celigo-ai-guidance:\n - |-\n The type of guardrail to apply.\n Pick exactly one — `pii` / `moderation` / `ai_agent` are mutually exclusive at\n the type level (a brief that wants both runs as two guardrails in sequence,\n not one).\n - |-\n ## How to choose\n\n Apply these rules **in order** and pick the first match. Do\n NOT skip ahead to `ai_agent` just because the brief is in\n natural language — `pii` and `moderation` are also expressed\n as natural-language briefs. The decision is about WHAT the\n brief asks the guardrail to detect, not how the brief is\n worded.\n\n 1. **`pii`** — pick this when the brief names ANY personally\n identifiable information, OR when the brief names entity\n categories (SSN, email, phone, credit card, address, name,\n passport, license, account number, etc.) that match\n `pii.entities`. Examples that route here:\n \"detect SSN and credit card numbers\", \"mask customer\n emails and phone numbers\", \"scan for PII\".\n\n 2. **`moderation`** — pick this when the brief asks the\n guardrail to detect harmful, inappropriate, unsafe, or\n policy-violating CONTENT (as opposed to PII data fields)\n AND the intent maps to any value in `moderation.categories`\n (`sexual`, `hate`, `harassment`, `self_harm`, `violence`,\n `illicit`, etc.) — including via close synonyms. Apply\n the synonym mapping liberally:\n - \"explicit\", \"obscene\", \"vulgar\", \"profane\",\n \"inappropriate language\", \"swearing\", \"NSFW\" → typically\n maps to `sexual` and/or `harassment`\n - \"abusive\", \"insulting\", \"bullying\", \"toxic\" → `harassment`\n - \"hateful\", \"discriminatory\", \"racist\", \"sexist\" → `hate`\n - \"threatening\", \"intimidating\" → `harassment_threatening`\n or `violence`\n - \"graphic violence\", \"gore\" → `violence_graphic`\n - \"suicide\", \"self-injury\" → `self_harm`\n - \"drugs\", \"weapons\", \"illegal activity\" → `illicit`\n Briefs like \"screen email body for explicit language\",\n \"block toxic chat messages\", \"flag harassment in support\n tickets\", \"moderate user-generated content for unsafe\n material\" all route to `moderation`. When the brief uses\n a synonym, populate `moderation.categories` with the\n closest enum value(s) — do NOT invent new category\n strings; the enum is fixed.\n\n 3. **`ai_agent`** — pick this ONLY when the brief asks for\n judgement that doesn't fit `pii` or `moderation`. Typical\n triggers: domain-specific compliance checks (HIPAA / SOX\n rule evaluation), business-policy validation (price\n bounds, approval thresholds), data-quality assertions\n (required fields populated, values plausible), or any\n \"evaluate against these custom rules\" framing where the\n rules are domain-specific and have no `pii.entities` /\n `moderation.categories` analogue. If you can map the\n brief's intent to a moderation category (even via a\n synonym above), use `moderation` instead — `ai_agent` is\n the catch-all, not the default.\n\n Tie-breaker for borderline cases: prefer the more specific\n type (`pii` > `moderation` > `ai_agent`). `pii` and\n `moderation` are local deterministic classifiers with fixed\n enums — they're cheaper, faster, and more predictable than\n `ai_agent` (which calls a real LLM per record). Default to\n the cheaper path whenever the intent fits.\n - |-\n ## Sub-configuration requirements (one per type)\n\n - **ai_agent** requires the `aiAgent` sub-configuration.\n - **pii** requires the `pii` sub-configuration with at least\n one entity from `pii.entities`.\n - **moderation** requires the `moderation` sub-configuration\n with at least one category from `moderation.categories`.\n confidenceThreshold:\n type: number\n minimum: 0\n maximum: 1\n default: 0.7\n x-celigo-ui-override: >-\n Required by the guardrail form (confidenceThreshold is required:true once a type is\n selected, default 0.7). Encoded to mirror the form so builders produce connectable\n configurations.\n description: |-\n Confidence threshold (0 to 1). Detections below this threshold are\n ignored. Lower values catch more issues but increase false positives.\n x-celigo-ai-guidance:\n - Confidence threshold for guardrail detection (0 to 1).\n - |-\n Only detections with confidence at or above this threshold will be flagged.\n Lower values catch more potential issues but may increase false positives.\n examples: [0.7, 0.5, 0.9]\n aiAgent:\n type: object\n description: |-\n AI agent check configuration; set when `type` is `ai_agent`. On\n `pii`/`moderation` guardrails a legacy server-written stub may\n appear here — it is inert, and current servers strip it on write.\n pii:\n type: object\n required: [entities]\n description: |-\n PII detection configuration. Required when `type` is `pii`.\n x-celigo-ai-guidance:\n - Configuration for PII (Personally Identifiable Information) detection.\n - |-\n Required when `guardrail.type` is \"pii\".\n At least one entity type must be specified.\n properties:\n entities:\n type: array\n description: |-\n PII entity types to detect. When `type` is `pii`, at least one\n entry is required; the inactive sibling on other guardrail types\n may be served with an empty list.\n x-celigo-ai-guidance:\n - PII entity types to detect in the data.\n - At least one entity must be specified when using PII guardrails.\n items:\n type: string\n enum:\n - credit_card_number\n - card_security_code_cvv_cvc\n - cryptocurrency_wallet_address\n - date_and_time\n - email_address\n - iban_code\n - bic_swift_bank_identifier_code\n - ip_address\n - location\n - medical_license_number\n - national_registration_number\n - persons_name\n - phone_number\n - url\n - us_bank_account_number\n - us_drivers_license\n - us_itin\n - us_passport_number\n - us_social_security_number\n - uk_nhs_number\n - uk_national_insurance_number\n - spanish_nif\n - spanish_nie\n - italian_fiscal_code\n - italian_drivers_license\n - italian_vat_code\n - italian_passport\n - italian_identity_card\n - polish_pesel\n - finnish_personal_identity_code\n - singapore_nric_fin\n - singapore_uen\n - australian_abn\n - australian_acn\n - australian_tfn\n - australian_medicare\n - indian_pan\n - indian_aadhaar\n - indian_vehicle_registration\n - indian_voter_id\n - indian_passport\n - korean_resident_registration_number\n x-enumDescriptions:\n credit_card_number: Credit or debit card number (e.g., Visa, Mastercard, Amex).\n card_security_code_cvv_cvc: Card security code printed on credit or debit cards (CVV/CVC).\n cryptocurrency_wallet_address: Cryptocurrency wallet address (e.g., Bitcoin, Ethereum).\n date_and_time: Date and/or time values that could identify an individual.\n email_address: Email address in standard format.\n iban_code: International Bank Account Number used for cross-border payments.\n bic_swift_bank_identifier_code: BIC/SWIFT code identifying a specific bank for international transfers.\n ip_address: IPv4 or IPv6 network address.\n location: Physical location or geographic coordinates.\n medical_license_number: Medical professional license or registration number.\n national_registration_number: National identification or registration number.\n persons_name: Full or partial name of a person.\n phone_number: Telephone number in any format.\n url: Web URL or URI.\n us_bank_account_number: United States bank account number.\n us_drivers_license: United States driver's license number.\n us_itin: United States Individual Taxpayer Identification Number.\n us_passport_number: United States passport number.\n us_social_security_number: United States Social Security Number (SSN).\n uk_nhs_number: United Kingdom National Health Service number.\n uk_national_insurance_number: United Kingdom National Insurance number.\n spanish_nif: Spanish tax identification number (NIF).\n spanish_nie: Spanish foreigner identification number (NIE).\n italian_fiscal_code: Italian fiscal code (Codice Fiscale).\n italian_drivers_license: Italian driver's license number.\n italian_vat_code: Italian VAT identification number (Partita IVA).\n italian_passport: Italian passport number.\n italian_identity_card: Italian national identity card number.\n polish_pesel: Polish national identification number (PESEL).\n finnish_personal_identity_code: Finnish personal identity code (henkilotunnus).\n singapore_nric_fin: Singapore National Registration Identity Card or Foreign Identification Number.\n singapore_uen: Singapore Unique Entity Number for business registration.\n australian_abn: Australian Business Number.\n australian_acn: Australian Company Number.\n australian_tfn: Australian Tax File Number.\n australian_medicare: Australian Medicare card number.\n indian_pan: Indian Permanent Account Number for tax purposes.\n indian_aadhaar: Indian Aadhaar unique identity number.\n indian_vehicle_registration: Indian vehicle registration number.\n indian_voter_id: Indian voter identification card number.\n indian_passport: Indian passport number.\n korean_resident_registration_number: South Korean resident registration number.\n examples:\n - [\"email_address\", \"phone_number\", \"persons_name\"]\n - [\"us_social_security_number\", \"credit_card_number\"]\n mask:\n type: boolean\n default: false\n description: |-\n When true, the result also carries `masked`: the evaluated input with each detected\n value replaced by asterisks. The record itself is unchanged. With whole-record input\n (no `mapping`, and no top-level `text` field on the record), `masked` is the record\n serialized as one JSON string. When false, PII is flagged only.\n x-celigo-ai-guidance:\n - |-\n Masking doesn't rewrite the record. To pass masked text on, select the field as the\n `text` input (`mapping`) and response-map `masked` back onto that field on the\n guardrail's flow step (`{\"extract\": \"masked\", \"generate\": \"\u003cfield>\"}`).\n moderation:\n type: object\n required: [categories]\n description: |-\n Content moderation configuration. Required when `type` is `moderation`.\n x-celigo-ai-guidance:\n - |-\n Required when `guardrail.type` is \"moderation\".\n At least one category must be specified.\n properties:\n categories:\n type: array\n description: |-\n Content moderation categories to check. When `type` is\n `moderation`, at least one entry is required; the inactive\n sibling on other guardrail types may be served with an empty\n list.\n x-celigo-ai-guidance:\n - At least one category must be specified when using moderation guardrails.\n items:\n type: string\n enum:\n - sexual\n - sexual_minors\n - hate\n - hate_threatening\n - harassment\n - harassment_threatening\n - self_harm\n - self_harm_intent\n - self_harm_instructions\n - violence\n - violence_graphic\n - illicit\n - illicit_violent\n x-enumDescriptions:\n sexual: Content depicting sexual activity or explicit sexual material.\n sexual_minors: Sexual content involving minors.\n hate: Content expressing hatred toward a group based on protected characteristics.\n hate_threatening: Hateful content that includes threats of violence or serious harm.\n harassment: Content that targets, intimidates, or bullies an individual.\n harassment_threatening: Harassment content that includes threats of violence or serious harm.\n self_harm: Content that promotes or depicts self-harm behaviors.\n self_harm_intent: Content expressing intent to engage in self-harm.\n self_harm_instructions: Content providing instructions for self-harm methods.\n violence: Content depicting or promoting physical violence.\n violence_graphic: Graphic or gory depictions of violence or injury.\n illicit: Content promoting illegal activities or unlawful behavior.\n illicit_violent: Content promoting illegal activities that involve violence.\n examples:\n - [\"hate\", \"violence\", \"harassment\"]\n - [\"sexual\", \"self_harm\", \"illicit\"]\n required:\n - type\n - confidenceThreshold\n if:\n required: [type]\n properties:\n type:\n const: pii\n then:\n required: [pii]\n properties:\n pii:\n required: [entities]\n properties:\n entities:\n minItems: 1\n else:\n if:\n required: [type]\n properties:\n type:\n const: moderation\n then:\n required: [moderation]\n properties:\n moderation:\n required: [categories]\n properties:\n categories:\n minItems: 1\n else:\n required: [aiAgent]\n properties:\n aiAgent:\n # Guardrails reject every `tools` array (422 guardrail_tools_not_allowed), so the\n # ai_agent branch uses the guardrail spec's tools-free copy of the provider blocks\n # rather than the tool-bearing AiAgentConfig used by AiAgentImport.\n $ref: \"../guardrail-schemas/guardrail-config.yml#/GuardrailAiAgentConfig\"\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":18900,"content_sha256":"d9466ea8ce0587336ca0226b2aed77486165a4dfa0daa477f4a9878f1f55b2ce"},{"filename":"references/schemas/http.yml","content":"Http:\n type: object\n description: |-\n Configuration for HTTP imports.\n Required whenever the connection referenced by _connectionId has type http, and on\n `NetSuiteHTTPImport` (NetSuite's REST APIs on a `netsuite` connection, where `nsDomainType`\n supplies the host).\n x-celigo-ai-guidance:\n - |-\n IMPORTANT:\n When the _connectionId field points to a connection where the type is http,\n this object MUST be populated for the import to function properly.\n This is a required configuration for all HTTP based imports,\n as determined by the connection associated with the import.\n - |-\n A `NetSuiteHTTPImport` uses this same object on a netsuite connection: each `relativeURI`\n entry carries the full path from `/services/rest/record/v1/\u003ctype>` (SuiteTalk) or\n `/app/site/hosting/restlet.nl?script=\u003cid>&deploy=\u003cn>` (custom RESTlet); the host and the\n token signature come from the connection.\n properties:\n sendPostMappedData:\n type: boolean\n default: true\n description: |-\n When true, the request body is the record as reshaped by the import's mapping step. When false, the original pre-mapped record is sent instead, bypassing the mappings for the payload.\n isRest:\n type: boolean\n description: |-\n Internal flag indicating the import was built with the legacy REST form rather than the unified HTTP framework. Set by the form; not configured directly.\n strictHandlebarEvaluation:\n type: boolean\n description: |-\n Intended to make handlebars expressions that reference a missing field raise an error instead of rendering an empty string. In live runs on an HTTP import it does not have that effect on `http.body` templates or mapping expressions — an undefined path still renders empty and the record succeeds with the flag true — and `relativeURI` / `headers[].value` templates already fail on an undefined path regardless of this flag. Not a safeguard against a wrong template root or a misspelled path.\n x-celigo-ai-guidance:\n - |-\n Do not rely on this flag to catch a mis-rooted or misspelled path. Live-verified\n 2026-09-24 on an HTTP import in real flow runs with the flag ``true``: an undefined\n ``{{record.\u003cfield>}}`` in ``http.body`` and an undefined ``{{\u003cfield>}}`` in a mapping\n expression both rendered empty and the record succeeded — the same outcome as with\n the flag ``false`` or unset. Relative URI and header templates fail on an undefined\n path regardless of this flag. To catch a wrong root, check the template against the\n record's shape (``record`` for an object, ``rows`` for a grouped record,\n ``batch_of_records`` when ``batchSize`` exceeds 1) before saving.\n formType:\n type: string\n enum:\n - assistant\n - http\n - rest\n - graph_ql\n - assistant_graphql\n x-enumDescriptions:\n assistant: Assistant-guided form for systems with a Celigo connector available.\n http: Standard HTTP import form for generic HTTP endpoints.\n rest: Legacy REST form type for older imports not supported by the new HTTP framework.\n graph_ql: GraphQL import form for GraphQL API endpoints.\n assistant_graphql: Assistant-guided GraphQL form for systems with a connector that supports GraphQL.\n description: Determines the UI form and configuration experience for this import.\n x-celigo-ai-guidance:\n - Use `assistant` as the default value if there is a connector available for the system.\n - |-\n Use `assistant_graphql` as the default value if there is a connector available\n for the system and the connector supports GraphQL.\n - |-\n The value selects the editor and changes one thing in a GraphQL request: under\n `graph_ql` and `assistant_graphql`, a `variables` member stored as a JSON string is\n parsed and sent as an object, while under `http` it is sent as the string; an object\n `variables` goes out unchanged under `graph_ql` and `http`. Live-verified 2026-10-22 with import\n previews against an echo endpoint: a string `variables` under `graph_ql`,\n `assistant_graphql` and `http`, and an object `variables` under `graph_ql` and `http`.\n - |-\n Each `http.body[]` entry stays a template STRING — a\n stringified `{\"query\", \"operationName\", \"variables\"}` envelope, one per operation for the\n update/create pair (`http.body[0]` / `http.body[1]`); never an object entry, which is stored\n as-is and does not render. The GraphQL form parses that string to show each member in its\n own field and re-stringifies it on save; an entry that is not parseable JSON at rest renders\n those fields empty, so keep templated variables inside a JSON string value. When\n `variables` is itself a JSON string, the string values inside it sit two JSON levels\n deep: write them as `\\\"{{jsonEncode record.x}}\\\"` with double braces, because a plain\n `{{record.x}}`, or a triple-brace `jsonEncode`, fails a record whose value holds a quote\n with 422 `invalid_json` before the request is sent. An object or array there is\n `{{jsonSerialize record.x}}`, also with double braces; `{{{jsonSerialize record.x}}}`\n fails the same way (live-verified 2026-10-22).\n - |-\n Create GraphQL imports on a `graph_ql` connection (or a connector connection bound to its\n GraphQL API): that setup offers the Form view toggle between the GraphQL editor and the\n raw HTTP form, and the GraphQL form has no relative URI field — the connection `baseURI`\n is expected to be the GraphQL endpoint.\n examples:\n - assistant\n - http\n - rest\n - graph_ql\n - assistant_graphql\n type:\n type: string\n enum:\n - file\n - records\n x-enumDescriptions:\n file: Import handles raw file content such as binary data, PDFs, or images.\n records: Import handles structured record data such as JSON objects or XML records.\n description: |-\n Controls whether the import sends structured record data or raw file content.\n Most HTTP imports use records; use file only when importing raw file content that will be processed downstream.\n x-celigo-ai-guidance:\n - Specifies the type of data being imported via HTTP.\n examples:\n - records\n - file\n requestMediaType:\n type: string\n enum:\n - xml\n - json\n - csv\n - urlencoded\n - form-data\n - octet-stream\n - plaintext\n x-enumDescriptions:\n xml: XML request body sent as application/xml.\n json: JSON request body sent as application/json, the most common format.\n csv: CSV data sent as text/csv.\n urlencoded: URL-encoded form data sent as application/x-www-form-urlencoded.\n \"form-data\": Multipart form data for file uploads and mixed content; the body is authored as a JSON array of parts and the platform generates the MIME boundary at send time.\n \"octet-stream\": Raw binary data sent as application/octet-stream.\n plaintext: Plain text content sent as text/plain.\n description: |-\n Content type used to serialize the request body sent to the target API.\n Most REST APIs use json.\n x-celigo-ai-guidance:\n - |-\n ``form-data`` changes the body contract entirely: author ``http.body``\n as a JSON array of ``{name, value, type}`` parts (see the multipart\n guidance on ``body``) instead of a Handlebars record template. Do not\n include a MIME boundary — integrator.io adds it automatically when\n the data is sent.\n examples:\n - json\n - xml\n _httpConnectorEndpointIds:\n type: array\n items:\n type: string\n format: objectId\n x-celigo-refModel: httpconnectorendpoints\n description: |-\n The connector endpoints this import calls, one per entry of `relativeURI` / `method` (for\n example a CREATE and an UPDATE endpoint), shown in the UI as the selected endpoints. On save the\n platform derives them (with the version and resource ids) from each `relativeURI` / `method`\n pair that matches an endpoint of the connector the connection is bound to, and replaces ids\n that belong to another connector; ids for the connection's own connector are kept as sent, and\n when nothing matches, or the connection has no connector, whatever was sent is kept.\n x-celigo-ai-guidance:\n - |-\n To build an import on a prebuilt connector, bind the CONNECTION (`http._httpConnectorId`) and\n write each operation's `relativeURI` and `method` into the positional arrays; the platform links\n the import to those endpoints on save, so the pointer fields may be omitted. Read the saved\n import back: missing entries mean a URI matched no endpoint of the connection's connector. Set\n `_httpConnectorVersionId`, `_httpConnectorResourceId` and `_httpConnectorEndpointIds` together\n only to pin same-connector endpoints the URIs would not match. Imports use this plural field;\n the singular `_httpConnectorEndpointId` is dropped on write. Endpoint discovery through the\n published operations: `GET /v1/httpconnectors/{_id}`, `…/{_id}/httpconnectorresources`,\n `…/{_id}/{_httpConnectorResourceId}/httpconnectorendpoints`, and\n `…/{_id}/{_httpConnectorResourceId}/{_id}` for an endpoint's `method` and `relativeURI`.\n Verified against the live API 2026-09-21.\n blobFormat:\n type: string\n enum:\n - utf8\n - ucs2\n - utf-16le\n - ascii\n - binary\n - base64\n - hex\n x-enumDescriptions:\n utf8: UTF-8 Unicode encoding, the universal default.\n ucs2: UCS-2 encoding for 16-bit Unicode characters.\n \"utf-16le\": UTF-16 Little Endian encoding.\n ascii: ASCII encoding for basic Latin characters (7-bit).\n binary: Raw binary encoding with no character conversion.\n base64: Base64 encoding for representing binary data as ASCII text.\n hex: Hexadecimal encoding representing each byte as two hex digits.\n description: |\n Character encoding format for blob/binary data imports.\n Only relevant when type is \"file\" or when handling binary content.\n examples:\n - utf8\n - base64\n batchSize:\n type: integer\n description: |-\n Maximum number of records submitted per HTTP request, which affects throughput and API rate limiting.\n REST services typically send one record per request; raise it only for batch endpoints or RPC/XML services that accept multiple records, consulting the target API documentation for the optimal value.\n Above 1, the body, relative URI, and header templates render once per request against `batch_of_records` — each entry carrying that record as `record` (or `rows` for a grouped record) — and a root-level `record` or `rows` is undefined.\n\n The default varies by API.\n x-celigo-ai-guidance:\n - |-\n A batched import's templates iterate ``{{#each batch_of_records}}`` and address\n ``this.record.\u003cfield>`` (``this.rows.0.\u003cfield>`` for grouped records), or iterate\n the bare records as ``{{#each data}}``; a ``{{record.\u003cfield>}}`` left at the root\n renders empty in the body and fails the request in a relative URI or header. With\n the default batch size the shape root is ``record`` / ``rows``, ``data`` holds that\n one record, and ``batch_of_records`` is undefined.\n examples:\n - 1\n - 10\n - 100\n successMediaType:\n type: string\n enum:\n - xml\n - json\n - plaintext\n x-enumDescriptions:\n xml: Expect XML-formatted success responses.\n json: Expect JSON-formatted success responses, the most common format.\n plaintext: Expect plain text success responses.\n description: Media type expected in successful responses from the target API. Most APIs return json.\n examples:\n - json\n - xml\n requestType:\n type: array\n items:\n type: string\n enum:\n - CREATE\n - UPDATE\n x-enumDescriptions:\n CREATE: Create new records in the destination system.\n UPDATE: Update existing records in the destination system.\n description: |-\n Operation type each request performs; for composite (upsert) imports, include both values.\n The array is positionally aligned with the relativeURI and method arrays — each index maps to one operation — and the existingExtract field determines which operation runs at runtime.\n x-celigo-ai-guidance:\n - Specifies the type of operation(s) this import performs.\n - |-\n Example composite upsert:\n ```json\n \"requestType\": [\"UPDATE\", \"CREATE\"],\n \"method\": [\"PUT\", \"POST\"],\n \"relativeURI\": [\"/customers/{{{data.0.customerId}}}.json\", \"/customers.json\"],\n \"existingExtract\": \"customerId\"\n ```\n examples:\n - [CREATE]\n - [UPDATE]\n - [UPDATE, CREATE]\n errorMediaType:\n type: string\n enum:\n - xml\n - json\n - plaintext\n x-enumDescriptions:\n xml: Expect XML-formatted error responses.\n json: Expect JSON-formatted error responses, the most common format.\n plaintext: Expect plain text error messages.\n description: Media type expected in error responses from the target API. Most APIs return json.\n examples:\n - json\n - xml\n followRedirects:\n type: boolean\n default: true\n description: |-\n When explicitly false, 3xx responses are not followed — the redirect\n response itself (status code, `Location` header, body) is what the\n import records as the result. Omitted or true follows allowed\n redirects (the default behavior).\n examples: [false]\n maxRedirects:\n type: integer\n minimum: 1\n maximum: 10\n description: |-\n Caps how many consecutive 3xx redirects are followed. Only applies\n when `followRedirects` is not false; to not follow at all, set\n `followRedirects: false` rather than `maxRedirects: 0`. Decimal\n values are rejected on save.\n examples: [3]\n relativeURI:\n type: array\n items:\n type: string\n x-celigo-agent:\n - handlebars\n x-celigo-handlebars-context: pre-mapped\n description: |-\n Relative URI path for each import request, as an array of strings (never an object); values can include Handlebars expressions for dynamic segments.\n Templates render against the pre-mapped record — the original input before the Import's mapping step — because URI construction usually needs business identifiers that mappings may rename or remove.\n For composite (upsert) imports, the array is positionally aligned with method and requestType, and existingExtract decides the index used at runtime: the UPDATE index when its field has a value, the CREATE index when it is empty or missing.\n Reference the existing record ID in the UPDATE URI with `{{{data.0.fieldName}}}` (triple braces with the data.0 prefix).\n Use separate array elements per operation rather than `{{#if}}` conditionals inside a single URI.\n URI templates render strictly: a path that is undefined on the record — such as `record.\u003cfield>` on a grouped record, whose root is `rows` — fails the request instead of rendering empty.\n A field the record lacks, or a `{{lookup.\u003cname>}}` whose lookup found nothing, fails that record before anything is sent, with `handlebars_template_parse_error` (source `application`, classification `parse`): `Failed to generate request url from template: \u003cURL>. Details: \"\u003cfield>\" not defined in the model.` That is missing data, not template syntax, and a retry of the same record fails the same way: send records that may lack the destination id through a composite UPDATE/CREATE import (`existingExtract`), or skip them with `ignoreMissing` and `ignoreExtract`.\n A field that holds an empty string renders empty, and the request is sent without the value. A field that holds null renders the text `null` (a URI of `/orders/{{record.id}}` is sent as `/orders/null`), and a field under an object that is null fails the record like a missing one (`\"\u003cfield>\" not defined in null`).\n x-celigo-ai-guidance:\n - '**CRITICAL: This MUST be an array of strings, NOT an object.**'\n - |-\n ## ARRAY SHAPE IS REQUIRED EVEN FOR FULLY-STATIC URLs\n\n The array wrap is a structural requirement of the import\n document, not a Handlebars/templating concern. The platform\n rejects scalar strings here regardless of whether the URI\n uses ``{{...}}`` templating:\n\n ✅ ``\"relativeURI\": [\"/fba/outbound/2020-07-01/fulfillmentOrders\"]``\n ✅ ``\"relativeURI\": [\"/customers/{{{record.customerId}}}\"]``\n ❌ ``\"relativeURI\": \"/fba/outbound/2020-07-01/fulfillmentOrders\"`` ← scalar string\n ❌ ``\"relativeURI\": \"/customers/{{{record.customerId}}}\"`` ← scalar string\n\n The array shape exists to support the composite create+update\n pattern (see \"Composite/upsert imports\" below), where the\n ``relativeURI``, ``method``, and ``requestType`` arrays are\n positionally aligned. Single-operation imports still use the\n same shape -- a 1-element array -- so the runtime never has\n to special-case the scalar form.\n - |-\n ## Single-operation imports (one element):\n ```json\n \"relativeURI\": [\"/api/v1/customers\"]\n ```\n - |-\n ## Composite/upsert imports (multiple elements — positionally aligned):\n For imports with both CREATE and UPDATE in requestType, relativeURI, method,\n and requestType arrays are **positionally aligned**. Each index corresponds\n to one operation. The existingExtract field determines which index is used at\n runtime: if the existingExtract field has a value → use the UPDATE index;\n if empty/missing → use the CREATE index.\n\n Example of a composite upsert:\n ```json\n \"relativeURI\": [\"/customers/{{{data.0.shopifyCustomerId}}}.json\", \"/customers.json\"],\n \"method\": [\"PUT\", \"POST\"],\n \"requestType\": [\"UPDATE\", \"CREATE\"],\n \"existingExtract\": \"shopifyCustomerId\"\n ```\n Here index 0 is the UPDATE operation (PUT to a specific customer) and index 1 is\n the CREATE operation (POST to the collection endpoint).\n\n **IMPORTANT**: For composite imports, use the `{{{data.0.fieldName}}}` handlebars\n syntax (triple braces with data.0 prefix) to reference the existing record ID in\n the UPDATE URI. Do NOT use `{{#if}}` conditionals in a single URI — use separate\n array elements instead.\n - |-\n ## WRONG FORMAT (DO NOT DO THIS):\n ```json\n \"relativeURI\": {\"/api/v1/customers\": \"CREATE\"}\n ```\n ```json\n \"relativeURI\": [\"{{#if id}}/customers/{{id}}.json{{else}}/customers.json{{/if}}\"]\n ```\n ```json\n // STRINGIFIED ARRAY — the multi-entry array got JSON-encoded\n // into a single string and re-wrapped in a 1-element array.\n // The save-time normalizer detects and unwraps this exact\n // shape; do not emit it deliberately.\n \"relativeURI\": [\"[\\\"/customers/{{{data.0.id}}}\\\",\\\"/customers\\\"]\"]\n ```\n ⚠️ On save the platform derives the connector pointer fields\n (``_httpConnectorVersionId`` / ``_httpConnectorResourceId`` /\n ``_httpConnectorEndpointIds``) from each ``relativeURI`` /\n ``method`` pair that matches an endpoint of the connector the\n connection is bound to (path parameters and query strings still\n match); pointers you send for that connector are kept, pointers\n from another connector are replaced, and a URI that matches\n nothing leaves whatever you sent in place — nothing is unset\n (verified 2026-09-21 on create, on PUT, and after a re-save). Keep\n each entry equal to the endpoint you mean so the pointer and the\n request agree. Get the URI shape right.\n - |-\n On a GraphQL-form import (`formType` `graph_ql` / `assistant_graphql`) the form owns\n this field: it shows no relative-URI field and writes `[\"/\"]` back on every save in the\n editor, because the request path is expected to come from the connection's `baseURI` —\n a `graph_ql` connection, or a connector connection bound to its GraphQL API. A path set\n here through the API on such an import does reach the wire, but it is invisible in the\n editor and reverts to `[\"/\"]` the next time the import is saved there; when the\n connection's base URI stops at an API root, move the import to a connection that\n supplies the GraphQL endpoint instead of patching this field.\n examples:\n - [\"/api/v1/customers\"]\n - [\"/customers/{{{data.0.shopifyCustomerId}}}.json\", \"/customers.json\"]\n method:\n type: array\n items:\n type: string\n enum:\n - GET\n - PUT\n - POST\n - PATCH\n - DELETE\n x-enumDescriptions:\n GET: Retrieve data from the server (rarely used for imports).\n PUT: Replace an existing resource entirely.\n POST: Create a new resource, the most common method for imports.\n PATCH: Partially update an existing resource.\n DELETE: Remove a resource from the server.\n description: |-\n HTTP method used for each import request.\n For composite (upsert) imports, the array is positionally aligned with relativeURI and requestType — e.g. `[\"PUT\", \"POST\"]` with `requestType: [\"UPDATE\", \"CREATE\"]` uses PUT for updates and POST for creates.\n x-celigo-ai-guidance:\n - |-\n For composite/upsert imports,\n specify multiple methods positionally aligned with relativeURI and requestType arrays.\n Each index maps to one operation.\n examples:\n - [POST]\n - [PUT, POST]\n _httpConnectorVersionId:\n type: string\n format: objectId\n description: |-\n Pins the import to a version of the published HTTP connector its connection is bound to\n (`connection.http._httpConnectorId`), so the UI renders that connector's endpoint forms instead\n of hand-authored requests. Linked on save together with `_httpConnectorResourceId` and\n `_httpConnectorEndpointIds` — see that field for the rule.\n _httpConnectorResourceId:\n type: string\n format: objectId\n x-celigo-refModel: httpconnectorresources\n description: |-\n The connector resource (a group of related endpoints, such as Orders) that this import's\n endpoints belong to. Linked on save together with `_httpConnectorEndpointIds` — see that field\n for the rule.\n _httpConnectorEndpointId:\n type: string\n format: objectId\n readOnly: true\n x-celigo-refModel: httpconnectorendpoints\n description: |-\n Not used by imports: values sent on create or update are dropped, and the platform does not\n populate it. Imports bind connector endpoints per operation through `_httpConnectorEndpointIds`.\n body:\n type: array\n items:\n type: [string, 'null']\n description: |-\n Request body template for the import — an array of Handlebars template strings\n positionally aligned with the `method` array (e.g. `[\"{{{record}}}\"]`). A `null`\n entry means no template for the method at that position. Each entry is a string:\n write a JSON body as a template string (`[\"{\\\"id\\\": \\\"{{record.id}}\\\"}\"]`), not as\n a JSON object, because an object entry does not render as JSON at runtime.\n Leave undefined for standard imports — the Import's mapping step transforms the source record and the mapped data is sent as the request body automatically.\n Set it only when the target API requires XML, SOAP, or a custom structure that mappings cannot produce, or when the user explicitly specifies a request body.\n When set, the mapping step still runs first and the template renders against the post-mapped record — use `{{record.\u003cmappedField>}}` to reference mapping outputs.\n Hand-templated bodies are harder to maintain and debug than mappings; when in doubt, leave this field undefined.\n\n Custom envelopes can combine static header or account fields with per-message\n field references inside a loop.\n\n Static wrappers around the mapped record are appropriate for message-bus class envelopes,\n HMAC signature wrappers, integration batch headers,\n named single-record array wrappers, and similar destination-required structures.\n\n Batch templates can wrap an envelope, such as a header, version, or marketplace,\n around serialized records.\n\n Use this pattern when each message needs a small number of fields rather than the whole record.\n x-celigo-ai-guidance:\n - |-\n Handlebars template that produces the HTTP request body.\n Renders against the POST-mapped record at runtime:\n the Import's mapping step runs first and shapes the source record,\n then this template renders against that mapped record.\n Reference the mapped record with the ``record`` root when wrapping it (e.g.\n ``{\"requestData\": {{{jsonSerialize record}}} }``).\n Keep a literal ``}`` apart from a closing ``}}`` or ``}}}`` (a space is enough): the save\n accepts ``{{{jsonSerialize record}}}}``, but every record then fails with\n ``handlebars_template_parse_error``.\n Avoid bare ``{{{jsonSerialize record}}}`` with no wrapper — leave the field undefined instead.\n Also avoid wrapping a single post-mapped scalar field in ``jsonSerialize`` (e.g.\n ``{{{jsonSerialize record.\u003cscalar>}}}``) — that pattern is brittle.\n - |-\n In a JSON body, write a string value with double braces inside the quotes\n (``\"{{record.name}}\"``): double braces JSON-escape the value at run time, with no HTML\n escaping. Triple braces insert the raw text, so a value holding a double quote,\n backslash, newline or tab fails that record with 422 ``invalid_json`` before anything\n is sent. Double braces write a null value -- a null field, or any field under an\n object that is null -- as the text ``null``, so ``\"{{record.customer.name}}\"`` sends\n the string \"null\" (triple braces rendered ``{{{record.customer.name}}}`` empty,\n whether ``customer`` or its ``name`` was null). Guard a nullable field with\n ``{{#if record.customer.name}}{{record.customer.name}}{{/if}}`` or\n ``{{#if record.customer}}{{record.customer.name}}{{/if}}``: both render empty for a\n null field and for a null object. The Handlebars processor renders the unguarded null\n empty, so judge it by an import preview or a run.\n - |-\n Every request context carries two data roots: ``data`` — the list of records in\n the request, always present (Pattern B) — and ONE shape-specific root that follows\n the record entering the step. An object record is ``record``; a GROUPED record (an\n array of rows — a ``groupByFields`` export, file key columns, a NetSuite grouped\n saved search) is ``rows`` (``{{rows.0.\u003cfield>}}``, ``{{#each rows}}``,\n ``{{{jsonSerialize rows}}}``) and ``record.*`` is undefined on it; when\n ``batchSize`` exceeds 1 the shape root is ``batch_of_records`` and neither ``record``\n nor ``rows`` exists at the top level. Inside ``data`` a grouped record is itself an\n array element (``{{#each data}}{{#each this}}…{{/each}}{{/each}}``). An object record\n that carries its own ``rows`` field is still ``record`` — ``{{#each record.rows}}``.\n Here an undefined path renders empty — and ``strictHandlebarEvaluation: true`` does\n not change that (live-verified 2026-09-24 in real flow runs: an undefined\n ``{{record.\u003cfield>}}`` in the body rendered empty and the record succeeded); in\n ``relativeURI`` and ``headers[].value`` templates an undefined path always fails the\n request — a wrong root there is a runtime failure, not a blank. Verify the body's\n root against the record shape; no flag will catch it.\n - |-\n ## LEAVE UNDEFINED for the most common case\n\n When in doubt, leave this field undefined. The platform sends\n the post-mapped record as JSON automatically — the body template\n only adds value when there's something static around the record\n (a message envelope, signature wrap, batch header, etc.) that\n the mapper shouldn't have to know about. If your template\n would be bare ``{{{jsonSerialize record}}}`` with nothing\n around it, you don't need a template at all.\n\n Leave undefined for:\n - REST APIs accepting JSON CRUD payloads (the destination\n already wants exactly the post-mapped record shape).\n - Any case where the destination wants the mapped record as-is.\n\n GraphQL destinations are different: the body is the stringified\n ``{query, operationName, variables}`` envelope described under\n ``formType``, with the post-mapped record passed in\n ``variables``.\n - |-\n ## SET THIS FIELD when one of the patterns below applies\n\n Choose the pattern that fits the destination's shape.\n\n ### Pattern A — ``{{{jsonSerialize record}}}`` inside a static wrapper\n\n Use this ONLY when the destination requires a small static\n structure around the mapped record that the user shouldn't have\n to author in the mapper itself — a message-bus class envelope,\n an HMAC signature wrap, an integration batch header, a named\n single-record array wrap, etc. The mapper stays focused on the\n destination's field shape; the body owns the transport-level\n scaffolding.\n\n Rare for non-GraphQL destinations, but real; the use cases\n that fit are obvious when the user describes them (\"wrap each\n request in a SQS-style message envelope\", \"add a batch header\n with timestamp + interface name\", etc.).\n\n Typical shapes:\n - Message-bus envelope:\n ``{\"MessageBody\": {\"class\": \"EntityUpdate\", \"payload\": {{{jsonSerialize record}}} }}``\n - Metadata wrap:\n ``{\"batchNumber\": \"{{timestamp dateTimeFormat CST}}\", \"interfaceType\": \"...\", \"requestData\": {{{jsonSerialize record}}} }``\n - Named single-record array wrap (when destination wants ``[record]`` but each request sends one):\n ``{\"products\": [{{{jsonSerialize record}}}]}``\n\n NEVER emit BARE ``{{{jsonSerialize record}}}`` with no wrapper.\n The platform sends the post-mapped record as JSON automatically\n when body is undefined; bare ``jsonSerialize record`` produces\n identical wire output, so it is redundant config the user has\n to maintain for zero benefit.\n\n ### Pattern B — Batch / page wrapper around mapped records\n\n Wrap an envelope (header, version, marketplace, etc.) around\n a serialized batch. The template root depends ONLY on\n ``batchSize`` (live-verified 2026-09-10): with ``batchSize`` 1\n (the default) the root carries ``record`` (an object record —\n or the fanned-out child on a ``oneToMany`` import) or ``rows``\n (a grouped/array record), and ``data`` is the one-element list\n of records in the request (a grouped record is that element,\n itself an array); with ``batchSize`` > 1 the root carries\n ``batch_of_records`` (one ``{record: …}`` item per object\n record, ``{rows: […]}`` for a grouped record) and ``data`` (the\n bare records), and ``record`` / ``rows`` are ABSENT. ``oneToMany`` does not make\n ``batch_of_records`` appear — children are batched after\n fan-out under the same rule.\n\n Example shape:\n ``{\"values\": {{#each data}} {{{jsonSerialize this}}} {{/each}} }``\n\n Example with comma-separation (``batchSize`` > 1):\n ``[{{#each batch_of_records}}{{{jsonSerialize this.record}}}{{#unless @last}},{{/unless}}{{/each}}]``\n\n ### Pattern C — Custom envelope with per-message fields\n\n Static envelope (header, account-id, etc.) with per-message\n field references inside a loop. Use when each message needs\n a small number of fields rather than the whole record.\n ``{{sku}}`` / ``{{quantity}}`` / etc. reference fields on the\n current ``{{#each data}}`` item.\n\n Example shape:\n ``{\"header\": {\"sellerId\": \"{{connection.http.unencrypted.sellerId}}\"}, \"messages\": [{{#each data}}{\"sku\": \"{{sku}}\", \"qty\": {{quantity}} }{{/each}}]}``\n - |-\n ## Anti-pattern — DO NOT wrap a post-mapped scalar in jsonSerialize\n\n Do NOT use ``{{{jsonSerialize record.\u003cscalar_field>}}}`` or\n ``{{{jsonSerialize (getValue \"record.\u003cscalar_field>\")}}}`` where\n the path resolves to a scalar string / number. The exact path\n depends on the destination's mapped shape — flat fields like\n ``record.sku`` or ``record.total``, or nested fields like\n ``record.line.qty`` if the mapping nests under ``line``. The\n anti-pattern is the ``jsonSerialize`` wrap, not any particular\n path shape.\n\n - Essentially never needed.\n - Failure mode: the body validator sees the PRE-mapped sample\n shape, can't resolve the post-mapped path during validation,\n and ``jsonSerialize`` on an unresolved path raises\n ``Helper {{jsonSerialize}} expects an argument`` — the\n template fails validation even when it would render at\n runtime. Variations (``getValue`` wrappers, alternate paths)\n fail the same way.\n\n If you find yourself wanting to reference a per-field\n post-mapped scalar wrapped in ``jsonSerialize``, your situation\n is almost certainly the \"leave body undefined\" case (the\n destination just wants the mapped record as JSON) or Pattern C\n (use ``{{sku}}`` / ``{{quantity}}`` directly without\n ``jsonSerialize`` inside a per-message envelope). To embed a\n scalar in a JSON string literal, use double braces inside the\n quotes (``\"{{record.\u003cfield>}}\"``) WITHOUT ``jsonSerialize``;\n triple braces there fail on a quote, backslash, newline or tab\n in the value. The anti-pattern is specifically the\n ``jsonSerialize`` wrap.\n - |-\n ## Multipart/form-data uploads are a different body contract\n\n When the request media type is ``form-data`` (on the connection\n or via ``requestMediaType``), this field is NOT a Handlebars\n record template — it is a JSON array of multipart parts, each\n ``{name, value, type}`` with optional ``filename`` (include the\n file extension) and ``mime-headers``:\n\n - The file part is ``\"type\": \"attachment\"`` with\n ``\"value\": \"{{blob}}\"`` — ``{{blob}}`` / ``{{{blob}}}`` are the\n only documented values for an attachment part. Import saves do\n NOT validate this (live-verified: a wrong value saves cleanly),\n so it must be right at authoring time. One file reference per\n import.\n - The file bytes come from integrator.io storage via the\n import's ``blobKeyPath``, which points at the record's\n blobKey from an upstream blob export/lookup.\n - Plain text fields the API wants alongside the file ride as\n additional ``\"type\": \"inline\"`` parts; an inline part whose\n value is a JSON object must be serialized:\n ``{{{jsonEncode (jsonSerialize field)}}}``.\n - Do not include a MIME boundary — integrator.io assembles the\n final multipart body and adds the boundary automatically at\n send time. That assembled body is not available in audit\n logs.\n - |-\n ## Hand-authored multipart bodies (multipart/related APIs)\n\n Some upload APIs expect a multipart flavor other than\n form-data — Google Drive's\n ``/upload/drive/v3/files?uploadType=multipart`` expects\n ``multipart/related`` — and the form-data parts-array\n machinery does not produce that flavor. For those APIs,\n author this field as the LITERAL MIME document instead: an\n explicit boundary, a metadata part (for Drive, JSON carrying\n ``name`` / ``parents``), and a content part whose body is\n ``{{blob}}`` (double braces) — the platform streams the\n staged file bytes at that reference.\n\n - Keep ``blob: true`` + ``blobKeyPath`` on the import; the\n blob reference still comes from an upstream blob\n export/lookup's response mapping.\n - Do NOT set ``http.type: \"file\"`` alongside an API-endpoint\n URL. That switches the step into file-provider folder mode,\n which treats the relativeURI as a folder path — the mixed\n config saves cleanly and fails every record at runtime with\n \"Directory is not found.\"\n - Match the flavor to the destination API's documentation\n before choosing: form-data APIs get the parts array above;\n multipart/related APIs get the hand-authored document. Never\n mix the two shapes in one body.\n examples:\n - ['{\"name\": \"{{record.name}}\", \"email\": \"{{record.email}}\"}']\n - ['{\"requestData\": {{{jsonSerialize record}}} }']\n - ['{\"values\": [{{#each batch_of_records}}{{{jsonSerialize this.record}}}{{#unless @last}},{{/unless}}{{/each}}]}']\n x-celigo-agent:\n - handlebars\n x-celigo-handlebars-context: post-mapped\n ignoreEmptyNodes:\n type: boolean\n description: |-\n When true, fields with empty values are stripped from the request body before it is sent,\n so the target API receives only populated fields. Shown as \"Remove empty fields from HTTP\n request body\" in the UI.\n examples: [true, false]\n existingExtract:\n type: string\n description: |-\n Field name or JSON path that drives the upsert decision for composite imports with both CREATE and UPDATE in requestType.\n When the field has a value in the incoming record, the UPDATE operation's relativeURI and method are used; when it is empty or missing, the CREATE operation runs.\n Must match a field populated by an upstream lookup or response mapping (e.g. a destination system ID like \"shopifyCustomerId\"); omit for single-operation imports.\n x-celigo-ai-guidance:\n - |-\n Used together with ``requestType: [\"UPDATE\", \"CREATE\"]`` (or the\n reverse order). The runtime evaluates this field on every\n incoming record:\n\n - If the field has a non-empty value → use the UPDATE operation\n (and the corresponding ``relativeURI`` / ``method`` at the\n matching positional index).\n - If the field is empty or missing → use the CREATE operation.\n - |-\n The field must be populated by an upstream step (a prior lookup node,\n a response mapping from an earlier write,\n or a source- system field maintained as a cross-reference like ``shopifyCustomerId``).\n This pattern adds **zero** per-record HTTP overhead because the existence check\n happens against a field that's already on the record.\n - |-\n ## Mutually exclusive with ``existingLookupName``\n\n ``existingExtract`` and ``existingLookupName`` are two ways to\n answer the same question (\"does this record already exist in the\n destination?\") for composite imports — set EXACTLY ONE. Use\n ``existingExtract`` when the incoming record already carries\n the destination id; use ``existingLookupName`` (with a named\n entry in ``http.lookups``) when the import needs to probe the\n destination per record because the record has no id.\n - |-\n ## When NOT to set this field\n\n Only valid for composite imports with BOTH ``CREATE`` and\n ``UPDATE`` in ``requestType``. Single-operation imports (pure\n create or pure update) use ``ignoreExtract`` / ``ignoreLookupName``\n instead — those work together with ``ignoreExisting`` /\n ``ignoreMissing`` to skip records rather than route them.\n examples:\n - \"shopifyCustomerId\"\n - \"netsuite_id\"\n - \"existingRecordId\"\n existingLookupName:\n type: string\n description: |-\n Name of an entry in `lookups` that resolves whether each record already exists in the destination, driving the composite upsert decision when `existingExtract` alone is insufficient (e.g. the check requires a call to the target system). Paired with composite CREATE/UPDATE imports.\n\n Use `existingLookupName` when the incoming record has no destination ID and\n needs a per-record probe to discover one;\n use `existingExtract` when the record already carries the ID.\n\n For single-operation imports,\n use `ignoreExtract` or `ignoreLookupName` with `ignoreExisting` or\n `ignoreMissing` to skip records instead of routing them.\n\n The lookup pattern adds one HTTP request per record at runtime,\n making it better suited to one-off migrations and low-volume reconciliation than\n high-volume recurring syncs.\n x-celigo-ai-guidance:\n - |-\n Used together with ``requestType: [\"UPDATE\", \"CREATE\"]`` (or the\n reverse order). The runtime runs the named lookup against every\n incoming record:\n\n - If the lookup returns a non-empty value → use the UPDATE\n operation (and the corresponding ``relativeURI`` / ``method``\n at the matching positional index). The lookup's extracted\n value is what the UPDATE URL handlebars resolve to via\n ``{{lookup.\u003cname>}}`` (e.g. ``PATCH /persons/{{lookup.\u003cname>}}``).\n - If the lookup returns empty / null → use the CREATE operation.\n - |-\n The value MUST match the ``name`` of a lookup entry in ``http.lookups``.\n The lookup config itself (HTTP path, method, extract,\n fallback chain) is defined separately, in that entry.\n - |-\n ## Mutually exclusive with ``existingExtract``\n\n ``existingExtract`` and ``existingLookupName`` are two ways to\n answer the same question (\"does this record already exist in\n the destination?\") for composite imports — set EXACTLY ONE.\n Use ``existingLookupName`` when the incoming record has no\n destination id and needs a per-record probe to discover one;\n use ``existingExtract`` when the record already carries the id.\n - |-\n ## When NOT to set this field\n\n Only valid for composite imports with BOTH ``CREATE`` and\n ``UPDATE`` in ``requestType``. Single-operation imports (pure\n create or pure update) use ``ignoreExtract`` / ``ignoreLookupName``\n instead — those work together with ``ignoreExisting`` /\n ``ignoreMissing`` to skip records rather than route them.\n examples:\n - \"lookupCustomerByEmail\"\n - pipedrive_person_lookup\n - netsuite_internal_id_lookup\n ignoreExtract:\n type: string\n description: |-\n Field name on the incoming record that drives the SKIP-vs-PROCESS\n decision when ``ignoreExisting`` or ``ignoreMissing`` is set.\n\n Pairs with either of the two skip-mode flags:\n\n - With ``ignoreExisting: true`` (CREATE-only imports that should\n avoid duplicates): if ``ignoreExtract`` has a value on the\n record, the record is treated as already-existing and is\n skipped; if empty, the record is created.\n - With ``ignoreMissing: true`` (UPDATE-only imports that should\n avoid creating new records): if ``ignoreExtract`` has a value,\n the record is treated as existing and is updated; if empty,\n the record is skipped.\n\n ``ignoreExtract`` checks a field that's already on the record —\n zero per-record HTTP overhead. Use ``ignoreLookupName`` instead\n when the record has no field to inspect and the existence check\n needs a per-record probe against the destination.\n\n **Mutually exclusive with ``ignoreLookupName``**\n\n Set EXACTLY ONE of ``ignoreExtract`` / ``ignoreLookupName`` when\n ``ignoreExisting`` or ``ignoreMissing`` is enabled.\n\n **When not to set this field**\n\n Only valid for single-operation imports with either\n ``ignoreExisting: true`` or ``ignoreMissing: true`` set.\n Composite (upsert) imports — those with both ``CREATE`` and\n ``UPDATE`` in ``requestType`` — use ``existingExtract`` /\n ``existingLookupName`` instead, because they route records\n between two write paths rather than skipping them.\n examples:\n - \"id\"\n - \"shopifyCustomerId\"\n - \"$.customer.email\"\n endPointBodyLimit:\n type: integer\n description: |\n Maximum size limit for the request body in bytes.\n Used to enforce API-specific size constraints.\n headers:\n type: array\n items:\n type: object\n properties:\n name:\n type: string\n description: Name of the HTTP header to include with each import request.\n value:\n type: string\n description: |-\n Value sent for the header. Supports handlebars expressions rendered against the pre-mapped record, with the same data roots as the body (`record` / `rows` / `batch_of_records`, plus `data`); an undefined path fails the request rather than rendering empty.\n x-celigo-agent:\n - handlebars\n x-celigo-handlebars-context: pre-mapped\n description: |-\n Custom HTTP headers included with import requests; values can contain Handlebars expressions.\n Value templates render against the pre-mapped record (the original input before the Import's mapping step) — use `{{record.\u003cfield>}}` to reference fields as they appear in the upstream source.\n x-celigo-ai-guidance:\n - Each header has a name and value, which can include handlebars expressions.\n examples:\n - [{\"name\": \"X-Custom-Header\", \"value\": \"custom-value\"}]\n response:\n type: object\n properties:\n resourcePath:\n type: array\n items:\n type: string\n description: |-\n Per-operation JSON path to the resource collection in each\n response. Outer-array length MUST match ``requestType``\n length: 1 entry for single-op imports, N entries for an\n N-operation composite import (duplicate the value when\n operations agree). See the ``response`` object docs\n above for the full positional-array contract.\n\n Required when ``batchSize > 1`` because the platform needs\n to know where the array of result records lives inside the\n response envelope. When the API returns a bare array at\n the top level, leave this field unset.\n\n Each entry is a DOT-PATH STRING (e.g. ``\"data.results\"``,\n ``\"items\"``) — do NOT split paths into segments across\n array elements.\n examples:\n - [\"data.results\"] # single-op\n - [\"data.results\", \"data.results\"] # composite, both agree (duplicated)\n - [\"data.updated\", \"data.created\"] # composite, different per op\n resourceIdPath:\n type: array\n items:\n type: [string, 'null']\n description: |-\n Per-operation JSON path to the unique ID field within each\n record in the response. Outer-array length MUST match\n ``requestType`` length — duplicate the value when both\n operations agree (the common case) rather than collapsing\n to a 1-entry array.\n\n Each entry is a DOT-PATH STRING, or null for an operation\n with no ID path configured. When not specified, the platform\n looks for standard `id` or `_id` fields automatically. When\n set, a response with no value at the path fails the record\n with `missing_id`, quoting the response.\n x-celigo-ai-guidance:\n - |-\n Common LLM mistake:\n splitting a single dot-path like ``\"data.id\"`` into array segments ``[\"data\",\n \"id\"]`` — DO NOT do this.\n See the ``response`` object docs above.\n - |-\n For a GraphQL mutation, point it at the id only a successful call returns\n (``[\"data.customerCreate.customer.id\"]``): a 200 that carries top-level ``errors``\n or the mutation's ``userErrors`` then fails the record with ``missing_id`` instead\n of counting as written. Live-verified 2026-10-22 against an echo endpoint.\n examples:\n - [\"id\"] # single-op\n - [\"data.id\"] # single-op, nested response\n - [\"data.id\", \"data.id\"] # composite, both agree (duplicated)\n successPath:\n type: array\n items:\n type: string\n description: |-\n Per-operation JSON path to a field that indicates whether\n the API call succeeded. Outer-array length MUST match\n ``requestType`` length — duplicate the value when both\n operations agree rather than collapsing to a 1-entry\n array on a composite import.\n\n Use this when the API returns HTTP 200 for everything and\n signals success / failure through a body field. Pairs with\n ``successValues`` to define which values at this path mean\n success.\n\n Each entry is a DOT-PATH STRING.\n examples:\n - [\"success\"] # single-op\n - [\"status\"] # single-op, status field\n - [\"success\", \"success\"] # composite, both agree (duplicated)\n successValues:\n type: array\n items:\n type: array\n items:\n type: string\n description: |-\n Per-operation list of values at ``successPath`` that mean\n success. TWO levels of array nesting:\n\n - Outer array: one entry per ``requestType`` index.\n Length MUST match ``requestType`` length — duplicate\n the inner array when both operations agree rather than\n collapsing to a 1-entry outer array on a composite\n import.\n - Inner array: the list of acceptable success values for\n that operation.\n\n All comparisons are STRING comparisons regardless of the\n actual response value type — the platform stringifies\n numbers, booleans, etc. before comparing.\n examples:\n - [[\"true\"]] # single-op: success when path == \"true\"\n - [[\"ok\", \"OK\"]] # single-op: multiple acceptable values\n - [[\"true\"], [\"true\"]] # composite, both agree (duplicated)\n - [[\"true\", \"1\"], [\"true\"]] # composite, UPDATE accepts more values\n failPath:\n type: array\n items:\n type: string\n description: |-\n Per-operation JSON path to a field that signals failure\n even when HTTP returns 200. Outer-array length MUST\n match ``requestType`` length — duplicate when both\n operations agree. Same array semantics as ``successPath``.\n\n Some APIs use only ``successPath`` (failure = absence of\n success); others publish both an explicit success indicator\n and an explicit failure indicator. When both are\n available, set both — the platform evaluates ``successPath``\n first and falls through to ``failPath`` only when the\n success check is inconclusive.\n\n Without ``failValues``, a truthy value at the path fails\n the response. A path that resolves to an array matches only\n when the array has exactly one element.\n x-celigo-ai-guidance:\n - |-\n A GraphQL API usually reports a failed mutation with HTTP 200 and an ``errors`` array, which\n counts as written when no rule catches it. ``failPath: [\"errors\"]`` fails the record\n for one error and lets two or more through; use ``[\"errors[0].message\"]`` with\n ``errorPath: \"errors[0].message\"`` so the record fails with the server's message.\n Live-verified 2026-10-22 with import runs against an echo endpoint.\n examples:\n - [\"error\"] # single-op\n - [\"success\"] # single-op, same path as successPath, opposite values\n - [\"error\", \"error\"] # composite, both agree (duplicated)\n - [\"errors[0].message\"] # single-op, GraphQL\n failValues:\n type: array\n items:\n type: array\n items:\n type: string\n description: |-\n Per-operation list of values at ``failPath`` that mean\n failure. Same two-level array shape and length contract\n as ``successValues`` — outer length MUST match\n ``requestType`` length, duplicate the inner array on\n composite imports when both operations agree.\n\n Often used as the inverse of ``successValues`` (e.g.\n ``successValues: [[\"true\"]]`` paired with\n ``failValues: [[\"false\"]]``).\n examples:\n - [[\"false\"]] # single-op\n - [[\"false\"], [\"false\"]] # composite, both agree (duplicated)\n - [[\"false\", \"error\"], [\"false\"]] # composite, UPDATE has more failure values\n errorPath:\n type: string\n description: |-\n JSON path to the error message inside the response body.\n\n **Note**: this field is NOT a per-operation array — it's a\n single string applied to every operation. Most APIs use\n the same error-message field shape for both UPDATE and\n CREATE responses, so the platform doesn't expose a per-op\n override here.\n examples:\n - error\n - errors[0].message\n - \"data.error\"\n allowArrayforSuccessPath:\n type: boolean\n description: When true, allows array values at successPath during success evaluation.\n x-celigo-ai-guidance:\n - |-\n Whether to evaluate ``successPath`` when its value is an array (rather than a single scalar).\n When ``true``,\n the success check passes if ANY element in the array matches ``successValues``.\n Defaults to ``false`` (single-value comparison only).\n hasHeader:\n type: boolean\n description: When true, treats the first record in the response as a header row (for CSV responses).\n x-celigo-ai-guidance:\n - |-\n For CSV responses:\n whether the first record is a header row that names the columns rather than carrying data.\n Ignored for JSON / XML responses.\n description: |-\n Configuration for parsing and interpreting HTTP responses returned\n from each write request the import issues.\n\n **Critical: positional per-operation arrays**\n\n Every field inside ``response`` (``resourcePath``, ``resourceIdPath``,\n ``successPath``, ``successValues``, ``failPath``, ``failValues``)\n is an ARRAY whose outer length is positionally aligned with\n ``requestType`` / ``method`` / ``relativeURI``. This is the\n SAME positional-alignment contract as those three sibling\n arrays — see ``relativeURI`` docs for the full explanation.\n\n Each outer-array index describes how to parse the response from\n the operation at that same index in ``requestType``:\n\n - **Single-operation import** (``requestType: [\"CREATE\"]``):\n every response field is a 1-element array.\n ``successPath: [\"ok\"]`` means \"for the one operation,\n look at path ``ok`` to detect success\".\n - **Composite upsert** (``requestType: [\"UPDATE\", \"CREATE\"]``):\n every response field is a 2-element array. Index 0 parses\n the UPDATE response, index 1 parses the CREATE response.\n\n **successValues / failValues — array of arrays**\n\n ``successValues`` and ``failValues`` carry an extra array level\n because each operation can have MULTIPLE acceptable values:\n\n - Outer array: one entry per operation (positional, as above).\n - Inner array: the list of acceptable values for that operation.\n\n Example:\n ```json\n \"requestType\": [\"UPDATE\", \"CREATE\"],\n \"successPath\": [\"ok\", \"ok\"],\n \"successValues\": [[\"true\", \"1\"], [\"true\"]]\n ```\n Means: the UPDATE response is successful when ``ok`` is either\n ``\"true\"`` or ``\"1\"``; the CREATE response is successful only\n when ``ok`` is ``\"true\"``.\n\n **Always duplicate when operations agree (REQUIRED for composite)**\n\n The outer-array length MUST match ``requestType`` length on\n every response field. When the UPDATE and CREATE responses\n of the same connector look the same (which is the common\n case — most APIs return the same shape from both endpoints),\n DUPLICATE the entry so the array has the same length as\n ``requestType``. Do NOT collapse to a 1-entry array on a\n composite import.\n\n ```json\n // ✅ Correct — explicit per-operation entries, even when identical\n \"requestType\": [\"UPDATE\", \"CREATE\"],\n \"resourceIdPath\": [\"data.id\", \"data.id\"],\n \"successPath\": [\"success\", \"success\"],\n \"successValues\": [[\"true\"], [\"true\"]]\n ```\n\n ```json\n // ❌ Wrong — 1-entry array on a composite import\n \"requestType\": [\"UPDATE\", \"CREATE\"],\n \"resourceIdPath\": [\"data.id\"],\n \"successPath\": [\"success\"],\n \"successValues\": [[\"true\"]]\n ```\n\n Why duplicate even when they agree:\n\n - **UI parity.** The Celigo UI renders one response-handling\n form per ``requestType`` index. A 1-entry array on a\n composite import leaves the second operation's form panel\n empty, which looks like a half-built config to the user\n reviewing the saved import — even if it works at runtime.\n - **Symmetric shape rule.** ``relativeURI`` / ``method`` /\n ``requestType`` REQUIRE the entry count to match across\n all three (the runtime decider uses positional alignment).\n Making ``response.*`` follow the same \"outer length ==\n requestType length\" rule means there's ONE shape rule to\n remember, not two.\n - **Unambiguous intent.** A 2-entry duplicated array\n documents in the saved config that the developer\n considered both operations. A 1-entry array could mean\n \"both ops use this\" or could mean \"I forgot to fill in\n the second op.\" Duplication eliminates the ambiguity.\n\n When operations DIFFER (rare — different response shapes\n per endpoint), set distinct values at each index. Either\n way the outer length always matches ``requestType``.\n\n **Common llm mistake (DO not do THIS)**\n\n Do NOT split a single dot-path into array segments thinking the\n outer array represents path segments:\n\n ```json\n // WRONG — the LLM expressed \"data.id\" as path segments\n \"requestType\": [\"UPDATE\", \"CREATE\"],\n \"resourceIdPath\": [\"data\", \"id\"]\n ```\n\n This is interpreted by the platform as: UPDATE response's id is\n at path ``data``, CREATE response's id is at path ``id`` — almost\n never what the user wants. The CORRECT shape is the single\n dot-path expressed as ONE string per operation, duplicated to\n match ``requestType`` length:\n\n ```json\n \"requestType\": [\"UPDATE\", \"CREATE\"],\n \"resourceIdPath\": [\"data.id\", \"data.id\"]\n ```\n _asyncHelperId:\n type: string\n format: objectId\n x-celigo-refModel: asynchelpers\n description: |-\n Reference to an AsyncHelper resource that polls an asynchronous\n destination API on this import's behalf.\n\n Set this ONLY when the destination API is genuinely asynchronous — it\n acknowledges the submitted payload (HTTP 202, a job ticket) and ingests\n it in the background, so completion must be polled for (e.g. bulk\n image/file ingestion, large bulk-load endpoints). Most imports are\n synchronous and need NO async helper; adding one to a synchronous\n destination just adds polling overhead plus a status and result export\n to maintain. When in doubt, leave it unset.\n\n The referenced helper bundles the polling config plus a required status\n export (polled to check progress) and a result export (fetches the\n final payload).\n examples:\n - 60a2c4e6f321d800129a1a3c\n x-celigo-agent:\n - async_helper\n ignoreLookupName:\n type: string\n description: |-\n Name of an entry in `lookups` used to check whether each record already exists, so `ignoreMissing`/`ignoreExisting` can skip it. Use it instead of `ignoreExtract` when existence must be resolved by a call to the target system rather than read from a field on the incoming record.\n x-celigo-ai-guidance:\n - |-\n ## When NOT to set this field\n\n Only valid for single-operation imports with either\n ``ignoreExisting: true`` or ``ignoreMissing: true`` set.\n Composite (upsert) imports — those with both ``CREATE`` and\n ``UPDATE`` in ``requestType`` — use ``existingExtract`` /\n ``existingLookupName`` instead, because they route records\n between two write paths rather than skipping them.\n - |-\n Name of a lookup defined in ``http.lookups`` that drives the SKIP-vs-PROCESS\n decision when ``ignoreExisting`` or ``ignoreMissing`` is set.\n - |-\n The value MUST match the ``name`` of a lookup entry in ``http.lookups``.\n This pattern adds **one HTTP request per record** at runtime — use\n ``ignoreExtract`` instead when the record already carries the field that signals\n existence.\n - |-\n Pairs with either of the two skip-mode flags:\n\n - With ``ignoreExisting: true`` (CREATE-only imports that should\n avoid duplicates): the named lookup probes the destination\n per record; if it returns a value, the record is treated as\n already-existing and is skipped; if it returns empty, the\n record is created.\n - With ``ignoreMissing: true`` (UPDATE-only imports that should\n avoid creating new records): same probe — a non-empty result\n means the record exists and gets updated; empty means skip.\n - |-\n ## Mutually exclusive with ``ignoreExtract``\n\n Set EXACTLY ONE of ``ignoreExtract`` / ``ignoreLookupName`` when\n ``ignoreExisting`` or ``ignoreMissing`` is enabled.\n examples:\n - \"lookupExistingOrder\"\n - shopify_customer_lookup\n - netsuite_internal_id_lookup\n lookups:\n x-celigo-agent:\n - lookup\n description: |-\n Lookups referenced by `existingLookupName`/`ignoreLookupName` to resolve record existence against the destination system at runtime.\n x-celigo-ai-guidance:\n - |-\n Lookup definitions executed over the HTTP connection.\n Each entry issues an HTTP request (or reads from cache / static map) and\n extracts a value that downstream layers reference by ``name``.\n - |-\n ## How callers reference a lookup by name\n\n Lookups are addressable by their ``name`` from three places:\n\n - **Field mappings** — a mapping entry sets\n ``lookupName: \"\u003cname>\"`` to resolve a field's value through\n the lookup. A ``lookupName`` that matches no entry in this\n array fails every record with ``Unable to find lookup in\n mapping configs - \u003cname>``.\n - **Handlebars expressions** — anywhere handlebars is\n evaluated (relativeURI, headers, body), ``{{lookup \"name\"\n value}}`` calls the lookup with ``value`` as input.\n - **Top-level decider fields** — ``http.existingLookupName`` /\n ``http.ignoreLookupName`` set to ``\"\u003cname>\"`` make a lookup\n drive the import's UPDATE-vs-CREATE (composite upsert) or\n SKIP-vs-PROCESS (``ignoreExisting`` / ``ignoreMissing``)\n decision, respectively. See those fields for the full\n decision semantics.\n - |-\n ## Mirror invariant\n\n The Celigo platform persists lookups in TWO locations on an HTTP\n import: top-level ``lookups`` and ``http.lookups``, and\n ``http.lookups`` is the copy that counts. A save keeps\n ``http.lookups`` and mirrors it to the top level, overwriting a\n top-level copy that differs; a ``lookups`` array sent only at the\n top level is dropped with no error, on create and on PUT, and both\n arrays read back null. Write ``http.lookups`` on every save;\n sending the same array at the top level too is harmless.\n - |-\n ## Static vs dynamic vs cached\n\n - **Static** — only ``map`` + optional ``default``. No HTTP\n request issued. Use for small controlled vocabularies.\n - **Dynamic** — ``method`` + ``relativeURI`` (+ ``body``,\n ``headers``, ``extract``). Issues a live HTTP request per\n lookup. Adds one HTTP request per record at runtime — factor\n this into flow design for high-volume syncs.\n - **Cached** — ``_lookupCacheId`` + ``extract``. Reads from a\n pre-populated LookupCache resource.\n items:\n type: object\n properties:\n name:\n type: string\n description: |\n Unique identifier for this lookup. Mapping fields reference it\n via ``lookupName`` and Handlebars expressions reference it via\n ``{{lookup.name}}``. Must be unique within this\n resource's ``lookups`` array.\n method:\n type: string\n enum:\n - GET\n - POST\n - PUT\n - PATCH\n - DELETE\n description: |\n HTTP method used to issue the lookup request. ``GET`` and\n ``POST`` are most common. Omit when using a static ``map`` or\n ``_lookupCacheId``.\n relativeURI:\n type: string\n x-celigo-agent:\n - handlebars\n description: |\n HTTP path (relative to the connection's base URL) that\n resolves the lookup. Use Handlebars with triple braces to\n inject values from the incoming record — e.g.\n ``/customers?email={{{email}}}`` or\n ``/accounts/{{{accountId}}}``.\n body:\n type: string\n x-celigo-agent:\n - handlebars\n description: |\n Request body template for POST / PUT / PATCH lookup methods, typically\n a JSON string with Handlebars placeholders (``{\"email\":\"{{email}}\"}``);\n ignored for GET / DELETE. The record's fields sit at the top level:\n ``{{email}}`` renders the record's value, while ``{{record.email}}``\n renders empty.\n x-celigo-ai-guidance:\n - >-\n Put the lookup's request template here, not in ``postBody``: a\n ``postBody`` saves without an error but is not kept under\n ``http.lookups``, so the lookup goes out with no body and the run\n reports no error.\n - >-\n Write string values with double braces inside the quotes\n (``\"{{email}}\"``): they are JSON-escaped at runtime. With triple\n braces a value holding a double quote breaks the request body, and\n the lookup quietly finds nothing, so an existence check writes the\n record as new.\n headers:\n type: array\n items:\n type: object\n properties:\n name:\n type: string\n value:\n type: string\n description: |\n Optional custom headers to attach to the lookup request.\n Useful for endpoints that need a different Accept header or\n an auxiliary auth token separate from the connection-level\n headers.\n extract:\n type: string\n description: |\n Path to the lookup value in the HTTP response body, written\n without a ``$.`` prefix: ``id``, ``results[0].id``, or ``ids``\n for an array of ids. A ``$.``-prefixed path, a ``[*]``\n wildcard, or a dot path through an array (``results.id``)\n resolves nothing, so the lookup misses: with\n ``allowFailures: false`` the record fails\n (``batch_lookup_evaluation_failed``); otherwise it goes on with\n ``default`` or without the value, and nothing errors. A\n composite import whose ``existingLookupName`` names such a\n lookup sends every record to CREATE. Mapper 2.0 mapping\n extracts are the opposite: they take the ``$.``.\n _lookupCacheId:\n type: string\n format: objectId\n x-celigo-refModel: lookupcaches\n description: |\n Optional reference to a LookupCache resource. When set, the\n lookup reads from the cache instead of issuing a live HTTP\n request — useful for stable reference data that should be\n pre-loaded.\n map:\n type: [object, 'null']\n description: |\n Optional static key→value object evaluated BEFORE the HTTP\n request. When the incoming value matches a key, the mapped\n value is returned without hitting the remote endpoint. Null\n on dynamic lookups with no static map.\n default:\n type: [string, 'null']\n description: |\n Value written when the lookup finds no match and the miss is\n allowed (``allowFailures`` true, or omitted on a static\n ``map``). A save refuses a ``default`` together with an\n explicit ``allowFailures: false`` (422 \"Lookups with default\n set can not have allowFailures explicitly set to false.\"), so\n a lookup either fails a miss (``allowFailures: false``, no\n default) or falls back to its default. Stored as null when no\n fallback is configured.\n x-celigo-ai-guidance:\n - >-\n Omit this field entirely when no fallback value is\n intended. An empty string is a value, not \"no default\":\n the save accepts `default: \"\"`, and a miss the lookup lets\n through writes `\"\"`.\n - >-\n A default raises no error: every unmatched value goes out\n as the default with a 200. Set one only when a single\n fallback is right for every value the lookup can't match;\n when a miss must stop the record, set allowFailures: false\n and leave the default out. An ambiguous answer (two or\n more matches) doesn't use the default either: see\n useDefaultOnMultipleMatches.\n allowFailures:\n type: [boolean, 'null']\n description: |\n When true, a lookup miss uses ``default`` (or leaves the mapped\n field out when there is none) and the record continues. When\n false, a miss fails the record: ``lookup_error`` (\"Could not find\n value for static lookup: …\") for a static ``map``,\n ``lookupcache_evaluation_failed`` for a cache, and\n ``batch_lookup_evaluation_failed`` (source ``internal``, \"\u003cfield>:\n The batch lookup could not be evaluated. Check the value in the\n key '\u003cvalue>', and try again.\") for a live request that matches\n nothing or whose ``extract`` resolves nothing. Omitted or null,\n a miss doesn't fail the record either: a static lookup writes\n its ``default`` if it has one, and a lookup without one (static,\n cache, or a live request that matches nothing or fails) leaves\n the mapped field out.\n\n False is also the only setting under which an ambiguous answer\n fails the record: two or more values at ``extract`` (an array of\n ids or of result objects) fail it with ``lookup_conflict``\n (\"Multiple matches found for lookup.\"). Omitted, null or true,\n the first value is sent with no error, and an array of result\n objects is written as the text ``[object Object]``. An indexed\n ``extract`` (``ids[0]``) takes the first match whatever this\n field says, so it hides an ambiguous search rather than fixing\n it.\n x-celigo-ai-guidance:\n - >-\n Set false explicitly when a miss must stop the record. Leaving\n the field out lets the record through with the default or\n without the looked-up value, so a destination that needs the\n real value gets a request without it and nothing errors.\n - >-\n Fix an ambiguous lookup in its search, not here: query on a\n key that identifies one record (an email, an external id)\n and keep false, so a search that still finds two fails\n instead of writing against the first match.\n useDefaultOnMultipleMatches:\n type: boolean\n description: |\n Meant to return ``default`` when the response holds more than\n one match. On an HTTP import it has not been seen to take\n effect: with ``allowFailures`` omitted or true the first match\n is sent whatever this flag says, with or without a\n ``default``, and with ``allowFailures: false`` the record\n fails with ``lookup_conflict``. Two or more matches fail the\n record only when ``allowFailures`` is false (see\n ``allowFailures``), so make the search match one record\n rather than relying on this flag.\n type: array\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":76874,"content_sha256":"1df501c4ddb311cde0fab24a49f5a6965ba843336d9cf2c807af74504d7256a5"},{"filename":"references/schemas/jdbc.yml","content":"Jdbc:\n type: object\n description: |-\n Configuration for JDBC import operations. Defines how data is written to a database\n via a JDBC connection.\n\n **Query type determines which fields are required**\n\n | queryType | Required fields | Do NOT set |\n |------------------|------------------------------|-------------------|\n | [\"per_record\"] | query (array of SQL strings) | bulkInsert |\n | [\"per_page\"] | query (array of SQL strings) | bulkInsert |\n | [\"bulk_insert\"] | bulkInsert object | query |\n | [\"bulk_load\"] | bulkLoad object | query |\n\n **Critical:** query IS AN ARRAY OF STRINGS\n The query field must be an array of plain strings, NOT a single string and NOT an array of objects.\n Correct: [\"INSERT INTO users (name) VALUES ('{{{name}}}')\"]\n Wrong: \"INSERT INTO users ...\"\n Wrong: [{\"query\": \"INSERT INTO users ...\"}]\n required:\n - queryType\n properties:\n query:\n type: array\n items:\n type: string\n x-celigo-agent:\n - sql\n description: |-\n Array of SQL query strings to execute. REQUIRED when queryType is [\"per_record\"] or [\"per_page\"].\n\n Each element is a complete SQL statement as a plain string. Typically contains a single query.\n\n Use Handlebars {{fieldName}} syntax to inject values from incoming records.\n\n **Format — array of strings**\n - CORRECT: [\"INSERT INTO users (name, email) VALUES ('{{{name}}}', '{{{email}}}')\"]\n - WRONG: \"INSERT INTO users ...\" (not an array)\n - WRONG: [{\"query\": \"INSERT INTO users ...\"}] (objects are invalid — causes Cast error)\n\n **Examples**\n - INSERT: [\"INSERT INTO users (name, email) VALUES ('{{{name}}}', '{{{email}}}')\"]\n - UPDATE: [\"UPDATE inventory SET qty = {{{quantity}}} WHERE sku = '{{{sku}}}'\"]\n - MERGE/UPSERT: [\"MERGE INTO target USING (SELECT CAST(? AS VARCHAR) AS email) AS src ON target.email = src.email WHEN MATCHED THEN UPDATE SET name = ? WHEN NOT MATCHED THEN INSERT (name, email) VALUES (?, ?)\"]\n\n **String vs numeric values in handlebars**\n - Use triple braces so the raw value is emitted; double braces emit the value already single-quoted, so wrapping them in quotes double-quotes it.\n - Strings: supply the quotes yourself — '{{{name}}}'\n - Numbers: no quotes — {{{quantity}}}\n queryType:\n type: array\n items:\n type: string\n enum:\n - bulk_insert\n - per_record\n - per_page\n - bulk_load\n x-enumDescriptions:\n bulk_insert: Batch INSERT via multi-row VALUES clause for efficient bulk data loading.\n per_record: Execute one SQL statement per record, giving full control over individual record SQL.\n per_page: Execute one SQL statement per page (batch) of records.\n bulk_load: Stage data as a file and load via database-native COPY/bulk mechanism for maximum throughput.\n x-celigo-ui-override: >-\n Required by the JDBC import form (jdbc.queryType is required:true). Encoded to mirror the\n form so builders produce connectable configurations.\n description: |\n Execution strategy for the SQL operation. REQUIRED. Must be an array with one value.\n\n **Decision tree**\n\n 1. If UPDATE or UPSERT/MERGE → [\"per_record\"] (set query field)\n 2. If INSERT with \"ignore existing\" / \"skip duplicates\" / match logic → [\"per_record\"] (set query field)\n 3. If pure INSERT with no duplicate checking → [\"bulk_insert\"] (set bulkInsert object)\n 4. If high-volume bulk load → [\"bulk_load\"] (set bulkLoad object)\n\n **Critical relationship to other fields**\n | queryType | REQUIRES | DO NOT SET |\n |------------------|-----------------------|--------------|\n | [\"per_record\"] | query (array) | bulkInsert |\n | [\"per_page\"] | query (array) | bulkInsert |\n | [\"bulk_insert\"] | bulkInsert object | query |\n | [\"bulk_load\"] | bulkLoad object | query |\n\n **Examples**\n - Per-record upsert: [\"per_record\"]\n - Bulk insert: [\"bulk_insert\"]\n - Bulk load: [\"bulk_load\"]\n bulkInsert:\n type: object\n description: |\n Bulk insert configuration. REQUIRED when queryType is [\"bulk_insert\"]. DO NOT SET when queryType is [\"per_record\"].\n\n Enables efficient batch insertion of records into a database table without per-record SQL.\n properties:\n tableName:\n type: string\n description: |\n Target database table name for bulk insert. REQUIRED.\n Can include schema qualifiers (e.g., \"schema.tableName\").\n examples: [\"public.orders\", \"dbo.Customers\"]\n batchSize:\n type: string\n description: |\n Number of records per batch during bulk insert.\n Larger values improve throughput but use more memory.\n Common values: \"1000\", \"5000\".\n examples: [\"1000\", \"5000\"]\n bulkLoad:\n type: object\n description: |\n Bulk load configuration. REQUIRED when queryType is [\"bulk_load\"]. Uses database-native bulk loading for maximum throughput.\n properties:\n tableName:\n type: string\n description: |\n Target database table name for bulk load. REQUIRED.\n Can include schema qualifiers (e.g., \"schema.tableName\").\n examples: [\"public.orders\", \"ANALYTICS.STG_ORDERS\"]\n primaryKeys:\n type: [array, 'null']\n items:\n type: string\n description: |\n Primary key column names for upsert/merge during bulk load.\n When set, existing rows matching these keys are updated; non-matching rows are inserted.\n Example: [\"id\"] or [\"order_id\", \"product_id\"] for composite keys.\n overrideMergeOrInsertQuery:\n type: boolean\n description: |-\n When true, a custom merge/insert query replaces the auto-generated statement for the\n bulk load, enabling ignore-existing logic or conditional updates.\n lookups:\n type: array\n items:\n type: object\n properties:\n name:\n type: string\n description: |-\n Unique identifier for this lookup. Mapping fields reference it\n via `lookupName` and Handlebars expressions reference it via\n `{{lookup \"name\" value}}`. Must be unique within this\n resource's `lookups` array.\n examples: [\"lookupUserByEmail\"]\n query:\n type: string\n x-celigo-agent:\n - sql\n - handlebars\n description: SQL query for the lookup (e.g., \"SELECT id FROM users WHERE email = '{{{email}}}'\").\n x-celigo-ai-guidance:\n - |-\n SQL `SELECT` statement that resolves the lookup value.\n Inject values from the incoming record with Handlebars — JDBC queries use triple braces:\n `SELECT id FROM users WHERE email = '{{{email}}}'`.\n Returns a single row;\n use `extract` to pick a column when multiple are selected.\n examples: [\"SELECT id FROM users WHERE email = '{{{email}}}'\"]\n extract:\n type: string\n description: |-\n JSONPath-style expression that selects the value from the\n query result row (e.g. `id`, `details.price`). Omit when\n the query returns a single scalar.\n examples: [\"id\"]\n _lookupCacheId:\n type: string\n format: objectId\n x-celigo-refModel: lookupCaches\n description: |-\n Optional reference to a LookupCache resource. When set, the\n lookup reads from the cache instead of issuing a live SQL\n query — useful for stable reference data that should be\n pre-loaded.\n allowFailures:\n type: boolean\n description: |-\n When true, a lookup miss (no matching row and no `default`)\n resolves to `null` and the import continues. When false\n (default), a miss halts the record.\n map:\n type: object\n description: |-\n Optional static key→value object evaluated BEFORE the SQL\n query. When the incoming value matches a key, the mapped\n value is returned without querying the database.\n default:\n type: string\n description: |-\n Value returned when the query matches no rows. When omitted\n and `allowFailures` is false, an unmatched lookup halts the\n record.\n description: |-\n Lookup definitions executed against the JDBC connection. Each entry\n runs a `SELECT` and extracts a value used by field mappings\n (referenced via `lookupName`) or Handlebars expressions\n (referenced via `{{lookup \"name\" value}}`).\n x-celigo-ai-guidance:\n - |-\n ## Mirror invariant\n The Celigo platform persists lookups in TWO locations on a JDBC\n import: top-level `lookups` and `jdbc.lookups`. Both arrays\n must hold byte-for-byte identical content, so write both on every\n save.\n x-celigo-agent:\n - lookup\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":9539,"content_sha256":"998fd26c60e035a7f737123fa6f6244cc346e81999ba1a5ee103f5f4fdf00b62"},{"filename":"references/schemas/mongodb.yml","content":"Mongodb:\n type: object\n description: |-\n Configuration for MongoDB imports. Contains only the MongoDB-specific properties; import-level fields such as ignoreExisting, ignoreMissing, name, and description are set on the parent import, not here.\n required:\n - method\n - collection\n properties:\n method:\n type: string\n enum:\n - insertMany\n - updateOne\n x-enumDescriptions:\n insertMany: Insert multiple documents into the MongoDB collection in a single batch operation.\n updateOne: Update a single existing document in the MongoDB collection that matches the filter criteria.\n x-celigo-ui-override: >-\n Required by the MongoDB import form (mongodb.method is required:true). Encoded to mirror\n the form so builders produce connectable configurations.\n description: Write operation performed against the MongoDB collection for each record.\n collection:\n type: string\n x-celigo-ui-override: >-\n Required by the MongoDB import form (mongodb.collection is required:true). Encoded to\n mirror the form so builders produce connectable configurations.\n description: |-\n MongoDB collection that receives the imported records.\n Collection names are case-sensitive and must not start with the reserved \"system.\" prefix.\n filter:\n type: string\n x-celigo-agent:\n - handlebars\n x-celigo-ui-override: >-\n Required by the MongoDB import form when method is updateOne (mongodb.filter is\n required:true, visibleWhen method=updateOne).\n description: |-\n MongoDB query filter that selects the document each updateOne applies to; required when\n method is updateOne. Supports standard MongoDB query operators and dot notation, and\n handlebars placeholders to reference the incoming record's fields.\n examples: ['{\"id\": \"{{record.NSRecordId}}\"}', '{\"email\": \"{{record.email}}\"}']\n document:\n type: string\n x-celigo-agent:\n - handlebars\n description: Content of the document written to the collection (used when method is insertMany), as a JSON string that can include nested documents and arrays and handlebars placeholders.\n examples: ['{\"name\": \"{{record.name}}\", \"email\": \"{{record.email}}\"}']\n update:\n type: string\n x-celigo-agent:\n - handlebars\n description: |-\n Modifications applied to matching documents (used when method is updateOne), using MongoDB update operators such as $set, $inc, or $push.\n A complete document without operators replaces the matched document entirely.\n examples: ['{\"$set\": {\"status\": \"{{record.status}}\"}}']\n upsert:\n type: boolean\n description: |-\n When true, inserts a new document built from the update criteria if no existing document matches the filter.\n When false, records with no matching document are skipped and nothing is inserted.\n ignoreExtract:\n type: string\n description: |-\n Path to the field in the source record that identifies existing records. When that field has a value, the record is treated as existing and skipped.\n Only used when the import-level ignoreExisting flag is true; do not set it otherwise.\n Enclose field names containing special characters in square brackets (for example, [vendor-code]), and reference array items with an index, such as items[0].id.\n x-celigo-ai-guidance:\n - Set this together with the import-level ignoreExisting flag; both must be set for skip-existing behavior to work, and ignoreExisting is not part of this mongodb object.\n - Use the field path from the incoming source data, not the target MongoDB field. When the prompt names an identifier field (for example, \"ignore existing customers indicated by the id field\"), use that field.\n ignoreLookupFilter:\n type: string\n description: |-\n JSON-stringified MongoDB query filter used to find existing documents in the target collection when the import-level ignoreExisting flag is true.\n If the query matches a document, the incoming record is treated as existing and skipped.\n Use Handlebars placeholders to reference values from the incoming record, for example \"{\\\"email\\\":\\\"{{email}}\\\"}\".\n Unlike ignoreExtract, which only checks whether a field on the incoming record has a value, this filter queries the MongoDB collection itself.\n x-celigo-ai-guidance:\n - The value must be a JSON string (stringify the filter object); filter keys must match field names in the MongoDB collection, and Handlebars variables must match fields in the incoming data.\n examples: ['{\"id\": \"{{data.id}}\"}', '{\"email\": \"{{email}}\"}']\n if:\n properties:\n method:\n const: updateOne\n required:\n - method\n then:\n required:\n - filter\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":4864,"content_sha256":"670d798537c9888e54b3fa120519b59c308fb19e44be2682faa0a7dcacec732e"},{"filename":"references/schemas/netsuite.yml","content":"NetSuite:\n type: object\n description: Configuration for NetSuite imports (legacy adaptor; use netsuite_da / NetSuiteDistributedImport for SuiteApp 2.0).\n # Requireds are conditional (see the if/then at the end of this schema): the server\n # writes stub `netsuite` objects ({preferences:{}} / {lookups:[]}) onto imports of\n # every adaptorType as bookkeeping, and file-provider configs (isFileProvider:true)\n # have no recordType.\n properties:\n lookups:\n type: array\n items:\n type: object\n properties:\n name:\n type: string\n description: Unique name for this lookup, referenced from mappings and handlebars templates with `{{lookup 'name' value}}`.\n recordType:\n type: string\n description: NetSuite record type searched by a dynamic lookup (for example, `customer`, `salesOrder`). Set with `searchField` or `expression`; omit for a static `map` lookup.\n searchField:\n type: string\n description: NetSuite field the dynamic lookup searches on (for example, `externalid`). Must be a searchable field on `recordType`. Use this or `expression`.\n expression:\n type: string\n description: NetSuite search expression evaluated at runtime to select matching records (for example, `[\"recipient\",\"is\",\"123\"]`). Use this or `searchField`.\n resultField:\n type: string\n description: |-\n Field on the matched NetSuite record whose value the lookup returns (for example, `internalid`).\n Supports dot notation for nested fields.\n map:\n type: [object, 'null']\n description: Static lookup table with input values as keys and their corresponding output values. Use instead of a NetSuite search for fixed value translations. The platform stores `null` here on search-driven lookups.\n default:\n type: [string, 'null']\n description: Fallback value returned when the lookup finds no match. Used together with `allowFailures`. Stored as `null` when not configured.\n allowFailures:\n type: boolean\n description: When true, an unmatched lookup falls back to `default` and processing continues; when false, an unmatched lookup fails the record.\n _id:\n type: object\n description: Server-assigned unique identifier for this lookup entry.\n readOnly: true\n description: Lookup definitions used to resolve reference values during the import, using either a static `map` or a dynamic NetSuite record search.\n operation:\n type: string\n enum:\n - add\n - update\n - addupdate\n - attach\n - detach\n - delete\n x-lowercase: true\n x-enumDescriptions:\n add: Create a new record in NetSuite.\n update: Update an existing record in NetSuite, identified via internalIdLookup.\n addupdate: Upsert — update the record if found, otherwise create a new one.\n attach: Attach one NetSuite record to another record.\n detach: Detach a previously attached NetSuite record from another record.\n delete: Delete an existing record from NetSuite, identified via internalIdLookup.\n x-celigo-ui-override: >-\n Required by the NetSuite import form (operation is required:true). Encoded to mirror the\n form so builders produce connectable configurations.\n description: |-\n Operation to perform on NetSuite records, as a plain string (the API lowercases the value).\n For `update`, `addupdate`, and `delete`, also set internalIdLookup so existing records can be found; for `add` with ignoreExisting, set internalIdLookup to check for duplicates before creating.\n Default to `addupdate` when the intent is \"sync\", \"upsert\", or \"create or update\"; use `add` for plain \"create\" or \"insert\".\n Do not wrap the value in an object like `{\"type\": \"addupdate\"}` — that fails validation.\n isFileProvider:\n type: boolean\n description: |-\n When true, enables file operations against NetSuite — browsing, uploading, downloading, updating, and deleting files.\n When false or omitted, file-related operations are unavailable through this import.\n customFieldMetadata:\n type: object\n description: Metadata describing the custom fields defined on the target NetSuite record type, used to validate and process custom field data during the import.\n recordType:\n type: string\n x-celigo-ui-override: >-\n Required by the NetSuite import form (recordType is required:true) for record imports.\n File-provider imports (isFileProvider true) are built by the file-provider form, which\n sets `file` instead of recordType — the schema's if/then/else mirrors that split.\n description: |-\n NetSuite record type this import writes to (e.g. customer, salesOrder, invoice).\n Use the exact NetSuite internal record type identifier; both standard and custom record types are supported.\n examples: [\"customer\", \"salesOrder\", \"customrecord_user\"]\n recordTypeId:\n type: string\n description: The unique identifier specifying the record type within NetSuite. Determines the applicable schema, fields, validation rules, and processing logic. Use exact NetSuite record type identifiers such as standard types or custom record IDs.\n retryUpdateAsAdd:\n type: boolean\n description: |-\n When true, an update that fails because the target record does not exist is automatically retried as an add.\n When false or unset, update failures error immediately without retrying.\n May create duplicate records if the update failed for a reason other than a missing record.\n batchSize:\n type: number\n description: |-\n Number of records sent to NetSuite per API call.\n Larger batches reduce the number of calls but increase memory use and timeout risk per call.\n internalIdLookup:\n type: object\n properties:\n extract:\n type: string\n description: Path used to extract the value for the internal ID lookup.\n examples: [\"externalId\"]\n searchField:\n type: string\n description: The NetSuite record field used as the key attribute for the internal ID lookup search. Should be a unique or indexed field to ensure accurate matches. Must correspond to a valid, searchable field on the target record type.\n examples: [\"externalid\"]\n expression:\n type: string\n description: |-\n NetSuite search expression that filters which records match the lookup.\n Supports logical operators (AND, OR, NOT), comparison operators, and nested conditions, evaluated at runtime against current data.\n description: |-\n Configuration for locating existing NetSuite records by internal ID.\n Required when operation is `update`, `addupdate`, or `delete`; also used with `add` plus ignoreExisting to check for duplicates.\n preferences:\n type: object\n properties:\n ignoreReadOnlyFields:\n type: boolean\n description: When true, silently skips read-only fields during update operations instead of raising errors.\n warningAsError:\n type: boolean\n description: When true, treats NetSuite warnings as errors, causing the operation to fail immediately.\n skipCustomMetadataRequests:\n type: boolean\n description: When true, skips custom metadata requests during NetSuite API operations to improve performance.\n description: Preferences that control how NetSuite handles the import operation.\n file:\n type: object\n properties:\n name:\n type: string\n description: |-\n Filename for the file in the NetSuite File Cabinet, without any folder or path information.\n Must be unique within its folder; include the file extension (e.g. .pdf, .csv).\n fileType:\n type: string\n description: |-\n NetSuite file type that governs how the file is processed, stored, and displayed.\n Must match the actual content format of the file.\n folder:\n type: string\n description: |-\n Folder in the NetSuite File Cabinet where the file is stored, as a numeric folder ID or a folder path.\n Updating this on an existing file moves the file to the specified folder.\n folderInternalId:\n type: string\n description: |-\n Internal ID of the File Cabinet folder where the file is stored.\n Must reference an existing folder; changing it moves the file to a different folder.\n internalId:\n type: string\n description: |-\n NetSuite internal ID of an existing file, used to target it for retrieval, updates, or deletion.\n Assigned by NetSuite when the file is created and cannot be changed.\n backupFolderInternalId:\n type: string\n description: Internal ID of the File Cabinet folder where backup files are stored.\n description: Configuration for the file to upload to or update in the NetSuite File Cabinet.\n # Requireds apply only to real configs, not server-written bookkeeping stubs whose\n # keys are all in {preferences, lookups}. File-provider configs (isFileProvider:true)\n # carry operation + file but no recordType. Stack-style conditional (no allOf).\n if:\n not:\n propertyNames:\n enum: [preferences, lookups]\n then:\n required:\n - operation\n if:\n properties:\n isFileProvider:\n const: true\n required:\n - isFileProvider\n else:\n required:\n - recordType\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":9828,"content_sha256":"c7dfc993937bc8514185ca9c7a2e886a4bd7064fc146a2f36b5fd77f25e1ea0e"},{"filename":"references/schemas/netsuitedistributed.yml","content":"NetsuiteDistributed:\n type: object\n description: |-\n Configuration for NetSuite Distributed (SuiteApp 2.0) import operations — the primary sub-schema for the NetSuiteDistributedImport adaptorType. The API field name is \"netsuite_da\".\n operation and recordType are required. The server may attach empty bookkeeping stubs of\n this object (empty lookups/mapping, or missingOrCorruptedDAConfig) to imports of other\n adaptor types; stubs omit these fields. Set internalIdLookup so existing records\n can be found when operation is update, addupdate, or delete.\n x-celigo-ai-guidance:\n - |-\n ## KEY FIELDS\n - operation: REQUIRED — the NetSuite operation (add, update, addupdate, etc.)\n - recordType: REQUIRED — the NetSuite record type (e.g., \"customer\", \"salesorder\")\n - mapping: field mappings from source data to NetSuite fields\n - internalIdLookup: how to find existing records for update/addupdate/delete\n - restletVersion: defaults to \"suiteapp2.0\"; fixed once the step is created (recreate/clone to change)\n properties:\n operation:\n type: string\n enum:\n - add\n - update\n - addupdate\n - attach\n - detach\n - delete\n x-lowercase: true\n x-enumDescriptions:\n add: Create a new record in NetSuite.\n update: Update an existing record in NetSuite, requires internalIdLookup.\n addupdate: Upsert — update the record if found, otherwise create a new one.\n attach: Attach one NetSuite record to another record.\n detach: Detach a previously attached NetSuite record from another record.\n delete: Delete an existing record from NetSuite, requires internalIdLookup.\n x-celigo-ui-override: >-\n Required by the NetSuite import form (operation is required:true). Encoded to mirror the\n form so builders produce connectable configurations.\n description: |-\n Operation to perform on the target NetSuite record, as a plain string. Required.\n For `update`, `addupdate`, and `delete`, also set internalIdLookup so existing records can be found.\n x-celigo-ai-guidance:\n - |-\n ## GUIDANCE\n - Default to \"addupdate\" when user says \"sync\", \"upsert\", or \"create or update\".\n - Use \"add\" when user says \"create\" or \"insert\" without mentioning updates.\n - For \"update\", \"addupdate\", and \"delete\", you MUST also set internalIdLookup.\n - |-\n ## IMPORTANT\n This field is a string, NOT an object. Set it directly: \"operation\": \"addupdate\"\n Do NOT send {\"type\": \"addupdate\"} — that causes a Cast error.\n recordType:\n type: string\n x-celigo-ui-override: >-\n Required by the NetSuite import form (recordType is required:true). Encoded to mirror the\n form so builders produce connectable configurations.\n description: |-\n NetSuite record type to import into (e.g. \"customer\", \"salesorder\", \"customrecord_myrecord\" for custom record types). Required.\n Must match a valid NetSuite record type identifier.\n x-celigo-ai-guidance:\n - |-\n Use the NetSuite internal record-type id: lowercase, no spaces,\n no camelCase (e.g.\n `salesorder` — NOT `salesOrder` — and `itemfulfillment` — NOT `itemFulfillment`).\n Custom records use `customrecord_\u003cid>`.\n - |-\n Common standard record types in active integrations (ordered most → least frequent):\n `salesorder`, `customer`, `invoice`, `itemfulfillment`,\n `customerrefund`, `returnauthorization`, `creditmemo`, `inventoryitem`,\n `customerpayment`, `customerdeposit`, `journalentry`, `itemreceipt`,\n `cashrefund`, `cashsale`, `purchaseorder`, `deposit`, `contact`,\n `transferorder`, `inventoryadjustment`, `vendorbill`, `vendor`, `job`,\n `noninventoryitem`, `employee`.\n examples: [\"customer\", \"salesorder\", \"customrecord_user\"]\n recordIdentifier:\n type: string\n description: |\n Custom record identifier, used to identify the specific record type when\n importing into custom record types.\n restletVersion:\n type: string\n enum:\n - suitebundle\n - suiteapp1.0\n - suiteapp2.0\n x-lowercase: true\n x-enumDescriptions:\n suitebundle: Legacy SuiteBundle RESTlet version; slated for deprecation — NetSuite no longer updates SuiteScript 1.0.\n suiteapp1.0: SuiteApp 1.0 RESTlet version using SuiteScript 1.0.\n suiteapp2.0: SuiteApp 2.0 RESTlet version using SuiteScript 2.0, the modern default.\n description: |-\n Version of the NetSuite RESTlet to use, as a plain string.\n Defaults to \"suiteapp2.0\" when useSS2Restlets is true, \"suitebundle\" otherwise; rarely needs to be set explicitly for modern integrations.\n The version is fixed when the step is created — the Advanced selector is disabled on existing steps, and migrating an existing step to a different version means recreating or cloning it.\n x-celigo-ai-guidance:\n - |-\n Do not wrap the value in an object like `{\"type\": \"suiteapp2.0\"}` — that causes a Cast error.\n - |-\n The NetSuite API version cannot be changed on an existing step — the UI\n disables the selector once the step is saved. When asked to migrate an\n existing step off the SuiteBundle (or between versions), do not update\n this field in place; the migration path is recreating or cloning the\n step with the new version.\n - |-\n SuiteScript hooks are a separate setting from the step's API version:\n hook code must be written for the same SuiteScript version as the step\n (1.0 and 2.x are not interchangeable), and migrating a hook is its own\n task — flag it whenever migrating a step that carries hooks.\n - |-\n Author new steps with suiteapp2.0 unless the user explicitly requires a\n legacy version — NetSuite no longer updates SuiteScript 1.0 and Celigo\n plans to deprecate the SuiteBundle option.\n useSS2Restlets:\n type: boolean\n description: |-\n When true, uses SuiteScript 2.0 RESTlets and restletVersion defaults to \"suiteapp2.0\".\n Defaults to true for modern integrations.\n Set at step creation together with restletVersion — the same creation-time-only constraint applies (see restletVersion).\n missingOrCorruptedDAConfig:\n type: boolean\n description: |-\n When true, the Distributed Adaptor configuration is missing or corrupted.\n Set by the system — do not set manually.\n batchSize:\n type: number\n description: |\n Number of records to process per batch. Controls how many records are sent\n to NetSuite in a single API call. Typical values: 50-200.\n x-celigo-ai-guidance:\n - |-\n Batching applies to the record writes. The `internalIdLookup` searches for a\n page of records are issued ahead of the writes and independently of `batchSize`,\n so `batchSize: 1` does not make the existing-record lookup run one record at a\n time.\n internalIdLookup:\n type: object\n description: |-\n Configuration for looking up existing NetSuite records to match against incoming data.\n Required when operation is \"update\", \"addupdate\", or \"delete\".\n x-celigo-ai-guidance:\n - |-\n Two ways to specify the lookup, picked per import:\n\n - **Expression form** — the default choice: set `expression` to a\n NetSuite search-expression literal (the canonical patterns are on\n the `expression` field — `[\"internalid\",\"is\",\"{{{id}}}\"]` for a\n record that carries the internal id, `externalid` and `cust*_*`\n variants, `anyof` for multi-match, AND/OR compounds). When\n `expression` is set, `searchField` / `operator` / `extract` are\n ignored.\n - **Simple form** — set `searchField` + `operator` (+ optional\n `extract`) for a single-criterion lookup against a built-in\n NetSuite field; the platform builds the equivalent search\n expression at runtime. For the internal id itself the expression\n form above is the preferred path, and `anyof` is available only\n through `expression`.\n - |-\n The lookup result is what identifies the target record; on `update` a source\n field mapped to `internalid` is overridden by it. When the incoming record\n already carries the NetSuite internal ID, point the lookup at that field with\n the expression form — `[\"internalid\",\"is\",\"{{{id}}}\"]` — rather than removing\n the lookup. Only when every record carries it: on `restletVersion:\n suiteapp2.0` a record whose field is empty does not fall through to\n \"create\" on `addupdate` — the empty value is searched as `isempty` and\n can match an existing record (see `expression`; SuiteBundle and SuiteApp\n 1.0 create in that case).\n properties:\n extract:\n type: string\n description: Path in the source record to extract the lookup value from (e.g. \"internalId\" or \"externalId\").\n x-celigo-ai-guidance:\n - |-\n Path on the incoming source record (NOT the post-mapped record) whose value drives the lookup.\n Used with the simple form (`searchField` + `operator`).\n Common values: `\"externalId\"`, `\"id\"`.\n Ignored when `expression` is set.\n examples: [\"externalId\"]\n searchField:\n type: string\n description: NetSuite field to search against (e.g. \"externalId\", \"email\", \"name\", \"tranId\").\n x-celigo-ai-guidance:\n - |-\n NetSuite field the simple-form lookup queries against.\n Common values: `\"externalId\"`, `\"email\"`, `\"tranId\"`, `\"name\"`.\n Matching on the record's internal id is the expression form —\n `[\"internalid\",\"is\",\"{{{id}}}\"]` — not a `searchField`.\n Ignored when `expression` is set.\n examples: [\"externalId\"]\n operator:\n type: string\n description: Comparison operator for the lookup (e.g. \"is\", \"contains\", \"startswith\").\n x-celigo-ai-guidance:\n - |-\n Comparison operator for the simple-form lookup.\n Common values: `\"is\"` (exact match), `\"contains\"`, `\"startswith\"`,\n `\"equalto\"`. Multi-match (`anyof`) is available only through\n `expression` (`[\"internalid\",\"anyof\",\"{{{id}}}\"]`).\n Ignored when `expression` is set.\n expression:\n type: string\n description: NetSuite search expression for complex lookup conditions, used for multi-field or conditional lookups.\n x-celigo-ai-guidance:\n - |-\n NetSuite search-expression literal as a JSON string.\n The expression is evaluated at runtime — Handlebars renders `{{...}}`\n substitutions FIRST against the incoming source record (pre-mapped),\n then the substituted expression is sent to NetSuite as a saved-search query.\n - |-\n ## Handlebars conventions\n\n - **No `record.` prefix** — substitutions reference top-level\n source-record fields directly. `{{{id}}}` / `{{{Id}}}` /\n `{{{externalid}}}` / `{{{[Internal ID]}}}` — NOT\n `{{{record.id}}}`. Top-level paths are the established\n convention.\n - **Triple braces `{{{...}}}`** are the preferred form;\n double-brace `{{...}}` also works for cases where you want\n the platform's default HTML-escape (rare in this field).\n - **Bracket notation for field names with spaces or special\n chars**: `{{{[Field With Spaces]}}}`, `{{{[Internal ID]}}}`.\n - |-\n ## Canonical patterns\n\n ### Simple lookup by internal ID (most common)\n\n `[\"internalid\",\"is\",\"{{{id}}}\"]`\n\n Use when the source record carries the destination NetSuite\n internal-id directly (e.g., from a prior lookup, response\n mapping, or cross-reference field).\n\n ### Multi-match by internal ID\n\n `[\"internalid\",\"anyof\",\"{{{id}}}\"]`\n\n Same as above but uses `anyof` to match an internal-id from a\n comma-separated list value. `anyof` is only available via the\n expression form (the simple `operator` field doesn't carry it).\n\n ### Lookup by external ID\n\n `[\"externalid\",\"is\",\"{{{Id}}}\"]` /\n `[\"externalid\",\"is\",\"{{{id}}}\"]`\n\n Use when the source record's primary key is stored in NetSuite as\n an `externalid` (cross-reference scenarios — Shopify, SFCC,\n Amazon, etc., where the platform-specific id is the externalid).\n\n ### Lookup by custom-record field\n\n `[\"custbody_celigo_etail_order_id\",\"is\",\"{{{id}}}\"]`\n `[\"custentity_celigo_sfio_sf_id\",\"is\",\"{{{Id}}}\"]`\n `[\"custrecord_celigo_shopify_payout_id\",\"is\",\"{{{id}}}\"]`\n\n Use when the destination NetSuite record carries a custom field\n holding the source-system id. The `cust*_*` field id MUST\n match the script-id of an existing custom field on the target\n record type (e.g., `custbody_*` on transactions,\n `custentity_*` on entities, `custrecord_*` on custom records\n and lists).\n\n ### Compound AND lookup (multiple criteria)\n\n `[[\"custbody_celigo_amzio_settlement_id\",\"is\",\"{{{[Settlement Id]}}}\"],\"AND\",[\"custbody_celigo_amzio_accounts\",\"is\",\"{{{[Amazon Account Internal Id]}}}\"]]`\n\n Use when the destination record needs more than one field to\n uniquely identify it (composite-key scenarios — Amazon\n settlements, multi-store integrations). Each criterion is a\n `[field, operator, value]` triple; the outer array alternates\n triples and operators (`\"AND\"` / `\"OR\"`).\n\n ### Compound with OR / nested grouping\n\n `[[\"custbody_celigo_etail_order_id\",\"is\",\"{{{order_id}}}\"],\"AND\",[\"custbody_celigo_etail_order_fulfilled\",\"is\",\"T\"],\"AND\",[[\"custbody_celigo_etail_fulfillment_id\",\"is\",\"{{{id}}}\"],\"OR\",[\"custbody_celigo_etail_fulfillment_id\",\"isempty\",\"\"]]]`\n\n `OR` works the same as `AND`; nesting groups via inner\n arrays. Use `\"isempty\"` (with empty-string value) to match\n unset fields.\n - |-\n ## Anti-patterns\n\n - DO NOT prefix substitutions with `record.` — top-level paths\n are the convention.\n - DO NOT use `jsonSerialize` here — search expressions take\n scalar literals, not serialized JSON. Use `{{{id}}}` or\n `{{{[Field Name]}}}` directly.\n - |-\n ## A substitution that renders empty is not \"no match\"\n\n When the source field is absent or empty, `[\"internalid\",\"is\",\"{{{id}}}\"]`\n renders `[\"internalid\",\"is\",\"\"]`, and the SuiteApp rewrites an empty\n `is` / `equalto` / `internalid anyof` value to `isempty` before the\n search. On `restletVersion: suiteapp2.0` that search is not treated as\n \"no match\": it can return existing records, and `addupdate` then\n updates one of them — an unrelated record receives the mapping (for\n example, a `cant_modify_sub` error on a customer the record never\n named). SuiteBundle and SuiteApp 1.0 discard a blank `internalid`\n match and create. Design so the operational lookup never renders\n empty: key it on a value every record carries (the source id stored\n as `externalid`), or route records without a destination id to a\n separate `add` import.\n - |-\n ## Body-level only\n\n This lookup identifies the record being imported, so bare\n body fields are the right paths here. A per-field lookup\n attached to a sublist row (`lookups[]` referenced from\n `mapping.lists[].fields[].lookupName`) is different: its\n expression addresses the current line with\n `\u003carray>[*].\u003cfield>` — see `lookups[].expression`.\n hooks:\n type: object\n description: |\n Script hooks for custom processing at different stages of the import.\n Each hook references a SuiteScript file and function.\n properties:\n preMap:\n type: object\n description: Runs before field mapping is applied.\n properties:\n fileInternalId:\n type: [string, 'null']\n description: NetSuite internal ID of the SuiteScript file. Null on hook\n stubs where no SuiteScript file has been selected.\n function:\n type: string\n description: Name of the function to execute.\n configuration:\n type: [object, 'null']\n description: Configuration object passed to the hook function. Null when\n the hook carries no static parameters.\n postMap:\n type: object\n description: Runs after field mapping, before submission to NetSuite.\n properties:\n fileInternalId:\n type: [string, 'null']\n description: NetSuite internal ID of the SuiteScript file. Null on hook\n stubs where no SuiteScript file has been selected.\n function:\n type: string\n description: Name of the function to execute.\n configuration:\n type: [object, 'null']\n description: Configuration object passed to the hook function. Null when\n the hook carries no static parameters.\n postSubmit:\n type: object\n description: Runs after the record is submitted to NetSuite.\n properties:\n fileInternalId:\n type: [string, 'null']\n description: NetSuite internal ID of the SuiteScript file. Null on hook\n stubs where no SuiteScript file has been selected.\n function:\n type: string\n description: Name of the function to execute.\n configuration:\n type: [object, 'null']\n description: Configuration object passed to the hook function. Null when\n the hook carries no static parameters.\n mapping:\n type: object\n description: |-\n NetSuite-specific Mapper 1.0 field mappings (the \"DA\" dialect).\n Used by `NetSuiteDistributedImport` exclusively — NetSuite's\n Distributed Adaptor cannot consume generic v1 mappings or the\n Mapper 2.0 shape because it needs sublist, subrecord, and\n conditional lookup semantics the generic schemas don't model.\n\n Field-level shape: `fields` for body columns, `lists` for\n sublists / line-item arrays.\n x-celigo-ai-guidance:\n - |-\n Validated via the dedicated `PUT /v1/netsuiteDA/previewImportMappingFields`\n endpoint (NOT `/v1/processors/mapperProcessor`).\n properties:\n fields:\n type: array\n items:\n type: object\n properties:\n extract:\n type: [string, 'null']\n description: |\n Path in the source record to extract the value from.\n Use dot notation for nested fields (e.g., \"address.city\").\n The platform stores `null` here on fields driven by\n `hardCodedValue` instead of an extract path.\n examples: [\"address.city\"]\n generate:\n type: string\n description: |\n The NetSuite field ID to write the value to (e.g., \"companyname\", \"email\", \"subsidiary\").\n examples: [\"companyname\"]\n hardCodedValue:\n type: string\n description: |\n A static value to always use instead of extracting from source data.\n Mutually exclusive with extract.\n lookupName:\n type: string\n description: Reference to a lookup defined in netsuite_da.lookups by name.\n dataType:\n type: string\n description: |\n Data type hint for the field value (e.g., \"string\", \"number\", \"date\", \"boolean\").\n internalId:\n type: boolean\n description: When true, the value is a NetSuite internal ID reference.\n immutable:\n type: boolean\n description: When true, this field is only set on record creation, not on updates.\n discardIfEmpty:\n type: boolean\n description: When true, skip this field mapping if the extracted value is empty.\n extractDateFormat:\n type: string\n description: |\n Date format of the extracted value (e.g., \"MM/DD/YYYY\", \"ISO8601\").\n Used to parse date strings from source data.\n extractDateTimezone:\n type: [string, 'null']\n description: Timezone of the extracted date value (e.g., \"America/New_York\"). Stored as `null` when not configured.\n subRecordMapping:\n type: object\n description: |-\n Nested Mapper 1.0 mapping for a NetSuite subrecord\n embedded in this field. A subrecord is a child\n record that only exists inside a parent (e.g. the\n Inventory Detail on a serialized-item line of a\n sales order); NetSuite reaches it through the\n parent.\n\n **Recursive structure.** The nested `mapping`\n object has the same `{fields, lists}` shape as the\n top-level `netsuite_da.mapping`. Recursion is\n allowed but rare.\n x-celigo-ai-guidance:\n - |-\n **Do not over-emit.** Per the Celigo help center:\n \"Unconfigured subrecord mappings can cause your NetSuite import to fail.\n Only include subrecord mappings on imports that require subrecord data.\" Emit\n this block ONLY when the destination record type genuinely requires the\n subrecord.\n - |-\n **Common `recordType` values** (most common\n first):\n * `inventorydetail` — by far the most common.\n Serial / lot / bin assignments on sales-order,\n item-fulfillment, purchase-order, and inventory-\n adjustment line items. Nested `mapping`\n typically has empty `fields` and a single\n `inventoryassignment` sublist.\n * `componentinventorydetail` — same concept on\n work-order / assembly-build component lines.\n * `itemvendorprice` — vendor-specific price tiers\n on item-vendor relationships.\n * `landedcost` — landed-cost allocations.\n * `itempricing` — quantity-based price tiers on\n contracts.\n * `mainaddress` / `shippingaddress` /\n `returnaddress` — address subrecords on\n transactions.\n * `countdetail` — per-line detail on physical-\n inventory counts.\n - |-\n **Where this block lives.** The enclosing field's `generate` name is a\n synthetic trigger (typically `celigo_*`) whose purpose is to carry this\n `subRecordMapping` block — it is NOT a real NetSuite column.\n The trigger → `recordType` correspondence is mostly `celigo_\u003crecordType>`\n but NOT a strict rule:\n `celigo_inventorydetail` covers BOTH `inventorydetail` and\n `componentinventorydetail` depending on the parent record,\n and address triggers use the `celigo_address_\u003cqualifier>` ↔ `\u003cqualifier>address` flip (e.g.\n `celigo_address_main` ↔ `mainaddress`).\n Match trigger names on existing resources rather than synthesizing new ones.\n properties:\n recordType:\n type: string\n description: |\n NetSuite's internal record-type id of the\n SUBRECORD (not the parent). E.g.\n `inventorydetail`, `itempricing`,\n `mainaddress`.\n jsonPath:\n type: string\n description: |\n JSONPath into the source record that selects the\n data feeding this subrecord. Semantics depend\n on placement:\n * Line-level (on `lists[].fields[].`\n `subRecordMapping`) — often ` configuring-imports — Skillopedia to reuse\n the enclosing sublist line's row, or a path\n into a nested array on that row.\n * Body-level (on `fields[].subRecordMapping`)\n — a path into the source record root.\n Defaults to ` configuring-imports — Skillopedia .\n mapping:\n type: object\n description: |\n Nested `{fields, lists}` mapping for the\n subrecord. Identical shape to the top-level\n `netsuite_da.mapping`. Subrecords often have\n empty `fields` and only a sublist (e.g.\n `inventorydetail` → `inventoryassignment`).\n lookups:\n type: array\n description: |\n Subrecord-scoped lookups. Same shape as the\n top-level `netsuite_da.lookups`. Fields inside\n this `subRecordMapping.mapping` reference them\n via `lookupName`. Often `[]`.\n referenceFieldId:\n type: string\n description: |\n Secondary reference-field identifier used on a\n small fraction of subrecord blocks. Exact\n semantics are not publicly\n documented by Celigo — match the value on\n existing resources when present; do NOT\n synthesize one. Leave unset unless a template\n clearly requires it.\n conditional:\n type: object\n description: |\n Conditional logic for when to apply this field mapping\n (\"Only perform mapping when\" in the UI). Lookup-based\n conditions evaluate the lookup named by `lookupName`,\n which must be defined on this import.\n x-celigo-ai-guidance:\n - |-\n A `conditional.lookupName` that matches no lookup defined on the\n import fails every record at runtime with\n `Cannot set properties of undefined (setting '_hasStarNotation')`\n (source `application`) — the message names neither the mapping row\n nor the lookup. Diagnose by cross-referencing every body and sublist\n row's `conditional.lookupName` against the import's defined lookups;\n a lookup renamed or removed while a condition still references the\n old name is the typical cause. Fix the reference or restore the\n lookup — changing the RESTlet / SuiteScript version does not resolve\n this error.\n - |-\n `when: record_updated` requires an operation that can update an\n existing record: on an `add`-only operation every record fails with\n `The mapping condition:\"record_updated\" not supported for the\n operation`. It works on `addupdate` and `update`.\n - |-\n The condition is one of the fixed `when` values — there is no\n predicate over a source field (\"when `customer.id` is not empty\",\n \"when `channel.name` is Point of Sale\" cannot be written here).\n `extract_not_empty` is the only source-driven gate: on a lookup row\n it reads the row's `extract`, and when that is empty the row is\n skipped and the field is left unset — nothing substitutes a value.\n Value-dependent behaviour (a fallback record when the key is\n missing) belongs inside the lookup's `expression` (see\n `lookups[].expression`).\n - |-\n Two rows with the same `generate` are not a fallback pair: the\n last row for a destination decides it alone — when its condition\n skips it the field stays empty even if an earlier row produced a\n value, and when it applies it overwrites the earlier row's value\n (`ignore_if_set` reads the target record, not an earlier row).\n properties:\n lookupName:\n type: string\n description: |-\n Lookup to evaluate for the condition. Must exactly match the\n `name` of a lookup defined on this import.\n when:\n type: string\n enum:\n - record_created\n - record_updated\n - extract_not_empty\n - lookup_not_empty\n - lookup_empty\n - ignore_if_set\n x-enumDescriptions:\n record_created: Apply this mapping only when a new record is being created.\n record_updated: Apply this mapping only when an existing record is being updated.\n extract_not_empty: Apply this mapping only when the extracted source value is non-empty.\n lookup_not_empty: Apply this mapping only when the lookup returns a value.\n lookup_empty: Apply this mapping only when the lookup returns no value.\n ignore_if_set: Skip this mapping if the target field already has a value.\n description: Condition that controls when this field mapping is applied.\n description: |\n Body-level field mappings. Each entry maps a source field to a NetSuite body field.\n lists:\n type: array\n items:\n type: object\n properties:\n generate:\n type: string\n description: |\n The NetSuite sublist ID (e.g., \"item\" for sales order line items,\n \"addressbook\" for address sublists).\n jsonPath:\n type: string\n description: |-\n LEGACY — do not set. Nominally the JSON path to the array of sublist records, but NetSuite DA ignores a top-level jsonPath here.\n x-celigo-ai-guidance:\n - |-\n NetSuite DA ignores a top-level `jsonPath` on sublist entries and will break\n iteration when one is present (only the first source element renders).\n Drive iteration from each child field's `extract` instead (see `fields[*].extract` below).\n fields:\n type: array\n items:\n type: object\n properties:\n extract:\n type: [string, 'null']\n description: |-\n Path from the source record root that produces\n the value for this sublist column. Paths are\n always anchored at the source record root —\n never relative to a pre-iterated row. The\n platform stores `null` here on fields driven by\n `hardCodedValue` instead of an extract path.\n x-celigo-ai-guidance:\n - |-\n Two extract patterns, often mixed in the same\n sublist:\n\n 1. `parent_array[*].field_name` (bracket-star)\n — iterates the source array and produces one\n sublist line per element. Use for values that\n vary per line (line-item sku, quantity, price,\n etc.):\n\n line_items[*].netsuite_item_id → item\n line_items[*].quantity → quantity\n line_items[*].price → rate\n\n 2. Bare or dotted path (`field_name` or\n `parent.field`) — pulls a source-root scalar\n and repeats it on every generated sublist\n line. Use for values that don't vary per line\n (e.g. a shared shipping carrier, order-level\n metadata that decorates every package):\n\n Tracking[*].TrackingNumber → packagetrackingnumber\n ShippingCarrier → packagedescr\n\n Here every package line gets the same\n `ShippingCarrier` value; iteration is\n driven by `Tracking[*]`.\n - |-\n The mapper drives iteration from the `[*]` segments among the child extracts;\n scalar extracts tag along.\n If no child uses `[*]` and none set `extract` at all (every child is a `hardCodedValue`),\n the sublist renders a single static line.\n - |-\n A row that resolves its value through `lookupName`\n (typically `extract: \"\"`) is driven by the sibling\n rows' `[*]` extracts, and the lookup's `expression`\n addresses the current line with the same\n `\u003carray>[*].\u003cfield>` path — see\n `netsuite_da.lookups[].expression`.\n generate:\n type: string\n description: NetSuite sublist field ID to write to.\n hardCodedValue:\n type: string\n description: Static value for this sublist field.\n lookupName:\n type: string\n description: Reference to a lookup by name.\n dataType:\n type: string\n description: Data type hint for the field value.\n internalId:\n type: boolean\n description: When true, the value is a NetSuite internal ID reference.\n isKey:\n type: boolean\n description: |-\n When true, this field is a key for matching the record's existing sublist lines: a\n mapped line updates the one existing line whose key fields hold the same values. `lists`\n says which records match lines, the keys used when no field sets `isKey`, and what a\n line that matches no line, or several, does.\n immutable:\n type: boolean\n description: When true, this field is only set on record creation, not on updates.\n discardIfEmpty:\n type: boolean\n description: When true, skips this field mapping if the extracted value is empty.\n extractDateFormat:\n type: string\n description: Date format of the extracted value.\n extractDateTimezone:\n type: [string, 'null']\n description: Timezone of the extracted date value. Stored as `null` when not configured.\n subRecordMapping:\n type: object\n description: |-\n Nested Mapper 1.0 mapping for a NetSuite\n subrecord embedded in this sublist-field. A\n subrecord is a child record that only exists\n inside a parent (e.g. the Inventory Detail on\n a serialized-item line of a sales order);\n NetSuite reaches it through the parent.\n\n **Recursive structure.** The nested\n `mapping` object has the same\n `{fields, lists}` shape as the top-level\n `netsuite_da.mapping`. Recursion is allowed\n but rare.\n x-celigo-ai-guidance:\n - |-\n **Do not over-emit.** Per the Celigo help center:\n \"Unconfigured subrecord mappings can cause your NetSuite import to fail.\n Only include subrecord mappings on imports that require subrecord data.\" Emit\n this block ONLY when the destination record type genuinely requires the\n subrecord.\n - |-\n **Common `recordType` values** (most\n common first):\n * `inventorydetail` — by far the most\n common. Serial / lot / bin assignments on\n sales-order, item-fulfillment, purchase-\n order, and inventory-adjustment line items.\n Nested `mapping` typically has empty\n `fields` and a single\n `inventoryassignment` sublist.\n * `componentinventorydetail` — same concept\n on work-order / assembly-build component\n lines.\n * `itemvendorprice` — vendor-specific price\n tiers on item-vendor relationships.\n * `landedcost` — landed-cost allocations.\n * `itempricing` — quantity-based price tiers\n on contracts. Nested `mapping` typically\n carries a `discount` sublist with\n `fromquantity` and `rate` rows.\n * `countdetail` — per-line detail on\n physical-inventory counts.\n - |-\n **Where this block lives.** The enclosing sublist-field's `generate` name is a\n synthetic trigger (typically `celigo_*`) whose purpose is to carry this\n `subRecordMapping` block — it is NOT a real NetSuite column.\n The trigger → `recordType` correspondence is mostly `celigo_\u003crecordType>`\n but NOT a strict rule:\n `celigo_inventorydetail` covers BOTH `inventorydetail` and\n `componentinventorydetail` depending on the parent record.\n Match trigger names on existing resources rather than synthesizing new ones.\n properties:\n recordType:\n type: string\n description: |\n NetSuite's internal record-type id of the\n SUBRECORD (not the parent). E.g.\n `inventorydetail`, `itempricing`,\n `componentinventorydetail`.\n jsonPath:\n type: string\n description: |\n JSONPath (relative to the current sublist\n line) to the rows that populate the\n subrecord. Typically ` configuring-imports — Skillopedia when the\n subrecord data is carried on the same\n line item (iteration over the sublist\n has already selected the line).\n Defaults to ` configuring-imports — Skillopedia .\n mapping:\n type: object\n description: |\n Nested `{fields, lists}` mapping for the\n subrecord. Identical shape to the\n top-level `netsuite_da.mapping`.\n Subrecords often have empty `fields` and\n only a sublist (e.g. `inventorydetail` →\n `inventoryassignment`).\n lookups:\n type: array\n description: |\n Subrecord-scoped lookups. Same shape as\n the top-level `netsuite_da.lookups`.\n Fields inside this\n `subRecordMapping.mapping` reference\n them via `lookupName`. Often `[]`.\n referenceFieldId:\n type: string\n description: |\n Secondary reference-field identifier used\n on a small fraction of subrecord blocks.\n Exact semantics are not publicly\n documented by Celigo — match\n the value on existing resources when\n present; do NOT synthesize one. Leave\n unset unless a template clearly requires\n it.\n conditional:\n type: object\n description: |-\n Conditional logic for when to apply this sublist field mapping\n (\"Only perform mapping when\" in the UI). Lookup-based conditions\n evaluate the lookup named by `lookupName`, which must be defined\n on this import.\n x-celigo-ai-guidance:\n - |-\n Same dangling-reference failure as body-field conditions: a\n `lookupName` that matches no lookup defined on the import breaks\n the run — on sublist rows this has been observed to stall the\n flow job (it never completes) instead of producing the\n `Cannot set properties of undefined (setting '_hasStarNotation')`\n error record. Cross-reference every row's `conditional.lookupName`\n against the import's defined lookups.\n - |-\n The operation contract differs from body-field conditions:\n `when: record_updated` on an `add`-only operation is rejected\n per-record on body rows (`The mapping condition:\"record_updated\"\n not supported for the operation`) but was NOT rejected on sublist\n rows in live verification. Do not rely on that tolerance — use\n `when: record_updated` only on operations that can update\n records (`addupdate` / `update`).\n - |-\n The condition is one of the fixed `when` values — the same\n closed set as body rows, with no predicate over a source\n field; `extract_not_empty` skips the row on the lines whose\n extract is empty and leaves that line's field unset. Two\n sublist rows with the same `generate` are not a fallback\n pair either, and the order rule is not the body rows': among\n extract rows the last row decides each line alone (a skipped\n last row leaves the line's field unset even when an earlier\n row produced a value), while a hard-coded row's value lands\n on every line whatever its position — a later conditional\n extract row does not override it.\n properties:\n lookupName:\n type: string\n description: |-\n Lookup to evaluate for the condition. Must exactly match the\n `name` of a lookup defined on this import.\n when:\n type: string\n description: Condition that controls when this field mapping is applied.\n enum:\n - record_created\n - record_updated\n - extract_not_empty\n - lookup_not_empty\n - lookup_empty\n - ignore_if_set\n x-enumDescriptions:\n record_created: Apply this mapping only when a new record is being created.\n record_updated: Apply this mapping only when an existing record is being updated.\n extract_not_empty: Apply this mapping only when the extracted source value is non-empty.\n lookup_not_empty: Apply this mapping only when the lookup returns a value.\n lookup_empty: Apply this mapping only when the lookup returns no value.\n ignore_if_set: Skip this mapping if the target field already has a value.\n description: Field mappings for each column in the sublist.\n description: |\n Sublist (line-item) mappings. Each entry maps source data to a NetSuite sublist.\n\n Each mapped line updates the one existing line whose key fields hold the same values: its\n `isKey` fields, or, with none, the sublist's default key. The existing lines are the stored\n record's on an update, and on a transform (`Created From`: the body field\n `celigo_nlobjTransformId` holds the source transaction's internal id) the lines the transform\n fills in. Default keys:\n\n - the item (`item`, else `itemkey` or `itemname`) on item fulfillments, item receipts,\n invoices, cash sales, cash refunds, return authorizations, bin worksheets, inbound shipment\n receipts and inventory counts, and on an assembly build's `component` sublist;\n - the applied transaction (`refnum`) on the `apply`, `credit` and `deposit` sublists of\n customer and vendor payments, credit memos, vendor credits and customer refunds;\n - `docnumber` on a deposit's `payment` sublist;\n - `vendor` and `language` on an item record's `itemvendor` and `translations` sublists.\n\n A sublist with neither, such as a sales order's or purchase order's `item`, matches nothing, so\n an update adds every mapped line after the existing ones. Two matching lines fail the record\n with `MULTIPLE_LINES_WITH_SAME_KEYS` (\"Multiple lines found with keys item=\u003cvalue> ...\"). A\n line that matches none is added as a new line, which an item fulfillment's `item` sublist\n refuses with `SSS_INVALID_SUBLIST_OPERATION`: its lines are its order's.\n x-celigo-ai-guidance:\n - |-\n To make an update replace a sublist's lines, also map the body field\n `celigo_replaceAllLines_\u003csublist id>` (for example `celigo_replaceAllLines_item`) to `true`:\n the existing lines no mapped line matched are removed, then the mapped lines are written. Where\n lines can't be removed it clears them instead: `itemreceive` false on an item fulfillment or\n item receipt, `apply` false on an `apply` sublist. On a sublist with no `isKey` field and no\n default key every existing line is unmatched, so all are replaced; with a default key, the lines\n it matches stay.\n - |-\n Key an update on a value each line holds once: a line id the record's create wrote to a line\n field, or, on a fulfillment, the order line's id mapped to `orderline`. Keying on the item fails\n with `MULTIPLE_LINES_WITH_SAME_KEYS` when an order holds the same item on two lines, and records\n that already hold repeated lines fail the same way once a key is set.\n x-celigo-agent:\n - mapping\n lookups:\n type: array\n items:\n type: object\n properties:\n name:\n type: string\n description: |-\n Unique identifier for this lookup. Mapping fields reference it\n via `lookupName` and Handlebars expressions reference it via\n `{{lookup \"name\" value}}`. Must be unique within this\n resource's `lookups` array.\n x-celigo-ai-guidance:\n - |-\n Mapping rows also reference lookups by this exact name in\n `conditional.lookupName` (\"Only perform mapping when\"). When renaming\n or removing a lookup, update every referencing row: a stale field-level\n `lookupName` fails records with\n `Unable to find lookup in mapping configs - \u003cname>`; a stale\n `conditional.lookupName` on a body row fails them with the unnamed\n `Cannot set properties of undefined (setting '_hasStarNotation')` error\n (see `mapping.fields[].conditional`), and on a sublist row has been\n observed to stall the flow job with no error record\n (see `mapping.lists[].fields[].conditional`).\n examples: [\"lookupCustomerByEmail\"]\n recordType:\n type: string\n description: |-\n NetSuite record type to search. Use the internal record-type\n id (lowercase, underscored), e.g. `customer`, `item`,\n `salesorder`, `customlist_xxx`.\n examples: [\"customer\"]\n searchField:\n type: string\n description: |-\n NetSuite field on `recordType` to match against. Use the\n internal field id — e.g. `email`, `externalid`,\n `internalid`, `entityid`, `custitem_xxx`.\n examples: [\"email\"]\n resultField:\n type: string\n description: |-\n NetSuite field whose value is returned when a record matches.\n Typically `internalid` when the lookup resolves to a record\n reference, or any queryable scalar field.\n examples: [\"internalid\"]\n expression:\n type: string\n x-celigo-agent:\n - handlebars\n description: |-\n Optional advanced-search expression (NetSuite saved-search\n DSL) for multi-criterion lookups. When present, supersedes\n `searchField` / `operator` — those fields are ignored.\n Handlebars inside the expression is rendered against the\n whole source record before the search runs; a lookup on a\n sublist row reaches the current line through\n `\u003carray>[*].\u003cfield>` (for example `line_items[*].sku`).\n x-celigo-ai-guidance:\n - |-\n ## Evaluation context — the whole source record, no `record.` prefix\n\n The expression's Handlebars renders against the incoming\n source record as the context root (the record is NOT\n wrapped under `record`), then the rendered JSON array is\n sent to NetSuite as the search criteria. Body-level fields\n are addressed bare: `{{{id}}}`, `{{{email}}}`.\n - |-\n ## Sublist-row lookups address the line with `\u003carray>[*].\u003cfield>`\n\n A lookup referenced from a sublist mapping row\n (`mapping.lists[].fields[].lookupName`) still receives the\n whole record; the current line's fields are reached with the\n same bracket-star path the sublist extracts use —\n `{{{line_items[*].sku}}}` resolves the line's item (one line\n per sublist row). A bare line field (`{{sku}}`,\n `{{#contains sku \"_\"}}`) is undefined at the record root: a\n helper that receives it fails every record with\n `INVALID_HANDLE_BAR_EXPRESSION` — `Could not compile handle\n bar \"\u003cexpression>\" because \"Cannot call method \"indexOf\" of\n undefined\"` — before anything reaches NetSuite. When an\n audit diff shows `\u003carray>[*].` removed from the expression\n right before such errors began, that edit is the cause; do\n not look for a line missing the field.\n - |-\n ## Rendered unmatched\n\n A rendered value NetSuite does not match fails the record\n with `VALUE_LOOKUP_FAILED` — `Could not find a match for\n \"\u003crendered criteria>\" for the record type \"\u003crecordType>\"`\n — quoting the criteria exactly as rendered, so the message\n shows what the expression produced for that line. With\n `allowFailures: true` the `default` value is used instead\n and no error is raised. Give every branch an output.\n - |-\n ## An empty substitution searches `isempty`\n\n When the source path a criterion reads is null or absent\n (`[\"custentity_celigo_etail_cust_id\",\"is\",\"{{{customer.id}}}\"]`\n on an order with no customer), the engine does not fail on\n the missing input: it searches the field with an `isempty`\n criterion, so the lookup now matches every target record\n whose field is BLANK. Two or more such records fail the\n record with `VALUE_LOOKUP_FAILED` — `More than one match\n for \"[\u003cfield>,\"isempty\"]\" for the record type \"\u003crecordType>\"`\n (the message quotes the search field, never the empty\n source path); none fail it as `Could not find a match`;\n exactly one silently resolves to that record. Keep an empty\n key out of the search inside the expression — branch on\n the key: `{{#if customer.id}}[[\"custentity_celigo_etail_cust_id\",\n \"is\",\"{{{customer.id}}}\"]]{{else}}[[\"internalid\",\"anyof\",\n \"\u003cfallback internal id>\"]]{{/if}}` routes records without a\n key to a fallback record while records with a key keep\n fail-on-miss; or render a value that can never match in the\n else branch and set `allowFailures: true` with `default`.\n A `searchField` / `operator` lookup (no `expression`) never\n reaches this state: an empty extract fails the record with\n `STATIC_MAPPING_ERROR` — `Exported value is empty/undefined\n for the field: $.\u003cpath>` — and resolves to `default` when\n `allowFailures` is true.\n - |-\n ## The mapping preview cannot evaluate `[*]` inside a block-helper argument\n\n `PUT /v1/netsuiteDA/previewImportMappingFields` (the mapping\n editor's Preview) evaluates lookups for real — resolved\n internal ids appear in `nlobjSublistIds`, misses and\n compile errors in `data.mappingErrors[]` — and handles\n `[*]` in plain substitutions and inside a branch's output.\n It does NOT apply the per-line evaluation to `[*]` used as a\n block-helper ARGUMENT (`{{#if line_items[*].x}}`,\n `{{#contains line_items[*].sku \"_\"}}`): the sublist comes\n back EMPTY with no error, whatever the expression would do\n in a run. An empty preview is therefore no evidence about\n such an expression — never report it as \"renders nothing\"\n or as invalid on that basis; validate it with a run.\n `\u003carray>.@index.\u003cfield>` is likewise not evaluable in the\n preview (the record is dropped with no error).\n operator:\n type: string\n description: |-\n Comparison operator for the `searchField` match. Common\n values: `is`, `contains`, `startswith`, `anyof`,\n `equalto`, `greaterthan`.\n includeInactive:\n type: boolean\n description: When true, includes inactive records in lookup results.\n x-celigo-ai-guidance:\n - |-\n When true, the search includes inactive NetSuite records.\n Default false — inactive records are excluded.\n useDefaultOnMultipleMatches:\n type: boolean\n description: |-\n When true and `allowFailures` is also true, a search that\n matches more than one record resolves to `default` instead\n of failing the record. Has no effect while `allowFailures`\n is false — an ambiguous match then fails the record whatever\n this flag says.\n x-celigo-ai-guidance:\n - |-\n Set it together with `allowFailures: true` whenever the\n search value can be empty or duplicated: with\n `allowFailures: true` alone an ambiguous match resolves to\n ONE of the matching records with no error, and with this\n flag alone the record still fails `More than one match`.\n Every ambiguous match then becomes `default`, a genuinely\n duplicated key included — when that must still fail, keep\n `allowFailures` false and handle the empty key in the\n `expression` instead (see `expression`).\n allowFailures:\n type: [boolean, 'null']\n description: |-\n When true, a lookup miss (no matching record and no\n `default`) resolves to `null` and the import continues, and\n a search matching more than one record resolves to one of\n them unless `useDefaultOnMultipleMatches` routes it to\n `default`. When false (default), a miss and an ambiguous\n match both halt the record. May be stored as null (treated\n as unset).\n map:\n type: [object, 'null']\n description: |-\n Optional static key→value object evaluated BEFORE the\n NetSuite search. When the incoming value matches a key, the\n mapped value is returned without hitting NetSuite. Null on\n dynamic (search-based) lookups with no static map.\n default:\n type: [string, 'null']\n description: |-\n Value returned when the search matches no records. When\n omitted and `allowFailures` is false, an unmatched lookup\n halts the record. May be stored as null (no default).\n description: |-\n Lookup definitions that resolve reference values from NetSuite.\n Referenced by name from field mappings via `lookupName` and from\n Handlebars expressions via `{{lookup \"name\" value}}`.\n x-celigo-ai-guidance:\n - |-\n ## Mirror invariant\n The Celigo platform persists NetSuite Distributed Import lookups in\n TWO locations: `netsuite_da.lookups` and top-level `lookups`.\n Both arrays must hold byte-for-byte identical content, so write\n both on every save.\n - |-\n ## One context for body and sublist lookups\n Every lookup's `expression` renders against the whole source\n record. A lookup referenced from a sublist row\n (`mapping.lists[].fields[].lookupName`) is evaluated once per\n line and addresses the current line with `\u003carray>[*].\u003cfield>`\n — never a bare line field. See `expression` for the failure\n modes and the preview caveat.\n x-celigo-agent:\n - lookup\n rawOverride:\n type: object\n description: |\n Raw override object for advanced use cases. When useRawOverride is true,\n this object is sent directly to the NetSuite API, bypassing normal mapping.\n useRawOverride:\n type: boolean\n description: |\n When true, uses rawOverride instead of the normal mapping configuration.\n isMigrated:\n type: boolean\n description: When true, this import was migrated from a legacy format. Set by the system.\n retryUpdateAsAdd:\n type: boolean\n description: |\n When true, if an update fails because the record doesn't exist, automatically\n retry as an add operation. Useful for initial syncs where records may not exist yet.\n customFieldMetadata:\n type: object\n description: |\n Metadata about custom fields on the target record type.\n Populated by the system from NetSuite metadata — do not set manually.\n file:\n type: object\n description: |\n File cabinet configuration for file-based imports into NetSuite.\n properties:\n name:\n type: string\n description: Filename for the file in NetSuite File Cabinet.\n fileType:\n type: string\n description: |\n NetSuite file type (e.g., \"PDF\", \"CSV\", \"PLAINTEXT\", \"EXCEL\", \"XML\").\n folder:\n type: string\n description: Folder path or name in the NetSuite File Cabinet.\n folderInternalId:\n type: string\n description: Internal ID of the target folder in NetSuite File Cabinet.\n internalId:\n type: string\n description: Internal ID of an existing file to update.\n backupFolderInternalId:\n type: string\n description: Internal ID of a backup folder for file versioning.\n isFileProvider:\n type: boolean\n description: |\n Whether this import handles files in the NetSuite File Cabinet.\n preferences:\n type: object\n properties:\n ignoreReadOnlyFields:\n type: boolean\n description: When true, silently skip read-only fields instead of raising errors.\n warningAsError:\n type: boolean\n description: When true, treat NetSuite warnings as errors that stop the import.\n skipCustomMetadataRequests:\n type: boolean\n description: When true, skip fetching custom field metadata to improve performance.\n description: |\n Import behavior preferences that control how NetSuite handles the import operation.\n recordTypeId:\n type: string\n description: |\n The internal record type ID. Used for custom record types in NetSuite where\n the numeric ID is needed in addition to the recordType string.\n # Requireds apply only to real configs, not server-written bookkeeping stubs whose\n # keys are all in {lookups, mapping, missingOrCorruptedDAConfig} — e.g.\n # {\"lookups\":[],\"mapping\":{\"fields\":[],\"lists\":[]}} or {\"missingOrCorruptedDAConfig\":true},\n # written onto imports of every adaptorType. Stack-style conditional (no allOf).\n if:\n not:\n propertyNames:\n enum: [lookups, mapping, missingOrCorruptedDAConfig]\n then:\n required:\n - operation\n - recordType\n if:\n properties:\n operation:\n enum: [update, addupdate, delete]\n required:\n - operation\n then:\n required:\n - internalIdLookup\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":65568,"content_sha256":"50521591bd2d8358755bd700bfedd7ce3feba5394d2eb52e1cfc47db9b85be41"},{"filename":"references/schemas/rdbms.yml","content":"Rdbms:\n type: object\n description: |-\n Configuration for RDBMS imports into SQL Server, MySQL, PostgreSQL, Snowflake, Oracle, MariaDB, and other relational databases.\n queryType determines which companion field is required: per_record, per_page, and first_page use the query field; bulk_insert uses the bulkInsert object; bulk_load uses the bulkLoad object — never set query and bulkInsert together.\n (The server itself stores an empty `query: []` next to bulk_insert/bulk_load configs and may leave an empty `bulkInsert: {}` stub next to others — empty companions are normal in responses; only a populated conflicting companion is invalid.)\n The query field is an array of plain SQL strings, not a single string and not an array of objects.\n x-celigo-ai-guidance:\n - |-\n ## CRITICAL: query IS AN ARRAY OF STRINGS\n The query field must be an array of plain strings, NOT a single string and NOT an array of objects.\n Correct: [\"MERGE INTO target USING ...\"]\n Wrong: \"MERGE INTO target ...\"\n Wrong: [{\"query\": \"MERGE INTO target ...\"}]\n # queryType is required on real configs only (see the conditional at the end of this\n # schema): the server writes stub `rdbms` objects ({} / {lookups:[]}) onto imports of\n # other adaptorTypes as bookkeeping.\n properties:\n lookups:\n # Live: array of lookup configs (matches common Lookups), not a string.\n $ref: '../common-schemas/lookups.yml#/Lookups'\n description: |\n Lookup definitions executed against the RDBMS connection. Each\n entry runs a `SELECT` and extracts a value used by field mappings\n (referenced via `lookupName`) or Handlebars expressions\n (referenced via `{{lookup \"name\" value}}`).\n\n **Mirror invariant**\n The Celigo platform persists lookups in TWO locations on an RDBMS\n import: top-level `lookups` and `rdbms.lookups`. Both arrays\n must hold byte-for-byte identical content, so write both on every\n save.\n items:\n type: object\n properties:\n name:\n type: string\n description: |\n Unique identifier for this lookup. Mapping fields reference it\n via `lookupName` and Handlebars expressions reference it via\n `{{lookup \"name\" value}}`. Must be unique within this\n resource's `lookups` array.\n query:\n type: string\n x-celigo-agent:\n - sql\n - handlebars\n description: |\n SQL `SELECT` statement that resolves the lookup value.\n Typically returns a single row with a single column; use\n `extract` to pick the column when the query returns multiple.\n Inject values from the incoming record with Handlebars and the\n `record.` prefix — double braces for strings, which the platform\n quotes, escaping their apostrophes, and triple braces for numbers:\n `SELECT id FROM users WHERE email = {{record.email}}`. On Snowflake\n the quoting leaves backslashes, which Snowflake reads as escape\n sequences (see `query`).\n extract:\n type: string\n description: |\n JSONPath-style expression that selects the value from the query\n result row (e.g. `id`, `details.price`). Omit when the\n query returns a single scalar — the first column is used\n automatically.\n _lookupCacheId:\n type: string\n format: objectId\n x-celigo-refModel: lookupcaches\n description: |\n Optional reference to a LookupCache resource. When set, the\n lookup reads from the cache instead of issuing a live SQL\n query — useful for stable reference data that should be\n pre-loaded.\n map:\n type: object\n description: |\n Optional static key→value object evaluated BEFORE the SQL query.\n When the incoming value matches a key, the mapped value is\n returned without querying the database. Useful for small\n controlled enumerations.\n default:\n type: string\n description: |\n Value returned when the query matches no rows. When omitted and\n `allowFailures` is false, an unmatched lookup halts the\n record.\n allowFailures:\n type: boolean\n description: |\n When true, a lookup miss (no matching row and no `default`)\n resolves to `null` and the import continues. When false\n (default), a miss halts the record.\n type: array\n x-celigo-agent:\n - lookup\n query:\n type: array\n items:\n type: string\n x-celigo-agent:\n - sql\n description: |-\n SQL statements to execute; required when queryType is `[\"per_record\"]`, `[\"per_page\"]`, or `[\"first_page\"]`, and typically contains a single statement.\n Each element is a complete SQL statement as a plain string — never a single bare string and never an array of objects (objects cause a Cast error).\n Inject values from the incoming record with Handlebars using the `record.` prefix: double braces for strings and dates (`{{record.name}}`; the platform quotes the value and escapes its apostrophes) and triple braces for numbers (`{{{record.quantity}}}`).\n Rendering is strict: a reference the render model cannot resolve (a bare `{{name}}` without prefix, or `record.*` when the arriving record is a grouped array) fails the step with `cannot_evaluate_handlebars` — it does not render an empty value.\n `record` and `rows` are aliases chosen by the arriving data's shape: an object record binds `record` (`rows` undefined); a grouped array-record (e.g. upstream `groupByFields`) binds `rows` (`record` undefined) — address grouped rows as `{{rows.0.field}}` or iterate `{{#each rows}}`.\n Double braces (`{{record.field}}`) output a string value wrapped in single quotes with embedded quotes doubled, while triple braces output it raw; block helpers such as `{{#each}}` and `{{#if}}` use double braces as normal. In Snowflake queries a JSON number or boolean renders bare under double braces too (observed in export queries).\n On Snowflake, the quoting leaves backslashes, which Snowflake reads as escape sequences: a double-brace value holding a backslash arrives changed, and `PARSE_JSON({{jsonSerialize …}})` fails on a double quote or line break in the data (observed in export queries). Pass such values in a dollar-quoted constant with triple braces (`${{{record.field}}}$`), which Snowflake reads as is, but only when the value can't contain `$`: a `$` in it ends the constant, and the rest is read as SQL.\n x-celigo-ai-guidance:\n - |-\n Use Handlebars to inject values from incoming records.\n RDBMS queries REQUIRE the `record.` prefix on every field substitution.\n - |-\n ## FORMAT — ARRAY OF STRINGS\n This field is an ARRAY OF STRINGS, not a single string and not an array of objects.\n - CORRECT: [\"INSERT INTO users (name, email) VALUES ({{record.name}}, {{record.email}})\"]\n - WRONG: \"INSERT INTO users ...\" (not an array)\n - WRONG: [{\"query\": \"INSERT INTO users ...\"}] (objects are invalid — causes Cast error)\n - WRONG: [\"INSERT INTO users (name) VALUES ({{name}})\"] (missing record. prefix — rendering is strict, so the step fails with cannot_evaluate_handlebars)\n - |-\n ## RDBMS HANDLEBARS SYNTAX\n\n - **MUST use `record.` prefix** on per-record field references:\n `{{record.fieldName}}` or `{{{record.fieldName}}}`, NOT\n `{{fieldName}}` — an unprefixed reference is undefined in the\n render model and fails the step (strict rendering).\n - **Grouped input binds `rows`, not `record`.** When the arriving\n record is a grouped array (upstream `groupByFields` or grouped\n file rows), `record` is undefined and the step fails; reference\n `{{rows.0.fieldName}}` or iterate `{{#each rows}}` instead. The\n two names are shape-dependent aliases over the same data.\n\n - **Double-brace `{{record.field}}` vs. triple-brace `{{{record.field}}}`\n is a QUOTING CHOICE, not a syntax rule.** In RDBMS context the\n platform's Handlebars renderer treats double-brace specially: it\n wraps a string value as a SQL string literal (single-quoted, with\n its apostrophes escaped), and in Snowflake export queries a number\n or boolean rendered bare, whether a JSON value or a numeric\n helper's result (`{{add 2 3}}`). Triple-brace emits the raw value\n with no quoting. In the same queries the wrapping was also seen on\n string helper output (`dateFormat`, `jsonSerialize`) and on\n `{{this}}` inside `{{#each}}`.\n\n - `{{record.name}}` → `'Alice'` (auto-single-quoted, embedded `'` doubled to `''`)\n - `{{{record.name}}}` → `Alice` (raw — never wrap it in single quotes of your own: the first apostrophe in the data ends the literal)\n - `{{record.qty}}` → `'5'` when `qty` arrives as the string `\"5\"`; a JSON number rendered bare in Snowflake export queries\n - `{{{record.qty}}}` → `5` (numeric literal either way — what you usually want for INT columns)\n\n Pick by what the SQL needs at that slot:\n\n | Slot type | Use |\n |------------------------------------------|---------------------------|\n | String column value | `{{record.field}}` |\n | Numeric column value (INT, DECIMAL, ...) | `{{{record.field}}}` |\n | Stored procedure raw arg (already typed) | `{{{record.field}}}` |\n | Identifier (table/column name) | `{{{record.field}}}` |\n | JSON literal, Snowflake (`PARSE_JSON`) | `PARSE_JSON(${{{jsonSerialize record.subobject}}}$)` (a dollar-quoted constant, read as is; only for data that can't contain `$`, which ends it) |\n | JSON literal, PostgreSQL (`JSONB`) | `{{jsonSerialize record.subobject}}` (renders the JSON as a quoted string; not yet verified on PostgreSQL) |\n\n - **Snowflake reads backslashes as escapes.** The double-brace\n quoting doubles embedded apostrophes but leaves backslashes, and\n Snowflake interprets backslash sequences inside a single-quoted\n literal: a value holding `C:\\temp` arrives with a tab in it, with\n no error, and `PARSE_JSON({{jsonSerialize …}})` fails with\n `Error parsing JSON` whenever a string in the object holds a double\n quote or a line break, because `jsonSerialize` escapes those with\n backslashes (observed in export queries). For values that can hold\n these characters, use a dollar-quoted constant with triple braces\n (`${{{record.field}}}$`), which Snowflake reads as is, but only\n when the value can't contain `$`: a `$` in it ends the constant,\n and the rest is read as SQL.\n - **Block helpers** (`{{#each}}`, `{{#if}}`, `{{/each}}`,\n `{{#unless}}`) use double braces as normal — they emit no value.\n - **Nested fields**: `{{record.properties.email}}` (auto-quoted)\n or `{{{record.properties.email}}}` (raw).\n - |-\n ## EXAMPLES\n\n - INSERT (string columns, auto-quoted): `[\"INSERT INTO customers (name, email) VALUES ({{record.name}}, {{record.email}})\"]`\n - INSERT (mixed numeric + string): `[\"INSERT INTO orders (qty, sku) VALUES ({{{record.qty}}}, {{record.sku}})\"]`\n - UPDATE (numeric SET, string WHERE): `[\"UPDATE inventory SET qty = {{{record.qty}}} WHERE sku = {{record.sku}}\"]`\n - UPSERT (MySQL): `[\"INSERT INTO users (id, name) VALUES ({{{record.id}}}, {{record.name}}) ON DUPLICATE KEY UPDATE name = {{record.name}}\"]`\n - MERGE (Snowflake): `[\"MERGE INTO target USING (SELECT {{record.email}} AS email) AS src ON target.email = src.email WHEN MATCHED THEN UPDATE SET name = {{record.name}}\"]`\n - Stored procedure (raw args): `[\"call netsuite.transformation_data_import({{record.record_type}}, 20)\"]`\n - JSON literal column (Snowflake VARIANT; the data must not contain `$`): `[\"INSERT INTO events (payload) SELECT PARSE_JSON(${{{jsonSerialize record.financialEvents}}}$)\"]`\n - Per-page batch (different Handlebars context — see `queryType: per_page` below): `[\"{{#each batch_of_records}}UPDATE staging SET done=1 WHERE id={{{this.record.id}}};{{/each}}\"]`\n examples:\n - [\"INSERT INTO orders (id, total) VALUES ({{{record.id}}}, {{{record.total}}})\"]\n - [\"{{#each batch_of_records}}UPDATE customers SET name={{record.name}} WHERE id={{{record.id}}};{{/each}}\"]\n queryType:\n type: [array, 'null']\n items:\n type: string\n enum:\n - INSERT\n - UPDATE\n - bulk_insert\n - per_record\n - per_page\n - first_page\n - bulk_load\n x-enumDescriptions:\n INSERT: Legacy value for insert operations; use bulk_insert or per_record instead.\n UPDATE: Legacy value for update operations; use per_page or per_record instead.\n bulk_insert: Batch INSERT via multi-row VALUES clause for efficient bulk data loading.\n per_record: Execute one SQL statement per record, giving full control over individual record SQL.\n per_page: Execute one SQL statement per page (batch) of records.\n first_page: Execute the SQL statement once for the first page only, typically for setup or cleanup scripts.\n bulk_load: Stage data as a file and load via database-native COPY/bulk mechanism for maximum throughput.\n x-celigo-ui-override: >-\n Required by the RDBMS import form (rdbms.queryType is required:true). The form defaults a\n new import to bulk_load wherever it offers that option — Snowflake always; Google BigQuery,\n Microsoft SQL Server (cloud connections only), Azure Synapse and Amazon Redshift when the\n account's enableSizeBasedAggregation feature flag is on for that database — and to\n bulk_insert otherwise. Encoded to mirror the form so builders produce connectable\n configurations.\n description: |-\n Execution strategy for the SQL operation, which determines the required companion field: per_record, per_page, and first_page require `query`; bulk_insert requires `bulkInsert`; bulk_load requires `bulkLoad`.\n Prefer the highest-performance type the operation supports: bulk_load (Snowflake, Google BigQuery, Microsoft SQL Server on a cloud connection, Azure Synapse and Amazon Redshift — handles INSERT, upsert via `bulkLoad.primaryKeys`, and custom merge or ignore-existing logic via `bulkLoad.overrideMergeQuery`), then bulk_insert for pure INSERTs, then per_page, then per_record only when the logic cannot be expressed as a batch operation.\n For UPDATE, UPSERT, or ignore-existing logic on databases without bulk_load support, use per_page or per_record with `query` — bulk_insert has no duplicate-checking logic. Prefer per_page when the statement can loop `batch_of_records`, and use per_record when the SQL can't be written as one batch statement.\n Set a single value; do not combine types or use the legacy INSERT and UPDATE values.\n per_page statements render against the whole batch (`batch_of_records`) rather than a single record — loop with `{{#each batch_of_records}}...{{record.fieldName}}...{{/each}}`, with the same quoting as `query` (double braces for strings, triple for numbers).\n\n `bulk_load` — Stages the records as files and loads them with the database's native COPY / bulk mechanism for maximum throughput; supports INSERT, upsert via bulkLoad.primaryKeys, and custom merge or ignore-existing logic via bulkLoad.overrideMergeQuery; available on Snowflake, Google BigQuery, Microsoft SQL Server (cloud connections), Azure Synapse and Amazon Redshift (NetSuite Analytics Warehouse has it through the JDBC import). The import form calls it \"Use optimized bulk load (recommended for larger imports)\" and, beside it, calls bulk_insert \"Use batch insert SQL query\".\n `bulk_insert` — Batch INSERT via multi-row VALUES clause for efficient bulk data loading; has no upsert or ignore-existing logic and is available for all RDBMS types.\n `per_page` — Executes one SQL statement per page (batch) of records; available for all RDBMS types.\n `per_record` — Executes one SQL statement per record, giving full control over individual record SQL; available for all RDBMS types.\n x-celigo-ai-guidance:\n - '**CRITICAL: This field DETERMINES whether `query` or `bulkInsert` is required.**'\n - |-\n ## DECISION TREE (Follow in order):\n\n **STEP 0: Is this an update of an existing import?** Keep its query type unless the user asks to change it — an import set to bulk_insert stays bulk_insert when the request is about something else (the query type is additive and opt-in; nothing migrates). The steps below are for a NEW import or an explicit request to change the type.\n\n **STEP 1: Does the database support bulk_load?**\n - Snowflake, Google BigQuery, Microsoft SQL Server (a cloud connection — not an on-premise agent connection), Azure Synapse and Amazon Redshift do (NetSuite Analytics Warehouse does through the JDBC import):\n → **USE `[\"bulk_load\"]`** with:\n - `bulkLoad.tableName` (required — omitting it fails the save with 422 `missing_required_field` \"rdbms imports require a tableName.\")\n - `bulkLoad.primaryKeys` for upsert/merge (auto-generates MERGE)\n - `bulkLoad.overrideMergeQuery: true` for custom SQL (ignore-existing, conditional updates, multi-table ops). The override SQL goes in `query` and references `{{preMergeTemporaryTable}}` (also written `{{import.rdbms.bulkLoad.preMergeTemporaryTable}}`) for the staging table and `{{tableName}}` for the destination.\n - No `primaryKeys` for pure INSERT (auto-generates INSERT)\n - No `bulkInsert` beside it (the server drops a `bulkInsert` sent with bulk_load)\n - **Amazon Redshift needs no AWS setup for bulk_load**: Celigo stages the files in Celigo-managed encrypted storage and Redshift loads them with COPY — no S3 bucket, IAM role, cluster role attachment or extra connection field. The save is accepted on either `redshift.authType` (accesskey and awsIam) and against a provisioned cluster or a Serverless workgroup alike. Never steer a Redshift import to bulk_insert for lack of setup.\n - The import form shows the bulk_load option only when the account's `enableSizeBasedAggregation` feature flag is on for that database (BigQuery, SQL Server, Synapse, Redshift; Snowflake always shows it). The flag is the form's switch, not the API's — a Redshift bulk_load import saves on an account whose flag has no redshift entry (off, to the form) — so author it, and tell the user the option appears on their import form once the flag is on for the account.\n\n **STEP 2: Database does NOT support bulk_load**\n - **Pure INSERT**: → **USE `[\"bulk_insert\"]`** (requires `bulkInsert` object)\n - **UPDATE / UPSERT / custom matching logic**: → **USE `[\"per_page\"]`** (requires `query` field). Only use `[\"per_record\"]` if the logic truly cannot be expressed as a batch operation.\n - |-\n ## DO NOT USE:\n - `[\"INSERT\"]` or `[\"UPDATE\"]` as standalone values\n - Multiple types combined\n - |-\n ## LEGACY VALUES ARE WRITE-REJECTED:\n Saving an import whose queryType contains INSERT (or the retired MERGE /\n COPY / \"BULK INSERT\" values) fails with 422 `Import query types \"BULK\n INSERT\", \"INSERT\", \"MERGE\", \"COPY\" are deprecated. Please change your\n Import to use new query types \"bulk_insert\", \"per_record\", \"per_page\",\n \"first_page\" instead`. Strings outside the enum entirely (e.g. lowercase\n \"insert\") fail with 422 `is not a valid enum value for path\n rdbms.queryType.0`.\n - |-\n ## EXAMPLES:\n\n ### INSERT with ignoreExisting (MUST use per_record)\n Prompt: \"Import customers, ignore existing based on email\"\n ```json\n \"queryType\": [\"per_record\"],\n \"query\": [\"INSERT INTO customers (name, email) VALUES ({{record.name}}, {{record.email}})\"]\n ```\n\n ### Pure INSERT (no duplicate checking)\n Prompt: \"Insert all records into users table\"\n ```json\n \"queryType\": [\"bulk_insert\"],\n \"bulkInsert\": { ... }\n ```\n\n ### UPDATE Operation\n Prompt: \"Update inventory counts\"\n ```json\n \"queryType\": [\"per_page\"],\n \"query\": [\"{{#each batch_of_records}}UPDATE inventory SET qty = {{{record.qty}}} WHERE sku = {{record.sku}};{{/each}}\"]\n ```\n Use `\"queryType\": [\"per_record\"]` with `\"query\": [\"UPDATE inventory SET qty = {{{record.qty}}} WHERE sku = {{record.sku}}\"]` when the SQL can't be written as one batch statement.\n\n ### Run Once Per Page\n Prompt: \"Execute this stored procedure for each page of results\"\n ```json\n \"queryType\": [\"per_page\"]\n ```\n **per_page uses a different Handlebars context than per_record.** The context is the entire batch (`batch_of_records`), not a single `record.` object. Loop through with `{{#each batch_of_records}}...{{record.fieldName}}...{{/each}}`, quoting as in per_record: double braces for strings, triple braces for numbers.\n\n ### Run Once at Start\n Prompt: \"Run this cleanup script before processing\"\n ```json\n \"queryType\": [\"first_page\"]\n ```\n bulkInsert:\n type: object\n description: |-\n Configuration for batch INSERTs via a multi-row VALUES clause; required when queryType is `[\"bulk_insert\"]` and must not be set otherwise.\n Only suited to pure INSERTs — for UPDATE or UPSERT, use queryType `[\"per_page\"]` (a statement that loops `batch_of_records`) or `[\"per_record\"]` with the `query` field instead, and for ignore-existing/skip-duplicates logic `[\"per_record\"]`.\n x-celigo-ai-guidance:\n - |-\n **CRITICAL: This object is ONLY used when `queryType` is `[\"bulk_insert\"]`.**\n\n ✅ **SET THIS** when:\n - `queryType` is `[\"bulk_insert\"]`\n - Pure INSERT operation with NO ignoreExisting/duplicate checking\n\n ❌ **DO NOT SET THIS** when:\n - `queryType` is `[\"per_page\"]` or `[\"per_record\"]` (use the `query` field instead)\n - Operation involves UPDATE, UPSERT, or ignoreExisting logic\n\n If the prompt mentions \"ignore existing\", \"skip duplicates\", or \"match on field\",\n use `queryType: [\"per_record\"]` with the `query` field instead of this object.\n properties:\n tableName:\n type: string\n description: |-\n The name of the database table into which the bulk insert operation will be executed. This value must correspond to a valid, existing table within the target relational database management system (RDBMS). It serves as the primary destination for inserting multiple rows of data efficiently in a single operation. The table name can include schema or namespace qualifiers if supported by the database (e.g., \"schemaName.tableName\"), allowing precise targeting within complex database structures. Proper validation and sanitization of this value are essential to ensure the operation's success and to prevent SQL injection or other security vulnerabilities.\n x-celigo-ai-guidance:\n - |-\n **Field behavior**\n - Specifies the exact destination table for the bulk insert operation.\n - Must correspond to an existing table in the database schema.\n - Case sensitivity and naming conventions depend on the underlying RDBMS.\n - Supports schema-qualified names where applicable.\n - Used directly in the SQL `INSERT INTO` statement.\n - Influences how data is mapped and inserted during the bulk operation.\n - |-\n **Dependency chain**\n - Relies on the database connection and authentication configuration.\n - Interacts with other bulkInsert parameters such as column mappings and data payload.\n - May be influenced by transaction management, locking mechanisms, and database constraints during the insert.\n - Dependent on the database's metadata for validation and existence checks.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Verify the existence and accessibility of the table in the target database before execution.\n - Ensure compliance with the RDBMS naming rules, including reserved keywords, allowed characters, and maximum length.\n - Support and correctly handle schema-qualified table names, respecting database-specific syntax.\n - Sanitize and validate input rigorously to prevent SQL injection and other security risks.\n - Apply appropriate quoting or escaping mechanisms based on the RDBMS (e.g., backticks for MySQL, double quotes for PostgreSQL).\n - Consider the impact of case sensitivity, especially when dealing with quoted identifiers.\n - |-\n ## EXAMPLES:\n - \"users\"\n - \"public.orders\"\n - \"inventory.products\"\n - \"sales_data_2024\"\n - \"dbo.CustomerRecords\"\n - |-\n ## IMPORTANT NOTES:\n - The target table must exist and have the appropriate schema and permissions to accept bulk inserts.\n - Incorrect, misspelled, or non-existent table names will cause the operation to fail.\n - Case sensitivity varies by database; for example, PostgreSQL treats quoted identifiers as case-sensitive.\n - Avoid using unvalidated dynamic input to mitigate security vulnerabilities.\n - Schema qualifiers should be used consistently to avoid ambiguity in multi-schema environments.\n examples: [\"public.orders\", \"dbo.Customers\"]\n batchSize:\n type: string\n description: |-\n The number of records to be inserted into the database in a single batch during a bulk insert operation. This parameter is crucial for optimizing the performance and efficiency of bulk data loading by controlling how many records are grouped together before being sent to the database. Proper tuning of batchSize balances memory consumption, transaction overhead, and throughput, enabling the system to handle large volumes of data efficiently without overwhelming resources or causing timeouts. Adjusting batchSize directly impacts transaction size, network utilization, error handling granularity, and recovery strategies, making it essential to tailor this value based on the specific database capabilities, system resources, and workload characteristics.\n x-celigo-ai-guidance:\n - |-\n **Field behavior**\n - Specifies the exact count of records processed and committed in a single batch during bulk insert operations.\n - Determines the frequency and size of database transactions, influencing overall throughput, latency, and system responsiveness.\n - Larger batch sizes can improve throughput but may increase memory usage, transaction duration, and risk of timeouts or locks.\n - Smaller batch sizes reduce memory footprint and transaction time but may increase the total number of transactions and associated overhead.\n - Defines the scope of error handling, as failures typically affect only the current batch, allowing for partial retries or rollbacks.\n - Controls the granularity of commit points, impacting rollback, recovery, and consistency strategies in case of failures.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Choose a batch size that respects the database's transaction limits, available system memory, and network conditions.\n - Conduct benchmarking and load testing with different batch sizes to identify the optimal balance for your environment.\n - Monitor system performance continuously and adjust batchSize dynamically if supported, to adapt to varying workloads.\n - Implement comprehensive error handling to manage partial batch failures, including retry logic or compensating transactions.\n - Verify compatibility with database drivers, ORM frameworks, or middleware, which may impose constraints or optimizations on batch sizes.\n - Consider the size, complexity, and serialization overhead of individual records, as larger or more complex records may necessitate smaller batches.\n - Factor in network latency and bandwidth to optimize data transfer efficiency and reduce potential bottlenecks.\n - |-\n ## EXAMPLES:\n - 1000: Suitable for moderate bulk insert operations, balancing speed and resource consumption effectively.\n - 50000: Ideal for high-throughput environments with ample memory and finely tuned database configurations.\n - 100: Appropriate for systems with limited memory or where minimizing transaction size and duration is critical.\n - 5000: A common default batch size providing a good compromise between performance and resource usage.\n bulkLoad:\n type: object\n properties:\n tableName:\n type: string\n description: |-\n Target table for the bulk load.\n Must reference an existing table accessible with the connection's credentials; schema-qualified names are supported.\n x-celigo-ai-guidance:\n - |-\n The name of the target table within the relational database management system\n (RDBMS) where the bulk load operation will insert,\n update, or merge data.\n This property specifies the exact destination table for the bulk data operation\n and must correspond to a valid,\n existing table in the database schema.\n It can include schema qualifiers if supported by the database (e.g.,\n schema.tableName),\n and must adhere to the naming conventions and case sensitivity rules of the target RDBMS.\n Proper specification of this property is critical to ensure data is loaded into\n the correct location without errors or unintended data modification.\n Accurate definition of the table name helps maintain data integrity and prevents\n operational failures during bulk load processes.\n - |-\n ## FIELD BEHAVIOR:\n - Identifies the specific table in the database to receive the bulk-loaded data.\n - Must reference an existing table accessible with the current database credentials.\n - Supports schema-qualified names where applicable (e.g., schema.tableName).\n - Used as the target for insert, update, or merge operations during bulk load.\n - Case sensitivity and naming conventions depend on the underlying database system.\n - Determines the scope and context of the bulk load operation within the database.\n - Influences transactional behavior, locking, and concurrency controls during the operation.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Verify the target table exists and the executing user has sufficient permissions for data modification.\n - Ensure the table name respects the case sensitivity and naming rules of the RDBMS.\n - Avoid reserved keywords or special characters unless properly escaped or quoted according to database requirements.\n - Support fully qualified names including schema or database prefixes if required to avoid ambiguity.\n - Validate compatibility between the table schema and the incoming data to prevent type mismatches or constraint violations.\n - Consider transactional behavior, locking, and concurrency implications on the target table during bulk load.\n - Test the bulk load operation in a development or staging environment to confirm correct table targeting.\n - Implement error handling to catch and report issues related to invalid or inaccessible table names.\n - |-\n ## EXAMPLES:\n - \"customers\"\n - \"sales_data_2024\"\n - \"public.orders\"\n - \"inventory.products\"\n - \"dbo.EmployeeRecords\"\n - |-\n ## IMPORTANT NOTES:\n - Incorrect or misspelled table names will cause the bulk load operation to fail.\n - The target table schema must be compatible with the incoming data structure to avoid errors.\n - Bulk loading may overwrite, append, or merge data depending on the load mode; ensure this aligns with business requirements.\n - Some databases require quoting or escaping of table names, especially if they\n examples: [\"public.orders\", \"ANALYTICS.STG_ORDERS\"]\n primaryKeys:\n type: array\n items:\n type: string\n description: |-\n Columns that uniquely identify each record in the target table, used to match existing rows during the load.\n Set for upsert behavior (a MERGE is auto-generated); omit (or empty) for a pure INSERT.\n Composite keys are supported — list every key column, and names must exactly match the target schema.\n x-celigo-ai-guidance:\n - |-\n primaryKeys specifies the list of column names that uniquely identify each\n record in the target relational database table during a bulk load operation.\n These keys are essential for maintaining data integrity by enforcing uniqueness\n constraints and enabling precise identification of rows for operations such as\n inserts,\n updates, upserts, or conflict resolution.\n Properly defining primaryKeys ensures that duplicate records are detected and\n handled appropriately,\n preventing data inconsistencies and supporting efficient data merging processes.\n This property supports both single-column and composite primary keys,\n requiring the exact column names as defined in the target schema,\n and plays a critical role in guiding the bulk load mechanism to correctly match\n and manipulate records based on their unique identifiers.\n - |-\n ## FIELD BEHAVIOR:\n - Defines one or more columns that collectively serve as the unique identifier for each row in the table.\n - Enforces uniqueness constraints during bulk load operations to prevent duplicate entries.\n - Facilitates detection and resolution of conflicts or duplicates during data insertion or updating.\n - Influences the behavior of upsert, merge, or conflict resolution mechanisms in the bulk load process.\n - Ensures that each record can be reliably matched and updated based on the specified keys.\n - Supports both single-column and composite keys, maintaining the order of columns as per the database schema.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Include all columns that form the complete primary key, especially for composite keys, maintaining the correct order as defined in the database schema.\n - Ensure column names exactly match those in the target database, respecting case sensitivity where applicable.\n - Validate that all specified primary key columns exist in both the source data and the target table schema before initiating the bulk load.\n - Avoid leaving the primaryKeys list empty when uniqueness enforcement or conflict resolution is required.\n - Consider the immutability and stability of primary key columns to prevent inconsistencies during repeated load operations.\n - Confirm that the primary key columns are indexed or constrained appropriately in the target database to optimize performance.\n - |-\n ## EXAMPLES:\n - [\"id\"] — single-column primary key.\n - [\"order_id\", \"product_id\"] — composite primary key consisting of two columns.\n - [\"user_id\", \"timestamp\"] — composite key combining user identifier and timestamp for uniqueness.\n - [\"customer_id\", \"account_number\", \"region\"] — multi-column composite primary key.\n - |-\n ## IMPORTANT NOTES:\n - Omitting primaryKeys when required can result in data duplication, failed loads, or incorrect conflict handling.\n examples:\n - [\"id\"]\n - [\"INTERNAL_ID\", \"LOCATION_INTERNAL_ID\"]\n overrideMergeQuery:\n type: boolean\n description: |-\n When true, a custom merge query replaces the auto-generated MERGE statement, enabling ignore-existing logic, conditional updates, or multi-table operations (\"Override merge/insert statement\" on the import form).\n The override SQL lives in `query` and references `{{preMergeTemporaryTable}}` (also written `{{import.rdbms.bulkLoad.preMergeTemporaryTable}}`) for the staging table and `{{tableName}}` for the destination table.\n The SQL itself in this field fails the save: 422 `invalid_field` \"Invalid value '\u003cthe statement>' for field 'rdbms.bulkLoad.overrideMergeQuery'. Reason: CastError.\"\n x-celigo-ai-guidance:\n - |-\n This is the switch, not the SQL. Set it to true and put the statement in `query` (one string); the import form shows the SQL box only while \"Override merge/insert statement\" is checked. With it false or absent the platform generates the INSERT / MERGE from `bulkLoad.tableName` and `bulkLoad.primaryKeys`, and `query` stays empty.\n - |-\n The statement runs after the files are loaded into the staging table: read from `{{preMergeTemporaryTable}}` (also written `{{import.rdbms.bulkLoad.preMergeTemporaryTable}}`) and write to `{{tableName}}`; the form warns when the staging-table placeholder is missing. Use it for ignore-existing, conditional updates or multi-table work the generated MERGE cannot express — for example `INSERT INTO {{tableName}} (id, total) SELECT id, total FROM {{preMergeTemporaryTable}}`.\n aggregation:\n type: object\n description: |-\n How the staged files are cut and handed to the database. The import form writes it for a new bulk-load import when the account's `enableSizeBasedAggregation` flag is on for that database (Snowflake included — its form offers bulk_load without the flag but writes `aggregation` only with it); the server fills `type` and `maxFileSizeInMB` when they are omitted, so `loadDataAsAvailable` is the only value to author.\n properties:\n type:\n type: string\n enum: [size]\n x-enumDescriptions:\n size: Files are cut by size\n description: Filled by the server on save; the only mode is size-based cutting.\n maxFileSizeInMB:\n type: integer\n description: Size of each staged file in MB; the server fills 250 when the field is omitted and the import form never sets it — leave it out.\n examples: [250]\n loadDataAsAvailable:\n type: boolean\n default: false\n description: |-\n When true, each staged file is sent for loading as soon as it is written, so the dataset arrives as several smaller loads (\"Load data as available\" on the import form). When false, every file is held until all are ready and they are loaded together.\n x-celigo-ai-guidance:\n - |-\n Send a real boolean and default to false unless the user asks for files to load as they are ready: a non-boolean value (for example the string \"yes\") makes the server drop the whole `aggregation` block without an error.\n - |-\n The block outlives a query-type change — after a PUT that switches the import to bulk_insert and omits `bulkLoad`, the read-back still carries `bulkLoad.aggregation`. That is server behavior, not a failed write; do not re-send the document to clear it.\n description: |-\n Configuration for bulk loading: the records are staged as files and loaded with the database's native COPY / bulk mechanism for maximum throughput.\n Required when queryType is `[\"bulk_load\"]`; available on Snowflake, Google BigQuery, Microsoft SQL Server (cloud connections), Azure Synapse and Amazon Redshift.\n x-celigo-ai-guidance:\n - |-\n Specifies whether bulk loading is enabled for relational database management\n system (RDBMS) operations,\n allowing for the efficient insertion of large volumes of data through a single,\n optimized operation.\n Enabling bulkLoad significantly improves performance during data import,\n migration,\n or batch processing by minimizing the overhead associated with individual row\n inserts and leveraging database-specific bulk insert mechanisms.\n This feature is particularly beneficial for initial data loads,\n large-scale migrations, or periodic batch updates where speed,\n resource efficiency, and reduced transaction time are critical.\n Bulk loading may temporarily alter database behavior—such as disabling indexes,\n constraints, or triggers—to maximize throughput,\n and often requires elevated permissions and careful management of transactional\n integrity to ensure data consistency.\n Proper use of bulkLoad can lead to substantial reductions in processing time and\n system resource consumption during large data operations.\n - |-\n ## FIELD BEHAVIOR:\n - When set to true, the system uses optimized bulk loading techniques to insert data in large batches, greatly enhancing throughput and efficiency.\n - When false or omitted, data insertion defaults to standard row-by-row operations, which may be slower and consume more resources.\n - Typically enabled during scenarios involving initial data population, large batch imports, or data migration processes.\n - May temporarily disable or defer enforcement of indexes, constraints, and triggers to improve performance during the bulk load operation.\n - Can affect database locking and concurrency, potentially locking tables or partitions for the duration of the bulk load.\n - Bulk loading operations may bypass certain transactional controls, affecting rollback and error recovery behavior.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Verify that the target RDBMS supports bulk loading and understand its specific syntax, capabilities, and limitations.\n - Assess transaction management implications, as bulk loading may alter or bypass triggers, constraints, and rollback mechanisms.\n - Implement robust error handling and post-load validation to ensure data integrity and consistency after bulk operations.\n - Monitor system resources such as CPU, memory, and I/O throughput during bulk load to avoid performance bottlenecks or outages.\n - Plan for potential impacts on database availability, locking behavior, and concurrent access during bulk load execution.\n - Ensure data is properly staged and preprocessed to meet the format and requirements of the bulk loading mechanism.\n - Coordinate bulk loading with maintenance windows or low-traffic periods to minimize disruption.\n updateLookupName:\n type: [string, 'null']\n description: Name of the lookup used for update operations in the legacy composite (queryType1) flow.\n x-celigo-ai-guidance:\n - |-\n Specifies the exact name of the lookup table or entity within the relational\n database management system (RDBMS) that is targeted for update operations.\n This property serves as a precise identifier to determine which lookup data set\n should be modified,\n ensuring accurate and efficient targeting within the database schema.\n It typically corresponds to a physical table name or a logical entity name\n defined in the database and must strictly align with the existing schema to\n enable successful updates without errors or unintended side effects.\n Proper use of this property is critical for maintaining data integrity during\n update operations on lookup data,\n as it directly influences which records are affected.\n Accurate specification helps prevent accidental data corruption and supports clear,\n maintainable database interactions.\n - |-\n ## FIELD BEHAVIOR:\n - Defines the specific lookup table or entity to be updated during an operation.\n - Acts as a key identifier for the update process to locate the correct data set.\n - Must be provided when performing update operations on lookup data.\n - Influences the scope and effect of the update by specifying the target entity.\n - Changes to this value directly affect which data is modified in the database.\n - Invalid or missing values will cause update operations to fail or target unintended data.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Ensure the value exactly matches the existing lookup table or entity name in the database schema.\n - Validate against the RDBMS naming conventions, including allowed characters, length restrictions, and case sensitivity.\n - Maintain consistent naming conventions across the application to prevent ambiguity and errors.\n - Account for case sensitivity based on the underlying database system's configuration and collation settings.\n - Avoid using reserved keywords, special characters, or whitespace that may cause conflicts or syntax errors.\n - Implement validation checks to catch misspellings or invalid names before executing update operations.\n - Incorporate error handling to manage cases where the specified lookup name does not exist or is inaccessible.\n - |-\n ## EXAMPLES:\n - \"country_codes\"\n - \"user_roles\"\n - \"product_categories\"\n - \"status_lookup\"\n - \"department_list\"\n - |-\n ## IMPORTANT NOTES:\n - Providing an incorrect or misspelled name will result in failed update operations or runtime errors.\n - This field must not be left empty when an update operation is intended.\n - It is only relevant and required when performing update operations on lookup tables or entities.\n - Changes to this value should be carefully managed to avoid unintended data modifications or corruption.\n - Ensure appropriate permissions exist for updating the specified lookup entity to prevent authorization failures.\n - Consistent use of this property across environments (development, staging, production) is essential\n updateExtract:\n type: [string, 'null']\n description: Path used to extract the value that drives update operations in the legacy composite flow.\n ignoreLookupName:\n type: [string, 'null']\n description: |-\n Name of an entry in `lookups` that probes the database per record to decide whether it\n already exists, so `ignoreExisting` can skip it (or `ignoreMissing` can skip records with\n no match). Use it when existence must be read from the database; use `ignoreExtract` when\n the incoming record already carries the field that signals existence.\n ignoreExtract:\n type: [string, 'null']\n description: |-\n Path to a field on the incoming record whose presence decides the skip: with\n `ignoreExisting`, a populated value marks the record as existing and it is skipped; with\n `ignoreMissing`, an empty value marks it as missing and it is skipped. Zero per-record\n overhead; set exactly one of `ignoreExtract` / `ignoreLookupName`.\n # Shape → requireds. Nested if/else-if (no allOf), stack-style.\n # Outer gate: queryType is required only on real configs — the server writes stub\n # `rdbms` objects ({} / {lookups:[]}) onto imports of every adaptorType.\n # Inner chain: queryType → required companion field. The `type: array` guard keeps a\n # null queryType from vacuously matching `contains` (JSON Schema `contains` passes on\n # non-arrays), which would force a companion field the nullable type deliberately\n # leaves optional.\n # Mutual exclusion is expressed as \"the other companion must be EMPTY if present\"\n # (maxItems/maxProperties 0), NOT as `not: required`: the server itself writes empty\n # bookkeeping companions — POSTing {queryType:[\"bulk_insert\"], bulkInsert:{...}} with\n # no query stores query:[] alongside, and legacy docs\n # carry bulkInsert:{} stubs next to per_record/bulk_load configs. A REAL companion\n # (non-empty query with bulk_insert, populated bulkInsert with per_record) still fails.\n if:\n not:\n propertyNames:\n enum: [lookups]\n then:\n required:\n - queryType\n if:\n required:\n - queryType\n properties:\n queryType:\n type: array\n contains:\n enum: [per_record, per_page]\n then:\n required:\n - query\n properties:\n bulkInsert:\n maxProperties: 0\n else:\n if:\n required:\n - queryType\n properties:\n queryType:\n type: array\n contains:\n const: first_page\n then:\n required:\n - query\n else:\n if:\n required:\n - queryType\n properties:\n queryType:\n type: array\n contains:\n const: bulk_insert\n then:\n required:\n - bulkInsert\n properties:\n query:\n maxItems: 0\n else:\n if:\n required:\n - queryType\n properties:\n queryType:\n type: array\n contains:\n const: bulk_load\n then:\n required:\n - bulkLoad\n # The one populated query that belongs beside bulk_load is the override\n # statement: allowed only while bulkLoad.overrideMergeQuery is true.\n if:\n required:\n - bulkLoad\n properties:\n bulkLoad:\n type: object\n required:\n - overrideMergeQuery\n properties:\n overrideMergeQuery:\n const: true\n then:\n properties:\n query:\n type: array\n else:\n properties:\n query:\n maxItems: 0\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":52215,"content_sha256":"6e9213d68fe7c266c9174196be87fe2a9fe020f005b6c6dd2ab7e3005f134f52"},{"filename":"references/schemas/request.yml","content":"Request:\n type: object\n description: |-\n Fields that can be sent when creating or updating an import. Set the adaptor-specific\n configuration object matching `adaptorType` (e.g. `netsuite_da` for `NetSuiteDistributedImport`).\n `_connectionId` is required except for the connection-less flavors (`ToolImport`,\n `AiAgentImport`, `GuardrailImport`, `TopicImport`): a `ToolImport` binds connections through\n the referenced tool's `overrides`, AI agent / guardrail imports only use a connection for\n BYOK, and an Event Streams publisher (`TopicImport`) is delivered by the platform itself.\n required:\n - name\n allOf:\n - $ref: './base.yml#/ImportBase'\n # _connectionId is waived for the connection-less flavors, recognized either by an explicit\n # adaptorType or by the presence of the flavor's config subdoc (the server infers the\n # adaptorType from that subdoc when it is omitted). Every other import still requires it.\n if:\n anyOf:\n - required: [adaptorType]\n properties:\n adaptorType:\n enum: [ToolImport, AiAgentImport, GuardrailImport, TopicImport]\n - required: [tool]\n - required: [aiAgent]\n - required: [guardrail]\n - required: [topic]\n then:\n # Event Streams publisher: the API saves a TopicImport with an empty `topic` (batchSize\n # default only); the topic binding is the form's rule.\n if: { properties: { adaptorType: { const: TopicImport } }, required: [adaptorType] }\n then:\n x-celigo-ui-override: >-\n Required by the Event Streams publisher form (fieldDefinitions/resources/imports/import.js:\n 'topic._topicId' is required:true). Encoded to mirror the form so builders produce\n publishers that are bound to a topic.\n required: [topic]\n properties:\n topic:\n required: [_topicId]\n else:\n required: [_connectionId]\n # NetSuite REST APIs reuse the `http` object on a netsuite connection; `nsDomainType` picks\n # the host and is 422-required (\"Path `nsDomainType` is required.\").\n if: { properties: { adaptorType: { const: NetSuiteHTTPImport } }, required: [adaptorType] }\n then: { required: [http, nsDomainType] }\n else:\n # Cloud file providers add a location rule keyed on `assistant` (the import-level field\n # the provider forms set). Form-derived: the API saves either shape without a directory.\n if: { properties: { assistant: { const: azuredatalakestoragegen2 } }, required: [assistant] }\n then:\n x-celigo-ui-override: >-\n Required by the Azure Data Lake Storage Gen2 import form\n (fieldDefinitions/resources/imports/azuredatalakestoragegen2.js: container\n 'file.directory.storageRootId_adls' is required:true and preSave persists it as\n file.directory.name; the full file path is required:true and preSave splits it into\n http.relativeURI (directory prefix) and file.fileName (basename)). Encoded to mirror\n the form so builders produce connectable configurations.\n required: [file]\n properties:\n file:\n required: [directory, fileName]\n properties:\n directory:\n required: [name]\n else:\n if: { properties: { assistant: { const: googlecloudstorage } }, required: [assistant] }\n then:\n x-celigo-ui-override: >-\n Required by the Google Cloud Storage import form\n (fieldDefinitions/resources/imports/googlecloudstorage.js: bucket\n 'file.directory.storageRootId_gcs' is required:true). Encoded to mirror the form so\n builders produce connectable configurations.\n required: [file]\n properties:\n file:\n required: [directory]\n properties:\n directory:\n required: [storageRootId]\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":3865,"content_sha256":"a719a0e9fd9d967866d30addf3bf0ba7012b283055b819635c29a64b1b3bb173"},{"filename":"references/schemas/response.yml","content":"Import:\n type: object\n required:\n - _id\n - name\n - adaptorType\n - apiIdentifier\n - createdAt\n - lastModified\n description: Import object as returned by the API.\n allOf:\n - $ref: './base.yml#/ImportBase'\n - $ref: '../common-schemas/resource-response.yml#/ResourceResponse'\n - $ref: '../common-schemas/ia-resource-response.yml#/IAResourceResponse'\n - type: object\n properties:\n aiDescription:\n $ref: '../common-schemas/ai-description.yml#/AIDescription'\n apim:\n $ref: '../common-schemas/apim.yml#/APIM'\n apiIdentifier:\n type: string\n readOnly: true\n description: API identifier assigned to this import.\n sandbox:\n type: boolean\n deprecated: true\n readOnly: true\n description: When true, this import belongs to a sandbox account.\n rest:\n type: object\n deprecated: true\n readOnly: true\n additionalProperties: true\n description: |-\n Legacy REST adaptor configuration, still returned on imports created before\n the REST-to-HTTP migration (`adaptorType: RESTImport`). Mirrors the shape of\n `http` with per-operation arrays (`method`, `relativeURI`, `body`,\n `responseIdPath`). On write the platform maintains the equivalent `http`\n configuration — configure new imports through `http` instead.\n _sourceId:\n type: string\n format: objectId\n readOnly: true\n description: Reference to the source resource this import was created from.\n _templateId:\n type: string\n format: objectId\n readOnly: true\n x-celigo-refModel: templates\n description: Template this import was created from.\n draft:\n type: boolean\n readOnly: true\n description: When true, this import is in draft state and has not been confirmed.\n draftExpiresAt:\n type: string\n format: date-time\n readOnly: true\n description: Timestamp when the draft version of this import expires.\n debugUntil:\n type: string\n format: date-time\n readOnly: true\n description: |-\n Turns on step debug capture until this time: while it lies in the future, every\n request this import sends — or, for a `ToolImport`, every record passing through\n it with the tool's output — is recorded in the step's debug log\n (`GET /v1/{flows|apis|tools}/{containerId}/{_id}/requests`). Read-only on PUT —\n written only by `PATCH /v1/imports/{_id}` on `/debugUntil`, at most 3600 seconds\n ahead (`422` `update_not_allowed` beyond that); `remove` stops capture.\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":2815,"content_sha256":"c8f0ad05a78544f6d0b4202571eab143266768133e474f8a2c41fa8df7494f88"},{"filename":"references/schemas/s3.yml","content":"S3:\n type: object\n description: |-\n Defines where files are written to an Amazon S3 bucket. Required when the _connectionId\n field references an AWS S3 connection; must not be included for other connection types.\n region and bucket locate the destination, fileKey sets the object key, and backupBucket\n retains a copy after a successful import.\n x-celigo-ai-guidance:\n - |-\n ## KEY FIELDS\n - region: REQUIRED — the AWS region of the target bucket (e.g. us-east-1)\n - bucket: REQUIRED — the destination S3 bucket name\n - fileKey: the S3 object key (path + filename) for the written file\n - backupBucket: optional — bucket where a copy is retained after a successful import\n required:\n - region\n - bucket\n properties:\n region:\n type: string\n default: us-east-1\n x-lowercase: true\n x-celigo-ui-override: >-\n Required by the S3 import form (fieldDefinitions/resources/imports/s3.js: s3.region is\n required:true, default us-east-1). Encoded to mirror the form so builders produce\n connectable configurations.\n description: |-\n AWS region where the S3 bucket resides. Must match the bucket's actual location to avoid\n connectivity errors and misrouted requests.\n x-celigo-ai-guidance:\n - |-\n The AWS region where the S3 bucket is physically located,\n representing the specific geographical area within AWS's global infrastructure\n that hosts the bucket.\n This designation directly affects data access latency, availability, redundancy,\n and compliance with regional data governance, privacy laws,\n and residency requirements.\n Selecting the appropriate region is critical for optimizing performance,\n minimizing costs related to data transfer and storage,\n and ensuring adherence to legal and organizational policies.\n The region must be specified using a valid AWS region identifier that accurately\n corresponds to the bucket's actual location to avoid connectivity issues,\n authentication failures, and improper routing of API requests.\n Proper region selection also influences disaster recovery strategies,\n service availability, and integration with other AWS services,\n making it a foundational parameter in S3 bucket configuration and management.\n - |-\n ## FIELD BEHAVIOR:\n - Defines the physical and logical location of the S3 bucket within AWS's global network.\n - Directly impacts data access speed, latency, and throughput.\n - Determines compliance with regional data sovereignty, privacy, and security regulations.\n - Must be a valid AWS region code that matches the bucket's deployment location.\n - Influences availability zones, fault tolerance, and disaster recovery planning.\n - Affects endpoint URL construction and API request routing.\n - Plays a role in cost optimization by affecting storage pricing and data transfer fees.\n - Governs integration capabilities with other AWS services available in the selected region.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Use official AWS region codes such as \"us-east-1\", \"eu-west-2\", or \"ap-southeast-2\" as defined by AWS.\n - Validate the region value against the current list of supported AWS regions to prevent misconfiguration.\n - Ensure the specified region exactly matches the bucket's physical location to avoid access and authentication errors.\n - Consider cost implications including storage pricing, data transfer fees, and cross-region replication expenses.\n - When migrating or replicating buckets across regions, update this property accordingly and plan for data transfer and downtime.\n - Verify service availability and feature support in the chosen region before deployment.\n - Regularly review AWS region updates and deprecations to maintain compatibility.\n - Use region-specific endpoints when constructing API requests to ensure proper routing.\n - |-\n ## EXAMPLES:\n - us-east-1 (Northern Virginia, USA)\n - eu-central-1 (Frankfurt, Germany)\n - ap-southeast-2 (Sydney, Australia)\n - sa-east-1 (São Paulo, Brazil)\n - ca-central-1 (Canada Central)\n examples: [\"us-east-1\", \"eu-west-1\", \"ap-southeast-2\", \"eu-central-1\", \"sa-east-1\", \"ca-central-1\"]\n bucket:\n type: string\n x-celigo-ui-override: >-\n Required by the S3 import form (fieldDefinitions/resources/imports/s3.js: s3.bucket is\n required:true). Encoded to mirror the form so builders produce connectable configurations.\n description: |-\n S3 bucket that receives the imported files. The bucket must already exist and the\n connection's AWS credentials must have s3:PutObject permission on it.\n x-celigo-ai-guidance:\n - |-\n The name of the Amazon S3 bucket where data or objects are stored or will be stored.\n This bucket serves as the primary container within the AWS S3 service,\n uniquely identifying the storage location for your data across all AWS accounts globally.\n The bucket name must strictly comply with AWS naming conventions,\n including being DNS-compliant, entirely lowercase, and free of underscores,\n uppercase letters, or IP address-like formats,\n ensuring global uniqueness and compatibility.\n It is essential that the specified bucket already exists and that the user or\n service has the necessary permissions to access,\n upload, or modify its contents.\n Selecting the appropriate bucket, particularly with regard to its AWS region,\n can significantly impact application performance, cost efficiency,\n and adherence to data residency and regulatory requirements.\n - |-\n ## FIELD BEHAVIOR:\n - Specifies the target S3 bucket for all data storage and retrieval operations.\n - Must reference an existing bucket with proper access permissions.\n - Acts as a globally unique identifier within the AWS S3 ecosystem.\n - Immutable after initial assignment; changing the bucket requires specifying a new bucket name.\n - Influences data locality, latency, cost, and compliance based on the bucket's region.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Validate bucket names against AWS S3 naming rules: lowercase letters, numbers, and hyphens only; no underscores, uppercase letters, or IP address formats.\n - Confirm the bucket's existence and verify access permissions before performing operations.\n - Consider the bucket's AWS region to optimize latency, cost, and compliance with data governance policies.\n - Implement comprehensive error handling for scenarios such as bucket not found, access denied, or insufficient permissions.\n - Align bucket selection with organizational policies and regulatory requirements concerning data storage and residency.\n - |-\n ## EXAMPLES:\n - \"my-app-data-bucket\"\n - \"user-uploads-2024\"\n - \"company-backups-eu-west-1\"\n - |-\n ## IMPORTANT NOTES:\n - Bucket names are globally unique across all AWS accounts and cannot be reused or renamed once created.\n - Access control is managed through AWS IAM roles, policies, and bucket policies, which must be correctly configured.\n - The bucket's region selection is critical for optimizing performance, minimizing costs, and ensuring legal compliance.\n - Bucket naming restrictions prohibit uppercase letters, underscores, and IP address-like formats to maintain DNS compliance.\n - |-\n ## DEPENDENCY CHAIN:\n - Depends on the availability and stability of the AWS S3 service.\n - Requires valid AWS credentials with appropriate permissions to access or modify the bucket.\n examples: [\"my-company-imports\", \"customer-data-integration\", \"celigo-etl-files\", \"my-app-data-bucket\", \"user-uploads-2024\", \"company-backups-eu-west-1\"]\n fileKey:\n type: string\n x-celigo-agent:\n - handlebars\n description: |-\n Object key under which each file is stored in the bucket. Include slashes to organize\n files in a folder-style hierarchy (for example, imports/2023/12/orders.json); keys are\n case-sensitive. Supports handlebars placeholders such as orders-{{timestamp}}.json to\n generate a unique key per run.\n x-celigo-ai-guidance:\n - |-\n The unique identifier or path string that specifies the exact location of a file\n stored within an Amazon S3 bucket.\n This key acts as the primary reference for locating, accessing, managing,\n and manipulating a specific file within the storage system.\n It can be a simple filename or a hierarchical path that mimics folder structures\n to logically organize files within the bucket.\n The fileKey is case-sensitive and must be unique within the bucket to prevent\n conflicts or overwriting.\n It supports UTF-8 encoded characters and can include delimiters such as slashes\n (\"/\") to simulate directories,\n enabling structured file organization.\n Proper encoding and adherence to AWS naming conventions are essential to ensure\n seamless integration with S3 APIs and web requests.\n Additionally, the fileKey plays a critical role in access control,\n lifecycle policies, and event notifications within the S3 environment.\n - |-\n ## FIELD BEHAVIOR:\n - Uniquely identifies a file within the S3 bucket namespace.\n - Serves as the primary reference in all file-related operations such as retrieval, update, deletion, and metadata management.\n - Case-sensitive and must be unique to avoid overwriting or conflicts.\n - Supports folder-like delimiters (e.g., slashes) to simulate directory structures for logical organization.\n - Used as a key parameter in all relevant S3 API operations, including GetObject, PutObject, DeleteObject, and CopyObject.\n - Influences access permissions and lifecycle policies when used with prefixes or specific key patterns.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Use a UTF-8 encoded string that accurately reflects the file's intended path or name.\n - Avoid unsupported or problematic characters; strictly follow S3 key naming conventions to prevent errors.\n - Incorporate logical folder structures using delimiters for better organization and maintainability.\n - Ensure URL encoding when the key is included in web requests or URLs to handle special characters properly.\n - Validate that the key length does not exceed S3's maximum allowed size (up to 1024 bytes).\n - Consider the impact of key naming on access control policies, lifecycle rules, and event triggers.\n - |-\n ## EXAMPLES:\n - \"documents/report2024.pdf\"\n - \"images/profile/user123.png\"\n - \"backups/2023/12/backup.zip\"\n - \"videos/2024/events/conference.mp4\"\n - \"logs/2024/06/15/server.log\"\n - |-\n ## IMPORTANT NOTES:\n - The fileKey is case-sensitive; for example, \"File.txt\" and \"file.txt\" are distinct keys.\n examples: [\"orders-{{timestamp}}.json\", \"imports/2023/12/data.csv\", \"upload.xml\", \"documents/report2024.pdf\", \"images/profile/user123.png\", \"backups/2023/12/backup.zip\", \"videos/2024/events/conference.mp4\", \"logs/2024/06/15/server.log\"]\n backupBucket:\n type: string\n description: |-\n S3 bucket where a copy of each imported file is retained after a successful import; if\n omitted, no backup copy is kept. Must reference an existing bucket the connection's\n credentials have s3:PutObject permission on.\n x-celigo-ai-guidance:\n - |-\n The name of the Amazon S3 bucket designated for securely storing backup data.\n This bucket serves as the primary repository for saving copies of critical information,\n ensuring data durability, integrity,\n and availability for recovery in the event of data loss, corruption,\n or disaster.\n It must be a valid, existing bucket within the AWS environment,\n configured with appropriate permissions and security settings to facilitate\n reliable and efficient backup operations.\n Proper configuration of this bucket—including enabling versioning to preserve\n historical backup states,\n applying encryption to protect data at rest,\n and setting lifecycle policies to manage storage costs and data retention—is\n essential to optimize backup management,\n maintain compliance with organizational and regulatory requirements,\n and control expenses.\n The bucket should ideally reside in the same AWS region as the backup source to\n minimize latency and data transfer costs,\n and may incorporate cross-region replication for enhanced disaster recovery capabilities.\n Additionally, the bucket should be monitored regularly for access patterns,\n storage usage,\n and security compliance to ensure ongoing protection and operational efficiency.\n - |-\n ## FIELD BEHAVIOR:\n - Specifies the exact S3 bucket where backup data will be written and stored.\n - Must reference a valid, existing bucket within the AWS account and region.\n - Used exclusively during backup processes to save data snapshots or incremental backups.\n - Requires appropriate write permissions to allow backup services to upload data.\n - Typically remains static unless backup storage strategies or requirements change.\n - Supports integration with backup scheduling and monitoring systems.\n - Facilitates data recovery by maintaining backup copies in a secure, durable location.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Verify that the bucket name adheres to AWS S3 naming conventions and is DNS-compliant.\n - Confirm the bucket exists and is accessible by the backup service or user.\n - Set up IAM roles and bucket policies to grant necessary write and read permissions securely.\n - Enable bucket versioning to maintain historical backup versions and support data recovery.\n - Implement lifecycle policies to manage storage costs by archiving or deleting old backups.\n - Apply encryption (e.g., AWS KMS) to protect backup data at rest and meet compliance requirements.\n - Consider enabling cross-region replication for disaster recovery scenarios.\n - Regularly audit bucket permissions and access logs to ensure security compliance.\n - Monitor storage usage and costs to optimize resource allocation and prevent unexpected charges.\n - Align bucket configuration with organizational policies for data retention, security, and compliance.\n - |-\n ## EXAMPLES:\n - \"my-app-backups\"\n - \"company-data-backup-2024\"\n - \"prod-environment-backup-bucket\"\n examples: [\"my-company-archive\", \"integration-backup\", \"my-app-backups\", \"company-data-backup-2024\", \"prod-environment-backup-bucket\"]\n serverSideEncryptionType:\n type: string\n description: |-\n Server-side encryption applied to uploaded objects. The connection form sets `AES256`\n (SSE-S3, S3-managed keys) when encryption is enabled; leave unset to use the bucket's\n default encryption.\n x-celigo-ai-guidance:\n - |-\n serverSideEncryptionType specifies the type of server-side encryption applied to\n objects stored in the S3 bucket,\n determining how data at rest is securely protected by the storage service.\n This property defines the encryption algorithm or method used by the server to\n automatically encrypt data upon storage,\n ensuring confidentiality, integrity, and compliance with security standards.\n It accepts predefined values that correspond to supported encryption mechanisms,\n influencing how data is encrypted, accessed,\n and managed during storage and retrieval operations.\n By configuring this property,\n users can enforce encryption policies that safeguard sensitive information\n without requiring client-side encryption management,\n thereby simplifying security administration and enhancing data protection.\n - |-\n ## FIELD BEHAVIOR:\n - Defines the encryption algorithm or method used by the server to encrypt data at rest.\n - Controls whether server-side encryption is enabled or disabled for stored objects.\n - Accepts only the value AES256 (SSE-S3, S3-managed keys); leave unset to use the bucket's default encryption.\n - Influences data handling during storage and retrieval to maintain data confidentiality and integrity.\n - Applies encryption automatically without requiring client-side encryption management.\n - Determines compliance with organizational and regulatory encryption requirements.\n - Affects how encryption keys are managed, accessed, and audited.\n - Impacts access control and permissions related to encrypted objects.\n - May affect performance and cost depending on the encryption method chosen.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Validate the input value against the supported encryption type: only \"AES256\".\n - Ensure the selected encryption type is compatible with the S3 bucket's configuration, policies, and permissions.\n - Gracefully handle scenarios where encryption is not specified, disabled, or set to an empty value.\n - Provide clear and actionable error messages if an unsupported or invalid encryption type is supplied.\n - Consider the impact on performance and cost when enabling encryption.\n - Document encryption settings clearly for users to understand the security posture of stored data.\n - Implement fallback or default behaviors when encryption settings are omitted.\n - Ensure that encryption settings align with audit and compliance reporting requirements.\n - Test encryption settings thoroughly to confirm data is encrypted and accessible as expected.\n - |-\n ## EXAMPLES:\n - \"AES256\" — Server-side encryption using Amazon S3-managed encryption keys (SSE-S3).\n examples: [\"AES256\"]\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":18917,"content_sha256":"6c389f07bc88d6f85f3d16757f6ae8e86bed4e17e062a85c7c6fbb13ae3c10b1"},{"filename":"references/schemas/salesforce.yml","content":"Salesforce:\n type: object\n description: |-\n Salesforce-specific configuration for the import: the operation to perform, the API to use, and object-level settings such as `sObjectType`, `idLookup`, and `upsert`. Properties like `ignoreExisting`, `ignoreMissing`, `name`, and `description` are not part of this object — they belong at the import level.\n x-celigo-ai-guidance:\n - |-\n Configuration for Salesforce imports containing operation type, API selection,\n and object-specific settings.\n - |-\n When asked to generate this configuration, return an object with these properties:\n - `operation` (REQUIRED): The Salesforce operation type (insert, update, upsert, etc.)\n - `api` (REQUIRED): The Salesforce API to use (default: \"soap\")\n - `sObjectType`: The Salesforce object type (e.g., \"Account\", \"Contact\", \"Vendor__c\")\n - `idLookup`: Configuration for looking up existing records\n - `upsert`: Configuration for upsert operations (when operation is \"upsert\")\n - And other operation-specific properties as needed\n # Requireds are conditional (see the if/then chain at the end of this schema):\n # blob configs (attachment / contentVersion) carry operation but no api and skip the\n # operation→idLookup/upsert rules; server bookkeeping also writes stub `salesforce`\n # objects ({lookups:[]} / {}) onto imports of every adaptorType.\n properties:\n lookups:\n description: |\n Lookup definitions used to resolve Salesforce record references or\n static value mappings. Each lookup has a unique ``name`` that mapping\n fields reference via ``lookupName`` and that Handlebars expressions\n reference via ``{{lookup \"name\" value}}``.\n\n **Mirror invariant**\n The Celigo platform stores lookups in TWO locations on a Salesforce\n import resource: top-level ``lookups`` and ``salesforce.lookups``.\n Both arrays must hold byte-for-byte identical content, so write\n both on every save.\n\n **Static vs dynamic lookups**\n - **Static** — only ``map`` + optional ``default``. No Salesforce\n query runs. Use for small controlled vocabularies (country codes,\n status labels, etc.).\n - **Dynamic** — ``sObjectType`` + ``resultField`` + ``whereClause``.\n Issues a SOQL query at runtime. Use ``map`` alongside the query to\n short-circuit known values before hitting Salesforce.\n items:\n type: object\n properties:\n name:\n type: string\n description: |\n Unique identifier for this lookup entry. Mapping fields reference\n it via ``lookupName``, and Handlebars expressions reference it\n via ``{{lookup \"name\" value}}``. Must be unique within this\n resource's ``lookups`` array. Conventionally a short slug or a\n generated hash — the value is only meaningful as a reference\n token.\n sObjectType:\n type: string\n description: |\n Salesforce sObject API name to query (e.g. ``Account``,\n ``Contact``, ``Solution__c``). For custom objects include the\n ``__c`` suffix. Case-sensitive — must match exactly as declared\n in the target Salesforce org.\n resultField:\n type: string\n description: |\n sObject field whose value is returned when the SOQL ``whereClause``\n matches a record. Typically ``Id`` when the lookup resolves to a\n Salesforce record reference, but any queryable field is valid.\n whereClause:\n type: string\n x-celigo-agent:\n - handlebars\n description: |\n SOQL ``WHERE`` clause that identifies the matching Salesforce\n record. Use Handlebars expressions (triple braces recommended:\n ``{{{field}}}``) to inject values from the incoming source record.\n Wrap the whole expression in parentheses.\n\n Examples:\n - ``(Id = {{{id [Customer SFDC ID]}}})``\n - ``(Email = '{{{email}}}')``\n - ``(External_Id__c = {{{string externalId}}} AND IsActive = true)``\n\n Celigo's Handlebars helpers (``string``, ``id``, ``double``,\n ``bool``, ``date``) produce correctly-typed SOQL literals —\n prefer them over manual quoting. String values wrap in single\n quotes; numeric/id values do not.\n map:\n type: [object, 'null']\n description: |\n Optional static key→value map evaluated BEFORE the SOQL query.\n When the incoming field value matches a key, the mapped value\n is returned without hitting Salesforce. Useful for small\n controlled enumerations where a full query would be overkill.\n default:\n type: [string, 'null']\n description: |\n Value returned when the SOQL query matches no records. When\n omitted and ``allowFailures`` is false, an unmatched lookup\n fails the record.\n allowFailures:\n type: boolean\n description: |\n When true, a lookup miss (no matching record and no ``default``)\n is tolerated — the mapped field resolves to ``null`` and the\n import continues. When false (default), a miss halts processing\n for the record.\n useDefaultOnMultipleMatches:\n type: boolean\n description: |\n Controls behaviour when the SOQL query matches more than one\n record. When true, the ``default`` value is returned instead of\n raising an ambiguous-match error. When false (default), multiple\n matches are treated as a failure.\n type: array\n x-celigo-agent:\n - lookup\n operation:\n type: string\n enum:\n - insert\n - update\n - upsert\n - upsertpicklistvalues\n - delete\n - addupdate\n x-lowercase: true\n x-enumDescriptions:\n insert: Create new records in Salesforce.\n update: Modify existing records in Salesforce, identified by record ID.\n upsert: Create or update records based on an External ID field match.\n upsertpicklistvalues: Create or update picklist values on a Salesforce field.\n delete: Remove records from Salesforce, moving them to the recycle bin.\n addupdate: Create new or update existing records using Celigo-specific matching logic.\n x-celigo-ui-override: >-\n Required by the Salesforce import form (operation is required:true). Encoded to mirror the\n form so builders produce connectable configurations.\n description: |-\n Controls how records are written to Salesforce. Each value pairs with a lookup strategy: `upsert` requires `upsert.externalIdField` plus `idLookup.extract`, while `update`, `delete`, `addupdate`, and `insert` with `ignoreExisting` require `idLookup.whereClause`. Values are converted to lowercase automatically.\n x-celigo-ai-guidance:\n - |-\n Specifies the type of Salesforce operation to perform on records.\n This determines how data is created, modified, or removed in Salesforce.\n - |-\n ## FIELD BEHAVIOR:\n - Defines the specific Salesforce action to execute (insert, update, upsert, etc.)\n - Dictates required fields and validation rules for the operation\n - Determines API endpoint and request structure\n - Automatically converted to lowercase before processing\n - Influences response format and error handling\n - |-\n ## OPERATION TYPES:\n\n ### **insert**\n - **Purpose:** Create new records in Salesforce\n - **Behavior:** Creates new records\n - **Use when:** Creating brand new records\n - **With ignoreExisting: true:** Checks for existing records and skips them (requires idLookup.whereClause)\n - **Without ignoreExisting:** Creates all records without checking for duplicates\n - **Example:** \"Insert new leads from marketing campaign\"\n - **Example with ignoreExisting:** \"Create vendors while ignoring existing vendors\" → operation: \"insert\" + ignoreExisting: true\n\n ### **update**\n - **Purpose:** Modify existing records in Salesforce\n - **Behavior:** Requires record ID; fails if record doesn't exist\n - **Use when:** Updating known existing records\n - **Example:** \"Update customer status based on order completion\"\n\n ### **upsert**\n - **Purpose:** Create or update records based on external ID\n - **Behavior:** Updates if external ID matches, creates if not found\n - **Use when:** Syncing data from external systems with external ID field\n - **Example:** \"Upsert customers by External_ID__c field\"\n - **REQUIRES:**\n - `idLookup.extract` field (maps incoming field to External ID)\n - `upsert.externalIdField` field (specifies Salesforce External ID field)\n - Both fields MUST be set for upsert to work\n\n ### **upsertpicklistvalues**\n - **Purpose:** Create or update picklist values dynamically\n - **Behavior:** Manages picklist value creation/updates\n - **Use when:** Maintaining picklist values from external data\n - **Example:** \"Sync product categories to Salesforce picklist\"\n\n ### **delete**\n - **Purpose:** Remove records from Salesforce\n - **Behavior:** Requires record ID; records moved to recycle bin\n - **Use when:** Removing obsolete or invalid records\n - **Example:** \"Delete expired opportunities\"\n\n ### **addupdate**\n - **Purpose:** Create new or update existing records (Celigo-specific)\n - **Behavior:** Similar to upsert but uses different matching logic\n - **Use when:** Upserting without requiring external ID field\n - **Example:** \"Sync contacts, create new or update existing by email\"\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Choose operation based on whether records are new, existing, or unknown\n - For **insert**: Ensure records are truly new to avoid errors\n - For **insert with ignoreExisting: true**: MUST set `idLookup.whereClause` to check for existing records\n - For **update/delete**: MUST set `idLookup.whereClause` to identify records\n - For **upsert**: MUST also include the `upsert` object with `externalIdField` property\n - For **upsert**: MUST also set `idLookup.extract` to map incoming field\n - For **addupdate**: MUST set `idLookup.whereClause` to define matching criteria\n - Use **upsertpicklistvalues** only for picklist management scenarios\n - |-\n ## COMMON PATTERNS:\n - **\"Create X while ignoring existing X\"** → operation: \"insert\" + api: \"soap\" + idLookup.whereClause\n - **\"Create or update X\"** → operation: \"upsert\" + api: \"soap\" + upsert.externalIdField + idLookup.extract\n - **\"Update X\"** → operation: \"update\" + api: \"soap\" + idLookup.whereClause\n - **\"Sync X\"** → operation: \"addupdate\" + api: \"soap\" + idLookup.whereClause\n - **ALWAYS include** api field (default to \"soap\" if not specified in prompt)\n - |-\n ## EXAMPLES:\n\n ### Insert new records:\n ```json\n {\n \"operation\": \"insert\",\n \"api\": \"soap\",\n \"sObjectType\": \"Lead\"\n }\n ```\n Prompt: \"Create new leads in Salesforce\"\n\n ### Insert with ignore existing:\n ```json\n {\n \"operation\": \"insert\",\n \"api\": \"soap\",\n \"sObjectType\": \"Vendor__c\",\n \"idLookup\": {\n \"whereClause\": \"Name = '{{{VendorName}}}'\"\n }\n }\n ```\n Prompt: \"Create vendor records while ignoring existing vendors\"\n\n **NOTE:** For \"ignore existing\" prompts, also remember that `ignoreExisting: true` belongs at a different level (not part of this salesforce configuration).\n\n ### Update existing records:\n ```json\n {\n \"operation\": \"update\",\n \"api\": \"soap\",\n \"sObjectType\": \"Account\",\n \"idLookup\": {\n \"whereClause\": \"Id = '{{{AccountId}}}'\"\n }\n }\n ```\n Prompt: \"Update existing accounts in Salesforce\"\n\n ### Upsert by external ID:\n ```json\n {\n \"operation\": \"upsert\",\n \"api\": \"soap\",\n \"sObjectType\": \"Contact\",\n \"idLookup\": {\n \"extract\": \"ExternalId\"\n },\n \"upsert\": {\n \"externalIdField\": \"External_ID__c\"\n }\n }\n ```\n Prompt: \"Create or update contacts by External_ID__c\"\n\n ### Addupdate (create or update):\n ```json\n {\n \"operation\": \"addupdate\",\n \"api\": \"soap\",\n \"sObjectType\": \"Customer__c\",\n \"idLookup\": {\n \"whereClause\": \"Name = '{{{CustomerName}}}'\"\n }\n }\n ```\n Prompt: \"Sync customers, create new or update existing\"\n - |-\n ## IMPORTANT NOTES:\n - Operation value is case-insensitive (automatically converted to lowercase)\n - Invalid operation values will cause API validation errors\n - Each operation has different requirements for record identification\n - **CRITICAL for upsert**: MUST include the `upsert` object with `externalIdField` property\n - **CRITICAL for upsert**: MUST set `idLookup.extract` to map incoming field to External ID\n - **upsert** requires external ID field to be configured in Salesforce\n - **addupdate** is Celigo-specific and may not work with all Salesforce APIs\n - Operations must align with user's Salesforce permissions\n - Some operations support bulk processing more efficiently than others\n api:\n type: string\n enum:\n - soap\n - rest\n - metadata\n - compositerecord\n x-lowercase: true\n x-enumDescriptions:\n soap: Salesforce SOAP API for transactional or legacy imports; supports AllOrNone transaction control and XML-based requests, with more overhead than REST\n rest: Standard Salesforce REST API for real-time single-record and small-batch imports; supports CRUD, flexible querying, rich error messages, and up to 2,000 records per API call\n metadata: Salesforce Metadata API for deploying configuration and setup data between orgs, such as custom objects, fields, and workflows; not for data records\n compositerecord: Salesforce Composite Graph API for creating hierarchical related record trees, such as parent-child relationships, atomically in one request\n x-celigo-ui-override: >-\n Required by the Salesforce import form (api is required:true, default soap). Encoded to\n mirror the form so builders produce connectable configurations.\n description: |-\n Selects the Salesforce API used to execute the import. Use `soap` for most imports and another API only when its specific capability is needed; default to `soap` when no API is specified. Values are converted to lowercase automatically.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Required by the import form for record imports; legacy attachment\n imports are stored without it and the platform falls back to soap\n - Determines the protocol and endpoint used for Salesforce communication\n - Influences request format, batch sizes, and response handling\n - Automatically converted to lowercase before processing\n - Affects available features and operation types\n - Impacts performance and error handling behavior\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - **MUST ALWAYS SET THIS FIELD** - Never leave it undefined\n - If prompt doesn't specify API type, use **\"soap\"**\n - Use **\"soap\"** only when AllOrNone transaction control is explicitly required\n - Use **\"metadata\"** only for configuration deployment, never for data\n - Use **\"compositerecord\"** for complex parent-child relationship imports\n - Consider API limits and performance characteristics for your data volume\n - Ensure the Salesforce connection supports the chosen API type\n - |-\n ## EXAMPLES:\n\n ### REST API:\n ```json\n {\n \"operation\": \"insert\",\n \"api\": \"rest\",\n \"sObjectType\": \"Lead\"\n }\n ```\n Prompt: \"Import leads into Salesforce\"\n\n ### SOAP API with transaction control:\n ```json\n {\n \"operation\": \"insert\",\n \"api\": \"soap\",\n \"sObjectType\": \"Invoice__c\",\n \"soap\": {\n \"headers\": {\n \"allOrNone\": true\n }\n }\n }\n ```\n Prompt: \"Import invoices with all-or-none guarantee\"\n\n ### Composite Record API:\n ```json\n {\n \"operation\": \"insert\",\n \"api\": \"compositerecord\",\n \"sObjectType\": \"Account\"\n }\n ```\n Prompt: \"Import accounts with related contacts and opportunities\"\n - |-\n ## IMPORTANT NOTES:\n - API value is case-insensitive (automatically converted to lowercase)\n - SOAP API has slightly more overhead but provides transaction guarantees\n - Metadata API is for configuration only, not data records\n - Composite Record API requires careful relationship mapping\n - Each API has different rate limits and performance characteristics\n - Ensure your Salesforce connection and permissions support the chosen API\n - |-\n **REQUIRED FIELD:** Specifies which Salesforce API to use for the import operation.\n Different APIs have different capabilities, performance characteristics,\n and use cases.\n - |-\n **CRITICAL:** This field MUST ALWAYS be set.\n If no specific API is mentioned in the prompt, **DEFAULT to \"soap\"**.\n - |-\n ## DEFAULT CHOICE:\n When in doubt or when no API is mentioned in the prompt → **USE \"soap\"**\n soap:\n type: object\n properties:\n headers:\n type: object\n properties:\n allOrNone:\n type: boolean\n description: |-\n When true, the batch is processed atomically: if any record fails, the entire transaction is rolled back and nothing is committed. When false or omitted, records that succeed are committed even if others in the batch fail.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - When true, enforces atomicity: all records in the batch must succeed or the entire operation is rolled back with no changes applied.\n - When false, permits partial success: successfully processed records are committed even if some records fail.\n - Applies mainly to batch DML operations including create, update, delete, and upsert.\n - Helps maintain data consistency by preventing partial updates when strict transactional integrity is required.\n - Influences how errors are reported and handled in the API response.\n - Affects transaction commit behavior and rollback mechanisms within the Salesforce platform.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Set to true to guarantee strict data consistency and avoid partial data changes that could lead to inconsistent states.\n - Set to false to allow higher throughput and partial processing when some failures are acceptable or expected.\n - Default value is false if the property is omitted, allowing partial success by default.\n - When false, implement robust error handling to process partial success responses and handle individual record errors appropriately.\n - Verify that the target Salesforce API operation supports the allOrNone header before use.\n - Consider the impact on transaction duration and system resources when enabling atomic operations.\n - Use in scenarios where data integrity is paramount, such as financial or compliance-related data updates.\n - |-\n ## EXAMPLES:\n - allOrNone = true: Attempting to update 10 records where one record fails validation results in no records being updated, preserving atomicity.\n - allOrNone = false: Attempting to update 10 records where one record fails validation results in 9 records updated successfully and one failure reported.\n - allOrNone = true: Deleting multiple records in a batch will either delete all specified records or none if any deletion fails.\n - |-\n allOrNone specifies whether the Salesforce API operation should be executed atomically,\n ensuring that either all parts of the operation succeed or none do.\n When set to true, if any record in a batch operation fails,\n the entire transaction is rolled back and no changes are committed,\n preserving data integrity across the dataset.\n When set to false, the operation allows partial success,\n meaning some records may be processed and committed even if others fail,\n which can improve performance but requires careful error handling to manage partial failures.\n This property is primarily applicable to data manipulation operations such as create,\n update, delete, and upsert,\n and is critical for maintaining consistent and reliable data states in\n multi-record transactions.\n description: SOAP headers included in requests sent to Salesforce, such as the `allOrNone` transaction-control header.\n batchSize:\n type: number\n description: |-\n Number of records sent to Salesforce in each SOAP API batch. Larger batches reduce the number of API calls; smaller batches reduce per-call load and the chance of timeouts.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Determines the count of records included in each batch request to the Salesforce SOAP API.\n - Directly influences the total number of API calls required when handling large datasets.\n - Affects processing throughput, latency, and memory usage during data operations.\n - Can be dynamically adjusted to optimize performance based on system capabilities and network conditions.\n - Impacts error handling and retry mechanisms by controlling batch granularity.\n - Influences network load and the likelihood of encountering API limits or timeouts.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Choose a batch size that adheres to Salesforce API limits and recommended best practices.\n - Balance between larger batch sizes (which reduce API calls but increase memory and processing load) and smaller batch sizes (which increase API calls but reduce resource consumption).\n - Ensure the batchSize value is a positive integer within the allowed range for the specific Salesforce operation.\n - Continuously monitor API response times, error rates, and resource utilization to fine-tune the batch size for optimal performance.\n - Default to Salesforce-recommended batch sizes when uncertain or when starting configuration.\n - Consider network stability and latency when selecting batch size to minimize timeouts and failures.\n - Validate batch size compatibility with other integration components and middleware.\n - Test batch size settings in a staging environment before deploying to production to avoid unexpected failures.\n - |-\n ## EXAMPLES:\n - `batchSize: 200` — Processes 200 records per batch, a common default for many Salesforce operations.\n - `batchSize: 500` — Processes 500 records per batch, often the maximum allowed for certain Salesforce SOAP API calls.\n - `batchSize: 50` — Processes smaller batches suitable for environments with limited memory or network bandwidth.\n - `batchSize: 100` — A moderate batch size balancing throughput and resource usage in typical scenarios.\n - |-\n The number of records to be processed in each batch during Salesforce SOAP API operations.\n This property controls the size of data chunks sent or retrieved in a single API call,\n optimizing performance and resource utilization by balancing throughput and\n system resource consumption.\n Adjusting the batch size directly impacts the number of API calls made,\n the processing speed, memory usage, and network load,\n enabling fine-tuning of data operations to suit specific environment constraints\n and performance goals.\n Proper configuration of batchSize helps prevent API throttling,\n reduces the risk of timeouts,\n and ensures efficient handling of large datasets by segmenting data into manageable portions.\n description: Settings that apply when `api` is `soap`, including SOAP request headers and batch size.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Defines all connection, authentication, and operational parameters necessary for Salesforce SOAP API interactions.\n - Facilitates the construction and transmission of properly structured SOAP requests and the parsing of SOAP responses.\n - Manages the session lifecycle, including token acquisition, refresh, and timeout handling to maintain persistent connectivity.\n - Supports comprehensive error detection and handling of SOAP faults, API exceptions, and network issues to maintain integration stability.\n - Enables customization of SOAP headers, namespaces, and envelope structures as required by specific Salesforce API calls.\n - Allows configuration of retry policies and backoff strategies in response to transient errors or rate limiting.\n - Supports environment-specific configurations, such as production, sandbox, or custom Salesforce domains.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Configure endpoint URLs and authentication credentials accurately based on the Salesforce environment (production, sandbox, or custom domains).\n - Validate SOAP envelope structure, namespaces, and compliance with the latest Salesforce WSDL specifications to prevent communication errors.\n - Implement robust error handling to capture, log, and respond appropriately to SOAP faults, API limit exceptions, and network failures.\n - Securely store and transmit sensitive information such as session IDs, OAuth tokens, passwords, and certificates, following best security practices.\n - Monitor Salesforce API usage limits and implement throttling or queuing mechanisms to avoid exceeding quotas and service disruptions.\n - Set timeout values thoughtfully to balance between preventing hanging requests and allowing sufficient time for valid operations to complete.\n - Regularly update the SOAP configuration to align with Salesforce API version changes, WSDL updates, and evolving security requirements.\n - Incorporate logging and monitoring to track SOAP request and response cycles for troubleshooting and performance optimization.\n - Ensure compatibility with Salesforce security protocols, including TLS versions and encryption standards.\n - |-\n Specifies the comprehensive configuration settings required for integrating with\n the Salesforce SOAP API.\n This property includes all essential parameters to establish, authenticate,\n and manage SOAP-based communication with Salesforce services,\n enabling a wide range of operations such as creating, updating, deleting,\n querying, and executing actions on Salesforce objects.\n It encompasses detailed configurations for endpoint URLs,\n authentication credentials (such as session IDs or OAuth tokens), SOAP headers,\n session management details, timeout settings,\n and error handling mechanisms to ensure reliable, secure,\n and efficient interactions.\n The configuration must strictly adhere to Salesforce's WSDL definitions and security standards,\n supporting robust SOAP API usage within the application environment.\n sObjectType:\n type: string\n description: |-\n Salesforce object the import writes to, such as `Account`, `Contact`, or a custom object like `Vendor__c`. Must exactly match the object's case-sensitive API name; custom objects include the `__c` suffix.\n x-celigo-ai-guidance:\n - |-\n sObjectType specifies the exact Salesforce object type that the API operation will target,\n such as standard objects like Account, Contact, Opportunity,\n or custom objects like CustomObject__c.\n This property is essential for defining the schema context of the API request,\n determining which fields are relevant,\n and directing the operation to the correct Salesforce data entity.\n It plays a critical role in data validation, processing logic,\n and endpoint routing within the Salesforce environment,\n ensuring that operations such as create, update, delete,\n or query are executed against the intended object type with precision.\n Accurate specification of sObjectType enables dynamic adjustment of request\n payloads and response handling based on the object’s schema and enforces\n compliance with Salesforce’s API naming conventions and permissions model.\n - |-\n ## FIELD BEHAVIOR:\n - Identifies the specific Salesforce object (sObject) type targeted by the API operation.\n - Determines the applicable schema, field availability, and validation rules based on the selected object.\n - Must exactly match a valid Salesforce object API name recognized by the connected Salesforce instance, including case sensitivity.\n - Routes the API request to the appropriate Salesforce object endpoint for the intended operation.\n - Influences dynamic field mapping, data transformation, and response parsing processes.\n - Supports both standard and custom Salesforce objects, with custom objects requiring the “__c” suffix.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Validate sObjectType values against the current Salesforce object metadata in the target environment to prevent invalid requests.\n - Enforce exact case-sensitive matching with Salesforce API object names (e.g., \"Account\" not \"account\").\n - Support and recognize both standard objects and custom objects, ensuring custom objects include the “__c” suffix.\n - Implement robust error handling for invalid, unsupported, or deprecated sObjectType values to provide clear feedback.\n - Regularly update the list of allowed sObjectType values to reflect changes in the Salesforce schema or environment.\n - Use sObjectType to dynamically tailor API request payloads, field selections, and response parsing logic.\n - Consider object-level permissions and authentication scopes when processing requests involving specific sObjectTypes.\n - |-\n ## EXAMPLES:\n - \"Account\"\n - \"Contact\"\n - \"Opportunity\"\n - \"Lead\"\n - \"CustomObject__c\"\n - \"Case\"\n - \"Campaign\"\n - |-\n ## IMPORTANT NOTES:\n - Custom Salesforce objects must be referenced using their exact API names, including the “__c” suffix.\n examples: [\"Account\", \"Contact\", \"Vendor__c\"]\n idLookup:\n type: object\n properties:\n extract:\n type: string\n description: |-\n Field in the incoming data whose value is matched against the Salesforce External ID field named in `upsert.externalIdField`. Set only when `operation` is `upsert`; all other operations use `whereClause` instead. Choose a field that is unique and always populated — missing or null values cause the upsert to fail. If both `extract` and `whereClause` are set, `extract` takes precedence for upsert.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Specifies which field from the incoming data contains the External ID value\n - **Used for upsert operations ONLY**\n - Maps the incoming data field to the Salesforce External ID field specified in `upsert.externalIdField`\n - Must reference a field that exists in the incoming data payload\n - The field value must be unique and always populated to ensure reliable matching\n - Generates field mapping at runtime to match incoming records with Salesforce records\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Choose a field from incoming data that contains unique, non-null values\n - Ensure the field is always populated in the incoming records to prevent operation failures\n - This field's value will be matched against the Salesforce External ID field specified in `upsert.externalIdField`\n - Use the exact field name as it appears in the incoming data (case-sensitive)\n - If sample data is available, validate that the field exists and has appropriate values\n - Test with sample data to ensure matching works correctly before production deployment\n - |-\n ## IMPORTANT NOTES:\n - **ONLY use for upsert operations**\n - For all other operations (update, delete, addupdate, insert with ignoreExisting), use `whereClause` instead\n - Field must exist in incoming data and be consistently populated\n - Works with `upsert.externalIdField` to enable External ID matching\n - Missing or null values will cause the upsert operation to fail\n - Field name is case-sensitive and must match exactly\n - If both `extract` and `whereClause` are set, `extract` takes precedence for upsert\n - |-\n Please specify which field from your export data should map to External ID.\n It is very important to pick a field that is both unique,\n and guaranteed to always have a value when exported (otherwise the Salesforce\n Upsert operation will fail).\n If sample data is available then a select list should be displayed here to help\n you find the right field from your export data.\n If sample data is not available then you will need to manually type the field id\n that should map to External ID.\n - |-\n ## WHEN TO USE:\n - **upsert operations ONLY** — REQUIRED when operation is \"upsert\"\n - Works together with `upsert.externalIdField` to enable External ID matching\n - |-\n ## WHEN NOT TO USE:\n - **update** — use `whereClause` instead\n - **delete** — use `whereClause` instead\n - **addupdate** — use `whereClause` instead\n - **insert with ignoreExisting** — use `whereClause` instead\n - |-\n ## EXAMPLE:\n\n ### Upsert by External ID:\n ```json\n {\n \"operation\": \"upsert\",\n \"idLookup\": {\n \"extract\": \"CustomerNumber\"\n },\n \"upsert\": {\n \"externalIdField\": \"External_ID__c\"\n }\n }\n ```\n Maps incoming field \"CustomerNumber\" → Salesforce field \"External_ID__c\"\n\n **How it works:**\n - Incoming record has `{\"CustomerNumber\": \"CUST-12345\", \"Name\": \"John\"}`\n - System extracts \"CUST-12345\" from CustomerNumber\n - Searches Salesforce External_ID__c field for \"CUST-12345\"\n - If found: updates the existing record\n - If not found: creates a new record with External_ID__c = \"CUST-12345\"\n - |-\n ## COMMON FIELD NAMES:\n - \"ExternalId\" — incoming field containing external identifier\n - \"CustomerId\" — incoming field with unique customer number\n - \"AccountNumber\" — incoming field with account number\n - \"SourceSystemId\" — incoming field from source system\n examples: [\"externalId\"]\n whereClause:\n type: string\n x-celigo-agent:\n - handlebars\n description: |-\n SOQL condition (without the `WHERE` keyword) that finds existing Salesforce records for each incoming record. Required when `operation` is `update`, `delete`, or `addupdate`, and for `insert` with `ignoreExisting`; for `upsert`, use `extract` instead. Reference incoming fields with triple-brace Handlebars and wrap string values in single quotes (e.g. `Email = '{{{Email}}}'`) — triple braces emit the raw value without HTML-escaping, which would corrupt characters like `&` or `'`. Combine conditions with `AND`/`OR`.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Specifies a SOQL conditional expression to match existing records in Salesforce\n - Omit the \"WHERE\" keyword - only provide the condition expression\n - Supports Handlebars {{{fieldName}}} syntax to reference incoming data fields\n - Supports SOQL operators: =, !=, >, \u003c, >=, \u003c=, LIKE, IN, NOT IN\n - Supports logical operators: AND, OR, NOT\n - Can use parentheses for grouping complex conditions\n - The clause is evaluated for each incoming record to find matching Salesforce records\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Write the clause following SOQL syntax rules, omitting the \"WHERE\" keyword\n - Use {{{fieldName}}} syntax to reference values from incoming records\n - Wrap string values containing spaces in single quotes: `Name = '{{{FullName}}}'`\n - Numeric and boolean values don't need quotes: `Age = {{{Age}}}`\n - Use indexed fields (Email, External ID) for better performance\n - Keep the clause selective to minimize records returned\n - Validate syntax before deployment to prevent runtime errors\n - |-\n ## EXAMPLES:\n\n ### Update by Email:\n ```json\n {\n \"operation\": \"update\",\n \"idLookup\": {\n \"whereClause\": \"Email = '{{{Email}}}'\"\n }\n }\n ```\n Finds records with matching Email for update\n\n ### Insert with ignore existing by Email:\n ```json\n {\n \"operation\": \"insert\",\n \"ignoreExisting\": true,\n \"idLookup\": {\n \"whereClause\": \"Email = '{{{Email}}}'\"\n }\n }\n ```\n Checks if a record with matching Email exists; if found, skips creation\n\n ### Addupdate (create or update) by Email:\n ```json\n {\n \"operation\": \"addupdate\",\n \"idLookup\": {\n \"whereClause\": \"Email = '{{{Email}}}'\"\n }\n }\n ```\n Updates if Email matches, creates new record if not found\n\n ### Delete by Account Number:\n ```json\n {\n \"operation\": \"delete\",\n \"idLookup\": {\n \"whereClause\": \"AccountNumber = '{{{AcctNum}}}'\"\n }\n }\n ```\n Finds records with matching AccountNumber for deletion\n\n ### Complex AND logic:\n ```json\n {\n \"operation\": \"update\",\n \"idLookup\": {\n \"whereClause\": \"Name = '{{{ProductName}}}' AND ProductFamily__c = '{{{Family}}}'\"\n }\n }\n ```\n Matches records where both Name AND ProductFamily match\n\n ### Complex OR logic:\n ```json\n {\n \"operation\": \"update\",\n \"idLookup\": {\n \"whereClause\": \"Email = '{{{Email}}}' OR Phone = '{{{Phone}}}'\"\n }\n }\n ```\n Matches records where Email OR Phone matches\n\n ### String with spaces:\n ```json\n {\n \"operation\": \"update\",\n \"idLookup\": {\n \"whereClause\": \"Name = '{{{FullName}}}'\"\n }\n }\n ```\n Note: Single quotes around {{{FullName}}} handle names with spaces like \"John Smith\"\n - |-\n ## IMPORTANT NOTES:\n - **REQUIRED** for update, delete, addupdate, and insert (with ignoreExisting)\n - DO NOT use for upsert operations - use `extract` instead\n - Use {{{fieldName}}} to reference incoming data fields\n - String values with spaces must be wrapped in single quotes\n - Numeric and boolean values don't need quotes\n - Poor clause design can impact performance and hit governor limits\n - Always test in sandbox before production deployment\n - |-\n ## WHEN NOT TO USE:\n - **upsert** — DO NOT use; use `extract` instead\n - **insert without ignoreExisting** — Not needed for simple inserts\n - |-\n To determine if a record exists in Salesforce,\n define a SOQL query WHERE clause for the sObject type selected above.\n For example, if you are importing contact records with a unique email,\n your WHERE clause should identify contacts with the same email.\n - |-\n Handlebars statements are allowed,\n and more complex WHERE clauses can contain AND and OR logic.\n For example, when the product name alone is not unique,\n you can define a WHERE clause to find any records where the product name exists\n AND the product also belongs to a specific product family,\n resulting in a unique product record that matches both criteria.\n - |-\n To find string values that contain spaces, wrap them in single quotes,\n for example:\n - Name = 'John Smith'\n - |-\n ## WHEN TO USE (REQUIRED FOR):\n - **update** — REQUIRED to identify which records to update\n - **delete** — REQUIRED to identify which records to delete\n - **addupdate** — REQUIRED to identify existing records (update if found, insert if not)\n - **insert with ignoreExisting: true** — REQUIRED to check if records exist (skip if found)\n examples: [\"Email = '{{{Email}}}'\", \"External_Id__c = '{{{externalId}}}'\"]\n description: |-\n Controls how existing Salesforce records are matched before writing. Set exactly one of `extract` (for `upsert`) or `whereClause` (for `update`, `delete`, `addupdate`, and `insert` with `ignoreExisting`); omit for plain inserts.\n\n Required when operation is addupdate, delete, update, or upsert.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Contains two strategies for finding existing records: `extract` and `whereClause`\n - **`extract`**: Used ONLY for upsert operations\n - **`whereClause`**: Used for update, delete, addupdate, and insert (with ignoreExisting)\n - Generates field mapping or SOQL queries at runtime to match records\n - Critical for preventing duplicate record creation and enabling record updates\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Use `extract` ONLY for upsert operations (with `upsert.externalIdField`)\n - Use `whereClause` for update, delete, addupdate, and insert with ignoreExisting\n - Do not set both `extract` and `whereClause` for the same operation\n - **ONLY set extract or whereClause - do not nest operation or other properties**\n - Ensure the lookup strategy matches the operation type\n - Test thoroughly in sandbox before production deployment\n - |-\n ## IMPORTANT NOTES:\n - **This object ONLY contains `extract` or `whereClause`**\n - **`extract`** is for upsert ONLY - maps to External ID field\n - **`whereClause`** is for all other operations requiring lookups\n - Do NOT nest `operation`, `upsert`, or other properties here - those belong at the parent salesforce level\n - For upsert: `extract` + `upsert.externalIdField` work together (both at salesforce level)\n - For other operations: `whereClause` enables SOQL-based record matching\n - Incorrect configuration can cause duplicates or failed operations\n - |-\n Configuration for looking up existing Salesforce records. **This object should\n ONLY contain either `extract` OR `whereClause` - NOT both,\n and NOT operation or other properties.**\n - |-\n ## WHAT THIS OBJECT CONTAINS:\n This object should contain ONLY ONE of these properties:\n - `extract` — for upsert operations ONLY\n - `whereClause` — for update, delete, addupdate, and insert (with ignoreExisting)\n\n **DO NOT include operation, upsert, or any other properties here** - those belong at the parent salesforce level.\n - |-\n ## WHEN TO USE EACH FIELD:\n\n ### Use `extract` (upsert ONLY):\n - **upsert** — REQUIRED: Maps incoming field to External ID field\n\n ### Use `whereClause` (all other operations):\n - **update** — REQUIRED: Identifies records to update via SOQL query\n - **delete** — REQUIRED: Identifies records to delete via SOQL query\n - **addupdate** — REQUIRED: Identifies existing records via SOQL query\n - **insert with ignoreExisting: true** — REQUIRED: Checks if records exist via SOQL query\n\n ### Leave empty:\n - **insert without ignoreExisting** — No lookup needed for simple inserts\n - |-\n ## WHAT TO SET (idLookup object content):\n\n ### For upsert:\n ```json\n {\n \"extract\": \"CustomerNumber\"\n }\n ```\n\n ### For update/delete/addupdate:\n ```json\n {\n \"whereClause\": \"Email = '{{{Email}}}'\"\n }\n ```\n - |-\n ## PARENT CONTEXT EXAMPLES (for reference):\n These show the complete parent-level structure. **idLookup itself should only contain extract or whereClause.**\n\n ### Upsert (parent context):\n ```json\n {\n \"operation\": \"upsert\",\n \"idLookup\": {\n \"extract\": \"CustomerNumber\"\n },\n \"upsert\": {\n \"externalIdField\": \"External_ID__c\"\n }\n }\n ```\n\n ### Update (parent context):\n ```json\n {\n \"operation\": \"update\",\n \"idLookup\": {\n \"whereClause\": \"Email = '{{{Email}}}'\"\n }\n }\n ```\n upsert:\n type: object\n properties:\n externalIdField:\n type: string\n description: |-\n API name of the Salesforce field, marked as External ID, that upsert operations match against. This is the target side of the match; `idLookup.extract` names the source field in the incoming data. If the value matches multiple Salesforce records, the upsert fails.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Identifies which Salesforce field serves as the external ID for record matching\n - Must be a field marked as \"External ID\" in Salesforce object configuration\n - Works together with `idLookup.extract` to enable upsert matching\n - The field must be indexed and unique in Salesforce\n - Supports text, number, and email field types configured as external IDs\n - Required when `operation: \"upsert\"`\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Use the exact Salesforce API name of the field (case-sensitive)\n - Verify the field is marked as \"External ID\" in Salesforce object settings\n - Ensure the field is indexed for optimal performance\n - Confirm the integration user has read/write access to this field\n - The field must have unique values to prevent ambiguous matches\n - Test in sandbox before production to verify configuration\n - |-\n ## EXAMPLES:\n\n ### Standard External ID field:\n ```\n \"AccountNumber\"\n ```\n Standard Account field marked as External ID\n\n ### Custom External ID field:\n ```\n \"Custom_External_Id__c\"\n ```\n Custom field marked as External ID\n\n ### Email as External ID:\n ```\n \"Email\"\n ```\n Email field configured as External ID on Contact\n\n ### NetSuite ID as External ID:\n ```\n \"netsuite_conn__NetSuite_Id__c\"\n ```\n NetSuite connector External ID field\n - |-\n ## IMPORTANT NOTES:\n - Field must be explicitly marked as \"External ID\" in Salesforce\n - Not all fields can be External IDs (must be text, number, or email type)\n - Missing or null values in this field will cause upsert to fail\n - If multiple records match the External ID, upsert will fail\n - Always use with `idLookup.extract` for upsert operations\n - |-\n ## HOW IT WORKS WITH idLookup.extract:\n - **`idLookup.extract`**: Specifies which incoming data field contains the matching value (e.g., \"CustomerNumber\")\n - **`externalIdField`**: Specifies which Salesforce field to match against (e.g., \"External_ID__c\")\n - Together they create the match: incoming \"CustomerNumber\" → Salesforce \"External_ID__c\"\n - |-\n ## HOW IT WORKS (Parent Context):\n At the parent `salesforce` level, the complete configuration looks like:\n ```json\n {\n \"operation\": \"upsert\",\n \"idLookup\": {\n \"extract\": \"CustomerNumber\"\n },\n \"upsert\": {\n \"externalIdField\": \"External_ID__c\"\n }\n }\n ```\n Where:\n - `operation: \"upsert\"` — at salesforce level\n - `idLookup.extract: \"CustomerNumber\"` — incoming field, at salesforce level\n - `upsert.externalIdField: \"External_ID__c\"` — **THIS field**, inside upsert object\n - |-\n ## COMMON EXTERNAL ID FIELDS:\n - \"AccountNumber\" — standard Account field\n - \"Email\" — when configured as External ID on Contact\n - \"Custom_External_Id__c\" — custom field on any object\n - \"External_ID__c\" — common naming convention\n - \"Source_System_Id__c\" — for multi-system integrations\n examples: [\"External_Id__c\"]\n description: |-\n Upsert matching configuration, required when `operation` is `upsert` and omitted for all other operations. Contains only `externalIdField`, which pairs with `idLookup.extract` at the parent level.\n\n Required when operation is upsert.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - **REQUIRED** when `operation: \"upsert\"`\n - Contains configuration specific to upsert behavior\n - **ONLY contains ONE property: `externalIdField`**\n - Works in conjunction with `idLookup.extract` (which is at the SAME level as this upsert object, NOT nested within it)\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Always set `externalIdField` when using upsert\n - Ensure the specified External ID field exists in Salesforce\n - Verify the field is marked as \"External ID\" in Salesforce\n - **ONLY set the externalIdField property - nothing else**\n - |-\n ## IMPORTANT NOTES:\n - **This object ONLY contains `externalIdField`**\n - Do NOT nest `operation`, `idLookup`, or other properties here\n - Those properties belong at the parent salesforce level, not nested in this upsert object\n - May be extended with additional upsert-specific options in the future\n - Only relevant when operation is \"upsert\"\n - |-\n ## WHEN TO USE:\n - **REQUIRED** when `operation` is \"upsert\" - MUST be included\n - Leave undefined for insert, update, delete, or addupdate operations\n - Without this object, upsert operations will fail\n - |-\n Configuration for Salesforce upsert operations.\n This object contains ONLY the `externalIdField` property.\n - '**CRITICAL: This component is REQUIRED when `operation: \"upsert\"`. Always include this component for upsert operations.**'\n - |-\n ## WHY IT'S REQUIRED:\n - Upsert operations need to know which Salesforce field to use for matching\n - The `externalIdField` property specifies the Salesforce External ID field\n - This works with `idLookup.extract` to enable record matching\n - Missing this object will cause the import to fail\n - |-\n ## DECISION:\n **Return true/include this component if `operation: \"upsert\"` is set. This is REQUIRED for all upsert operations.**\n - |-\n ## WHAT TO SET:\n **This object should ONLY contain:**\n ```json\n {\n \"externalIdField\": \"External_ID__c\"\n }\n ```\n\n **DO NOT include operation, idLookup, or any other properties here** - those belong at the parent salesforce level.\n - |-\n ## PARENT CONTEXT (for reference only):\n At the parent `salesforce` level, you will also set:\n - `operation: \"upsert\"` (at salesforce level)\n - `idLookup.extract` (at salesforce level)\n - `upsert.externalIdField` (THIS object)\n upsertpicklistvalues:\n type: object\n properties:\n type:\n type: string\n enum:\n - picklist\n - multipicklist\n x-lowercase: true\n x-enumDescriptions:\n picklist: Standard single-select picklist allowing one value at a time; the most common picklist type\n multipicklist: Multi-select picklist allowing multiple values separated by semicolons; use for categories, tags, or multi-value selections such as Tags, Skills, or Interests fields\n description: Picklist field type being managed. Values are converted to lowercase automatically.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Determines whether the picklist allows single or multiple value selection\n - Affects UI rendering and validation in Salesforce\n - Automatically converted to lowercase before processing\n - |-\n Specifies the type of picklist field being managed.\n Salesforce supports single-select picklists and multi-select picklists.\n fullName:\n type: string\n description: |-\n Fully qualified API name of the picklist field, in `ObjectName.FieldName__c` form (for example, `Account.MyPicklist__c`). Must match the case-sensitive Salesforce API name.\n x-celigo-ai-guidance:\n - |-\n ## EXAMPLES:\n - \"Account.MyPicklist__c\"\n - \"Contact.Status__c\"\n - \"CustomObject__c.Category__c\"\n examples: [\"Account.MyPicklist__c\"]\n label:\n type: string\n description: Display label for the picklist field in the Salesforce UI. Can contain spaces and does not need to match the API name.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Human-readable label shown in Salesforce interface\n - Can contain spaces and special characters\n - Does not need to match API name\n - |-\n ## EXAMPLES:\n - \"My Picklist\"\n - \"Product Category\"\n - \"Customer Status\"\n visibleLines:\n type: number\n description: |-\n Number of values visible without scrolling in the Salesforce UI selection box. Only applies when `type` is `multipicklist`.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Only applies to multipicklist fields\n - Controls height of the picklist selection box\n - Default is typically 4-5 lines\n - |-\n ## EXAMPLES:\n - 3 — shows 3 values at once\n - 5 — shows 5 values at once\n - 10 — shows more values for large lists\n - |-\n For multi-select picklists,\n specifies how many lines are visible in the UI without scrolling.\n description: |-\n Picklist field definition managed when `operation` is `upsertpicklistvalues`; omit for all other operations. Manages the field's metadata — name, label, and display settings — and requires Salesforce Metadata API permissions.\n\n Required when operation is upsertpicklistvalues.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Used ONLY when `operation: \"upsertpicklistvalues\"`\n - Manages picklist field metadata, not picklist values themselves\n - Enables programmatic picklist field management\n - Requires appropriate Salesforce metadata permissions\n - |-\n ## IMPORTANT NOTES:\n - This manages the FIELD itself, not the individual picklist VALUES\n - Requires Salesforce API and Metadata API permissions\n - Should only be set when operation is \"upsertpicklistvalues\"\n - Most imports won't use this - it's for metadata management\n - |-\n ## WHEN TO USE:\n - When operation is set to \"upsertpicklistvalues\"\n - For managing picklist field definitions programmatically\n - When syncing picklist configurations between orgs\n - For automated picklist field deployment\n - |-\n Configuration for managing Salesforce picklist field metadata when using the\n `upsertpicklistvalues` operation.\n This enables dynamic creation or updates to picklist field definitions.\n removeNonSubmittableFields:\n type: boolean\n description: |-\n When true, strips fields Salesforce will not accept — read-only, system-generated, and formula fields — from the payload before submission, preventing rejection errors. When false or omitted, all fields are submitted as-is. Only the outgoing payload is affected; the source data is unchanged.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - When enabled (set to true), the system analyzes the outgoing data payload and removes any fields identified as non-submittable based on Salesforce metadata, field-level security, and API constraints.\n - When disabled (set to false) or omitted, all fields present in the payload are submitted as-is, which may lead to errors if restricted, read-only, or system-managed fields are included.\n - Enhances data integrity by proactively filtering out fields that could cause rejection or failure during Salesforce data operations.\n - Applies only to the outgoing submission payload, ensuring the original source data remains unchanged and intact.\n - Dynamically adapts to different Salesforce objects, API versions, and user permissions by referencing up-to-date metadata and security settings.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Enable this flag in integration scenarios where the data schema may include fields that Salesforce does not accept for the target object or API version.\n - Maintain an up-to-date cache or real-time access to Salesforce metadata to accurately reflect field-level permissions, submittability, and API changes.\n - Implement logging or reporting mechanisms to track which fields are removed, aiding in auditing, troubleshooting, and transparency.\n - Use in conjunction with other data validation, transformation, and permission-checking steps to ensure clean, compliant data submissions.\n - Thoroughly test in development and staging environments to understand the impact on data flows, error handling, and downstream processes.\n - Consider user roles and permissions, as submittability may vary depending on the authenticated user's access rights and profile settings.\n - |-\n ## EXAMPLES:\n - `removeNonSubmittableFields: true` — Automatically removes read-only or system-managed fields such as CreatedDate, LastModifiedById, or formula fields before submission.\n - `removeNonSubmittableFields: false` — Submits all fields, potentially causing errors if restricted or read-only fields are included.\n - |-\n Indicates whether fields that are not eligible for submission to Salesforce—such as read-only,\n system-generated, formula,\n or otherwise restricted fields—should be automatically excluded from the data\n payload before it is sent.\n This feature helps prevent submission errors by ensuring that only valid,\n submittable fields are included in the API request,\n thereby improving data integrity and reducing the likelihood of operation failures.\n It operates exclusively on the outgoing submission payload without modifying the\n original source data,\n making it a safe and effective way to sanitize data before interaction with Salesforce APIs.\n document:\n type: object\n properties:\n id:\n type: string\n description: |-\n Salesforce-assigned ID of the Document record. The 15-character form is case-sensitive; the 18-character form adds a case-insensitivity checksum.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Acts as the primary key uniquely identifying a document record within Salesforce.\n - Immutable after creation; should never be changed or reassigned.\n - Used consistently to reference the document across API calls, queries, and integrations.\n - Case-sensitive and may vary in length (15 or 18 characters).\n - Serves as a critical reference point for establishing relationships with other records.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Must be generated exclusively by Salesforce or the managing system to guarantee uniqueness.\n - Should be handled as a string to accommodate Salesforce’s ID formats.\n - Validate incoming IDs against Salesforce ID format standards to ensure correctness.\n - Avoid manual generation or modification of IDs to prevent data inconsistencies.\n - Use the 18-character format when possible to leverage the checksum for case-insensitive environments.\n - |-\n ## EXAMPLES:\n - \"0151t00000ABCDE\"\n - \"0151t00000ABCDEAAA\"\n - |-\n ## IMPORTANT NOTES:\n - IDs are case-sensitive; 15-character IDs are case-sensitive, while 18-character IDs include a case-insensitive checksum.\n - Do not manually create or alter IDs; always use Salesforce-generated values.\n - This ID is critical for performing document updates, deletions, retrievals, and establishing relationships.\n - Using the correct ID format is essential for API compatibility and data integrity.\n - |-\n Unique identifier for the document within the Salesforce system,\n serving as the primary key that distinctly identifies each document record.\n This identifier is immutable once assigned and must not be altered or reused to\n maintain data integrity.\n It is essential for referencing the document in API calls, queries,\n integrations, and any operations involving the document.\n The ID is a base-62 encoded string that can be either 15 characters long,\n which is case-sensitive, or 18 characters long,\n which includes a case-insensitive checksum to enhance reliability in external systems.\n Proper handling of this ID ensures accurate linkage and manipulation of document\n records across various Salesforce processes and integrations.\n - |-\n ## DEPENDENCY CHAIN:\n - Required for all operations on salesforce.document entities, including CRUD actions.\n - Other fields and processes may reference this ID to link related records or trigger actions.\n - Integral to workflows, triggers, and integrations that depend on document identification.\n - |-\n ## TECHNICAL DETAILS:\n - Salesforce record IDs are base-62 encoded strings.\n name:\n type: string\n description: Display name of the document in Salesforce, used to identify it in folders, search, and API operations.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Acts as the primary human-readable identifier for the document.\n - Used in UI elements, search queries, and API calls to locate or reference the document.\n - Typically mandatory when creating or updating document records.\n - Should be unique within the scope of the relevant Salesforce object, folder, or library to avoid ambiguity.\n - Influences sorting, filtering, and organization of documents within lists or folders.\n - Changes to the name may trigger updates or validations in dependent systems or references.\n - Supports international characters to accommodate global users.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Choose descriptive yet concise names to enhance clarity, usability, and discoverability.\n - Validate input to exclude unsupported special characters and ensure compliance with Salesforce naming rules.\n - Enforce uniqueness constraints within the relevant context to prevent conflicts and confusion.\n - Consider the impact of renaming documents on existing references, hyperlinks, integrations, and audit trails.\n - Account for potential length restrictions and implement truncation or user prompts as necessary.\n - Support internationalization by allowing UTF-8 encoded characters where appropriate.\n - Implement validation to prevent use of reserved words or prohibited characters.\n - |-\n ## EXAMPLES:\n - \"Quarterly Sales Report Q1 2024\"\n - \"Employee Handbook\"\n - \"Project Plan - Alpha Release\"\n - \"Marketing Strategy Overview\"\n - \"Client Contract - Acme Corp\"\n - |-\n ## IMPORTANT NOTES:\n - The maximum length of the name may be limited (commonly up to 255 characters), depending on Salesforce configuration.\n - Renaming a document can affect external references, hyperlinks, or integrations relying on the original name.\n - Case sensitivity in name comparisons may vary based on Salesforce settings and should be considered when implementing logic.\n - Special characters and whitespace handling should align with Salesforce’s accepted standards to avoid errors.\n - |-\n The name of the document as stored in Salesforce,\n serving as a unique and human-readable identifier within the Salesforce environment.\n This string value represents the document's title or label,\n enabling users and systems to easily recognize, retrieve,\n and manage the document across user interfaces, search functionalities,\n and API interactions.\n It is a critical attribute used to reference the document in various operations\n such as creation,\n updates, searches, display contexts, and integration workflows,\n ensuring clear identification and organization within folders or libraries.\n The name plays a pivotal role in document management by facilitating sorting,\n filtering, and categorization,\n and it must adhere to Salesforce’s naming conventions and constraints to\n maintain system integrity and usability.\n folderId:\n type: string\n description: |-\n Salesforce folder that stores the document. The document inherits the folder's sharing and visibility settings; changing this value moves the document to a different folder.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Identifies the exact folder containing the document.\n - Associates the document with a designated folder for structured organization.\n - Must reference a valid, existing folder ID within Salesforce.\n - Typically represented as a string conforming to Salesforce ID formats (15- or 18-character).\n - Influences document visibility and access based on folder permissions.\n - Changing the folderId moves the document to a different folder, affecting its context and access.\n - Inherits folder-level sharing and security settings automatically.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Verify that the folderId exists and is accessible in the Salesforce instance before assignment.\n - Validate the folderId format to ensure it matches Salesforce’s 15- or 18-character ID standards.\n - Use this field to filter, query, or categorize documents by their folder location in API operations.\n - When creating or updating documents, setting this field assigns or moves the document to the specified folder.\n - Handle permission checks to ensure the user or integration has rights to assign or access the folder.\n - Consider the impact on sharing rules, workflows, and automation triggered by folder changes.\n - Ensure case sensitivity is preserved when handling folder IDs.\n - |-\n ## EXAMPLES:\n - \"00l1t000003XyzA\" (example of a Salesforce folder ID)\n - \"00l5g000004AbcD\"\n - \"00l9m00000FghIj\"\n - |-\n ## IMPORTANT NOTES:\n - The folderId must be valid and the folder must be accessible by the user or integration performing the operation.\n - Using an incorrect or non-existent folderId will cause errors or result in improper document categorization.\n - Folder-level permissions in Salesforce can restrict the ability to assign documents or access folder contents.\n - Changes to folderId may affect document sharing, visibility settings, and trigger related workflows.\n - Folder IDs are case-sensitive and must be handled accordingly.\n - |-\n The unique identifier of the folder within Salesforce where the document is stored.\n This ID is essential for organizing, categorizing,\n and managing documents efficiently within the Salesforce environment.\n It enables precise association of documents to specific folders,\n facilitating streamlined retrieval, access control,\n and hierarchical organization of document assets.\n By linking a document to a folder,\n it inherits the folder’s permissions and visibility settings,\n ensuring consistent access management and supporting structured document workflows.\n Proper use of this identifier allows for effective filtering, querying,\n and management of documents based on their folder location,\n which is critical for maintaining an organized and secure document repository.\n contentType:\n type: string\n description: MIME type of the document content, such as `application/pdf` or `image/png`.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Defines the media type of the document content to guide processing, rendering, and handling.\n - Identifies the specific file format (e.g., PDF, JPEG, DOCX) for compatibility and validation purposes.\n - Facilitates content negotiation and appropriate response handling between systems and applications.\n - Typically formatted as a standard MIME type string (type/subtype), optionally including parameters such as charset.\n - Influences document handling in workflows, security scans, user interfaces, and API interactions.\n - Serves as a key attribute for determining how the document is stored, transmitted, and displayed.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Always use valid MIME types conforming to official standards (e.g., \"application/pdf\", \"image/png\") to ensure broad compatibility.\n - Verify that the contentType accurately reflects the actual file content to prevent processing errors or misinterpretation.\n - Update the contentType promptly if the document format changes to maintain consistency and reliability.\n - Utilize this field to enable correct content handling in APIs, integrations, client applications, and automated workflows.\n - Include relevant parameters (such as charset for text-based documents) when applicable to provide additional context.\n - Consider validating the contentType against the file extension and content during upload or processing to enhance data integrity.\n - |-\n ## EXAMPLES:\n - \"application/pdf\" for PDF documents.\n - \"image/jpeg\" for JPEG image files.\n - \"application/vnd.openxmlformats-officedocument.wordprocessingml.document\" for Microsoft Word DOCX files.\n - \"text/plain; charset=utf-8\" for plain text files with UTF-8 encoding.\n - \"application/json\" for JSON formatted documents.\n - \"audio/mpeg\" for MP3 audio files.\n - |-\n The MIME type of the document,\n specifying the exact format of the file content stored within the Salesforce document.\n This field enables systems and applications to accurately identify, process,\n render, or handle the document by indicating its media type, such as PDF, image,\n audio, or text formats.\n It adheres to standard MIME type conventions (type/subtype) and may include\n additional parameters like character encoding when necessary.\n Properly setting this field ensures seamless integration, correct display,\n appropriate application usage,\n and reliable content negotiation across diverse platforms and services.\n It plays a critical role in workflows, security scanning, compliance validation,\n and API interactions by guiding how the document is stored, transmitted,\n and interpreted.\n developerName:\n type: string\n description: |-\n Unique API name used to reference the document in code, metadata, and integrations. Distinct from the display name; uses only letters, numbers, and underscores with no spaces.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Serves as a unique and immutable identifier for the document within the Salesforce org.\n - Widely used in code, APIs, metadata, deployment processes, and integrations to reference the document reliably.\n - Should remain stable and not be altered frequently to maintain integration and system stability.\n - Enforces naming conventions that disallow spaces and special characters, favoring camelCase or underscores.\n - Case-insensitive in many contexts but consistent casing is recommended for clarity and maintainability.\n - Acts as a key reference point in metadata relationships and dependency mappings.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Ensure the developerName is unique within the Salesforce environment to prevent naming collisions.\n - Follow Salesforce naming rules: start with a letter, use only alphanumeric characters and underscores.\n - Avoid reserved keywords, spaces, special characters, and punctuation marks.\n - Keep the identifier concise yet descriptive enough to clearly convey the document’s purpose or function.\n - Validate the developerName against Salesforce’s character set, length restrictions, and naming policies before deployment.\n - Plan for the developerName to remain stable post-deployment to avoid breaking references and integrations.\n - Use consistent casing conventions (camelCase or underscores) to improve readability and reduce errors.\n - Incorporate meaningful terms that reflect the document’s role or content for easier identification.\n - |-\n ## EXAMPLES:\n - \"InvoiceTemplate\"\n - \"CustomerAgreementDoc\"\n - \"SalesReport2024\"\n - \"ProductCatalog_v2\"\n - \"EmployeeOnboardingGuide\"\n - |-\n ## IMPORTANT NOTES:\n - Changing the developerName after deployment can disrupt integrations, metadata references, and dependent components.\n - It is distinct from the document’s display name, which can include spaces and be more user-friendly.\n - While Salesforce treats developerName as case-insensitive in many scenarios, maintaining consistent casing improves readability and maintenance.\n - |-\n developerName is a unique,\n developer-friendly identifier assigned to the document within the Salesforce environment.\n It acts as a stable and consistent reference used primarily in programmatic\n contexts such as Apex code,\n API calls, metadata configurations, deployment scripts,\n and integration workflows.\n This identifier ensures reliable access, manipulation,\n and management of the document across various Salesforce components and external systems.\n The value should be concise, descriptive,\n and strictly adhere to Salesforce naming conventions to prevent conflicts and\n ensure maintainability.\n Typically, it employs camelCase or underscores,\n excludes spaces and special characters,\n and is designed to remain unchanged over time to avoid breaking dependencies or integrations.\n isInternalUseOnly:\n type: boolean\n description: When true, marks the document for internal use only — it must not be shared with parties outside the organization.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - When set to true, the document is strictly restricted to internal users and must not be accessible or shared with any external entities.\n - When false or omitted, the document may be accessible to external users, subject to other access control mechanisms.\n - Commonly used to flag documents containing sensitive business strategies, internal communications, proprietary data, or confidential policies.\n - Influences user interface elements and API responses to hide, restrict, or clearly label document visibility accordingly.\n - Changes to this flag can trigger notifications, workflow actions, or audit events to ensure proper handling and compliance.\n - May affect document indexing, search results, and export capabilities to prevent unauthorized dissemination.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Always verify this flag before granting document access in both user interfaces and backend APIs to ensure compliance with internal use policies.\n - Use in conjunction with role-based access controls, group memberships, document classification labels, and data sensitivity tags to enforce comprehensive security.\n - Carefully manage updates to this flag to prevent inadvertent exposure of confidential information; consider implementing approval workflows and multi-level reviews for changes.\n - Implement audit logging for all changes to this flag to support compliance, traceability, and forensic investigations.\n - Ensure all integrated systems, third-party applications, and data export mechanisms respect this flag to maintain consistent access restrictions.\n - Provide clear UI indicators, warnings, or access restrictions to users when accessing documents marked as internal use only.\n - Regularly review and validate the accuracy of this flag to maintain up-to-date security postures.\n - |-\n ## EXAMPLES:\n - true: A confidential internal report on upcoming product launches accessible only to employees.\n - false: A publicly available user manual or product datasheet intended for customers and partners.\n - true: Internal HR policies and procedures documents restricted to company staff.\n - false: Press releases or marketing materials distributed externally.\n - true: Financial forecasts and budget plans shared exclusively within the finance department.\n - true: Internal audit findings and compliance reports\n - |-\n isInternalUseOnly indicates whether the document is intended exclusively for\n internal use within the organization and must not be shared with external\n parties.\n This flag is critical for protecting sensitive, proprietary,\n or confidential information by restricting access strictly to authorized internal users.\n It serves as a definitive marker within the document’s metadata to enforce\n internal visibility policies,\n prevent accidental or unauthorized external distribution,\n and support compliance with organizational security standards.\n Proper handling of this flag ensures that internal communications,\n strategic plans, financial data,\n and other sensitive materials remain protected from exposure outside the company.\n isPublic:\n type: boolean\n description: When true, the document is accessible to all users; when false, access is limited to authorized users by Salesforce permissions and sharing settings.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - When true, the document is accessible by anyone, including unauthenticated users.\n - When false, access is restricted to users with explicit permissions or roles.\n - Directly influences the document’s visibility in search results, sharing interfaces, and external indexing.\n - Changes to this property take immediate effect, altering who can view the document.\n - Modifications may trigger updates in access control enforcement mechanisms.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Use this property to clearly define and enforce document sharing and visibility policies.\n - Always set to false for sensitive, confidential, or proprietary documents to prevent unauthorized access.\n - Implement strict validation to ensure only users with appropriate administrative rights can modify this property.\n - Consider organizational compliance requirements, privacy regulations, and data security standards before making documents public.\n - Regularly monitor and audit changes to this property to maintain security oversight and compliance.\n - Plan changes carefully, as toggling this setting can have immediate and broad impact on document accessibility.\n - |-\n ## EXAMPLES:\n - true: A publicly available product catalog, marketing brochure, or press release intended for external audiences.\n - false: Internal project plans, financial reports, HR documents, or any content restricted to company personnel.\n - |-\n ## IMPORTANT NOTES:\n - Making a document public may expose it to indexing by search engines if accessible via public URLs.\n - Changes to this property have immediate effect on document accessibility; ensure appropriate change management.\n - Ensure alignment with corporate governance, legal requirements, and data protection policies when toggling public access.\n - Public documents should be reviewed periodically to confirm that their visibility remains appropriate.\n - Unauthorized changes to this property can lead to data leaks or compliance violations.\n - |-\n ## DEPENDENCY CHAIN:\n - Access control depends on user roles, profiles, and permission sets configured within the system.\n - |-\n Indicates whether the document is publicly accessible to all users or restricted\n exclusively to authorized users within the system.\n This property governs the document's visibility and access permissions,\n determining if authentication is required to view its content.\n When set to true,\n the document becomes openly available without any access controls,\n allowing anyone—including unauthenticated users—to access it freely.\n Conversely,\n setting this flag to false enforces security measures that limit access based on user roles,\n permissions, and organizational sharing rules,\n ensuring that only authorized personnel can view the document.\n This setting directly impacts how the document appears in search results,\n sharing interfaces, and external indexing services,\n making it a critical factor in managing data privacy, compliance,\n and information governance.\n description: |-\n Salesforce Document record written by the import, combining the file content with metadata such as name, folder, and content type.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Contains both the binary content (usually base64-encoded) and descriptive metadata of a file linked to a Salesforce record.\n - Supports a diverse array of file types including PDFs, images, reports, contracts, spreadsheets, and other common document formats.\n - Facilitates operations such as uploading new files, retrieving existing files, updating metadata, deleting documents, and managing versions within Salesforce.\n - May be required or optional depending on the specific API operation, object context, or organizational business rules.\n - Changes to the document content or metadata can trigger Salesforce workflows, triggers, validation rules, or automation processes.\n - Maintains versioning and audit trails when integrated with Salesforce Content or Files features.\n - Respects Salesforce security and sharing settings to control access and protect sensitive information.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Encode the document content in base64 format when transmitting via API to ensure data integrity and compatibility.\n - Validate file size and type against Salesforce platform limits and organizational policies before upload to prevent errors.\n - Provide comprehensive metadata including file name, description, content type (MIME type), file extension, and relevant tags or categories to facilitate search, classification, and governance.\n - Manage user permissions and access controls in accordance with Salesforce security and sharing settings to safeguard sensitive data.\n - Use appropriate Salesforce API endpoints (e.g., /sobjects/Document, Attachment, ContentVersion) and HTTP methods aligned with the document type and intended operation.\n - Implement robust error handling to manage responses related to file size limits, unsupported formats, permission denials, or network issues.\n - |-\n The 'document' property represents a digital file or record associated with a Salesforce entity,\n encompassing a wide range of file types such as attachments, reports, images,\n contracts, spreadsheets, and other files stored within the Salesforce platform.\n It acts as a comprehensive container that includes both the file's binary\n content—typically encoded in base64—and its associated metadata,\n such as file name, description, MIME type, and tags.\n This property enables seamless integration, retrieval, upload, update,\n and management of files within Salesforce-related workflows and processes.\n By linking relevant documents directly to Salesforce records,\n it enhances data organization, accessibility, collaboration,\n and compliance across various business functions.\n Additionally, it supports versioning, audit trails,\n and can trigger automation processes like workflows, validation rules,\n and triggers,\n ensuring robust document lifecycle management within the Salesforce ecosystem.\n attachment:\n type: object\n properties:\n id:\n type: string\n description: |-\n Salesforce-assigned ID of the Attachment record. Prefer the 18-character case-insensitive form over the 15-character case-sensitive form in integrations.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Acts as the unique and immutable primary key for the attachment record.\n - Required for all API operations involving the attachment, including retrieval, updates, and deletions.\n - Ensures consistent and precise identification of the attachment in all Salesforce API interactions.\n - Remains constant and unchangeable throughout the lifecycle of the attachment once created.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Must strictly adhere to Salesforce ID formats: either 15-character case-sensitive or 18-character case-insensitive alphanumeric strings.\n - Should be dynamically obtained from Salesforce API responses when creating or querying attachments to ensure accuracy.\n - Avoid hardcoding or manually entering IDs to prevent data integrity issues and API errors.\n - Validate the ID format programmatically before use to ensure compatibility with Salesforce API requirements.\n - Prefer using the 18-character ID in integration scenarios to minimize case sensitivity issues.\n - |-\n ## EXAMPLES:\n - \"00P1t00000XyzAbCDE\"\n - \"00P1t00000XyzAbCDEAAA\"\n - \"00P3t00000LmNOpQR1\"\n - |-\n ## IMPORTANT NOTES:\n - The 15-character ID is case-sensitive, whereas the 18-character ID is case-insensitive and recommended for integration scenarios.\n - Using an incorrect, malformed, or outdated ID will result in API errors or failed operations.\n - The ID does not convey any descriptive information about the attachment’s content or metadata.\n - Always retrieve and use the ID directly from Salesforce API responses to ensure validity and prevent errors.\n - The ID is unique within the Salesforce org and cannot be reused or duplicated.\n - |-\n ## DEPENDENCY CHAIN:\n - Generated automatically by Salesforce upon creation of the attachment record.\n - Utilized by other API calls and properties that require precise attachment identification.\n - |-\n Unique identifier for the attachment within the Salesforce system,\n serving as the immutable primary key for the attachment object.\n This ID is essential for accurately referencing, retrieving, updating,\n or deleting the specific attachment record through API calls.\n It must conform to Salesforce's standardized ID format,\n which can be either a 15-character case-sensitive or an 18-character\n case-insensitive alphanumeric string,\n with the 18-character version preferred for integrations due to its case\n insensitivity and reduced risk of errors.\n While the ID itself does not contain metadata about the attachment's content,\n it is crucial for linking the attachment to related Salesforce records and\n ensuring precise operations on the attachment resource across the platform.\n name:\n type: string\n description: Filename of the attachment, including the file extension (for example, `contract_agreement.pdf`).\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Stores the filename of the attachment as a string value.\n - Acts as the key identifier for the attachment within the Salesforce environment.\n - Displayed prominently in the Salesforce UI and API responses when viewing or managing attachments.\n - Should be unique within the context of the parent record to avoid confusion.\n - Includes file extensions (e.g., .pdf, .jpg, .docx) to denote the file type and facilitate correct processing.\n - Case sensitivity may apply depending on the context, affecting retrieval and display.\n - Used in URL generation and API endpoints, impacting accessibility and referencing.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Validate filenames to exclude prohibited characters (such as \\ / : * ? \" \u003c > |) and other special characters that could cause errors or security vulnerabilities.\n - Ensure the filename length does not exceed Salesforce’s maximum limit, typically 255 characters.\n - Always include the appropriate and accurate file extension to enable correct file type recognition.\n - Use clear, descriptive, and consistent naming conventions that reflect the content or purpose of the attachment.\n - Perform validation checks before upload to prevent failures or inconsistencies.\n - Avoid renaming attachments post-upload to maintain integrity of references and links.\n - Consider localization and encoding standards to support international characters where applicable.\n - |-\n ## EXAMPLES:\n - \"contract_agreement.pdf\"\n - \"profile_picture.jpg\"\n - \"sales_report_q1_2024.xlsx\"\n - \"meeting_notes.txt\"\n - \"project_plan_v2.docx\"\n - |-\n ## IMPORTANT NOTES:\n - Filenames are case-sensitive in certain contexts, which can impact retrieval and display.\n - Avoid special characters that may interfere with URLs, file systems, or API processing.\n - Renaming an attachment after upload can disrupt existing references or links to the file.\n - |-\n The name property represents the filename of the attachment within Salesforce\n and serves as the primary identifier for the attachment.\n It is crucial for enabling easy recognition, retrieval,\n and management of attachments both in the user interface and through API operations.\n This property typically includes the file extension,\n which specifies the file type and ensures proper handling, display,\n and processing of the file across different systems and platforms.\n The filename should be meaningful, descriptive,\n and concise to provide clarity and ease of use,\n especially when managing multiple attachments linked to a single parent record.\n Adhering to proper naming conventions helps maintain organization,\n improves searchability,\n and prevents conflicts or ambiguities that could arise from duplicate or unclear filenames.\n parentId:\n type: string\n description: |-\n Salesforce record the attachment is linked to, such as an Account, Contact, or Opportunity. Required when creating an attachment; access to the attachment follows the parent record's permissions.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Represents the Salesforce record ID that the attachment is associated with.\n - Mandatory when creating or updating an attachment to specify its parent record.\n - Must correspond to an existing Salesforce record ID of a valid object type that supports attachments.\n - Enables querying, filtering, and reporting of attachments based on their parent record.\n - Enforces referential integrity between attachments and parent records.\n - Changes to the parent record, including deletion or modification, can affect the visibility, accessibility, and lifecycle of the attachment.\n - Access to the attachment is governed by the permissions and sharing settings of the parent record.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Ensure the parentId is a valid 15- or 18-character Salesforce record ID, with preference for 18-character IDs to avoid case sensitivity issues.\n - Verify the existence and active status of the parent record before assigning the parentId.\n - Confirm that the parent object type supports attachments to prevent errors during attachment operations.\n - Handle user permissions and sharing settings to ensure appropriate access rights to the parent record and its attachments.\n - Use Salesforce APIs or metadata services to validate the parentId format, object type, and existence.\n - Implement robust error handling for invalid, non-existent, or unauthorized parentId values during attachment creation or updates.\n - Consider the impact of parent record lifecycle events (such as deletion or archiving) on associated attachments.\n - |-\n ## EXAMPLES:\n - \"0011a000003XYZ123\" (Account record ID)\n - \"0031a000004ABC456\" (Contact record ID)\n - \"0061a000005DEF789\" (Opportunity record ID)\n - |-\n The unique identifier of the parent Salesforce record to which the attachment is linked.\n This field establishes a direct and essential relationship between the\n attachment and its parent object,\n such as an Account, Contact, Opportunity,\n or any other Salesforce record type that supports attachments.\n It ensures that attachments are correctly associated within the Salesforce data model,\n enabling accurate retrieval, management,\n and organization of attachments in relation to their parent records.\n By linking attachments to their parent records,\n this field facilitates data integrity, efficient querying,\n and seamless navigation between related records.\n It is a mandatory field when creating or updating attachments and must reference a valid,\n existing Salesforce record ID.\n Proper use of this field guarantees referential integrity,\n enforces access controls based on parent record permissions,\n and impacts the visibility and lifecycle of the attachment within the Salesforce environment.\n contentType:\n type: string\n description: MIME type of the attachment content, such as `application/pdf` or `image/png`.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Defines the media type of the attachment to guide processing, rendering, and handling decisions.\n - Used by clients, servers, and intermediaries to determine appropriate actions such as inline display, download prompts, or specialized processing.\n - Supports validation of file format during upload, storage, and download operations to ensure data integrity.\n - Influences security measures including filtering, scanning for malware, and enforcing content policies.\n - Affects user interface elements like preview generation, icon selection, and content categorization.\n - May be used in logging, auditing, and analytics to track content types handled by the system.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Must strictly adhere to the standard MIME type format as defined by IANA (e.g., \"type/subtype\").\n - Should accurately reflect the actual content format to prevent misinterpretation or processing errors.\n - Use generic types such as \"application/octet-stream\" only when the specific type cannot be determined.\n - Include optional parameters (e.g., charset, boundary) when relevant to fully describe the content.\n - Ensure the contentType is set or updated whenever the attachment content changes to maintain consistency.\n - Validate the contentType against the file extension and actual content where feasible to enhance reliability.\n - Consider normalizing the contentType to lowercase to maintain consistency across systems.\n - Avoid relying solely on contentType for security decisions; combine with other metadata and content inspection.\n - |-\n ## EXAMPLES:\n - \"image/jpeg\"\n - \"application/pdf\"\n - \"text/plain; charset=utf-8\"\n - \"application/vnd.ms-excel\"\n - \"audio/mpeg\"\n - \"video/mp4\"\n - \"application/json\"\n - \"multipart/form-data; boundary=something\"\n - \"application/zip\"\n - |-\n The MIME type of the attachment content,\n specifying the exact format and nature of the file (e.g., \"image/png\",\n \"application/pdf\", \"text/plain\").\n This information is critical for correctly interpreting, processing,\n and displaying the attachment across different systems and platforms.\n It enables clients and servers to determine how to handle the file,\n whether to render it inline, prompt for download,\n or apply specific processing rules.\n Accurate contentType values facilitate validation, security scanning,\n and compatibility checks,\n ensuring that the attachment is managed appropriately throughout its lifecycle.\n Properly defined contentType values improve interoperability, user experience,\n and system security by enabling precise content handling and reducing the risk\n of errors or vulnerabilities.\n isPrivate:\n type: boolean\n description: |-\n When true, the attachment is visible only to users explicitly authorized to access it; when false or omitted, it inherits the visibility of its parent record. This setting only restricts access further — it never grants access beyond the parent record's permissions.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - When set to true, the attachment is accessible only to users with explicit authorization, effectively hiding it from unauthorized users.\n - When set to false or omitted, the attachment inherits the visibility of its parent record and is accessible to all users who have access to that record.\n - Directly affects the attachment’s visibility scope and access control within Salesforce.\n - Changes to this field immediately impact user access permissions for the attachment.\n - Does not override broader organizational sharing settings but further restricts access.\n - The privacy setting applies in conjunction with existing record-level and object-level security controls.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Use this field to enforce strict data privacy and protect sensitive attachments in compliance with organizational policies and regulatory requirements.\n - Ensure that user roles, profiles, and sharing rules are properly configured to respect this privacy setting.\n - Validate that the value is a boolean (true or false) to prevent data inconsistencies.\n - Evaluate the implications on existing sharing rules and record-level security before modifying this field.\n - Consider implementing audit logging to track changes to this privacy setting for security and compliance purposes.\n - Test changes in a sandbox environment to understand the impact on user access before deploying to production.\n - Communicate changes to relevant stakeholders to avoid confusion or unintended access issues.\n - |-\n ## EXAMPLES:\n - `true` — The attachment is private and visible only to users explicitly authorized to access it.\n - `false` — The attachment is public within the context of the parent record and visible to all users who have access to that record.\n - |-\n ## IMPORTANT NOTES:\n - Setting this field to true does not grant access to users who lack permission to view the parent record; it only further restricts access.\n - Modifying this setting can significantly affect user access and should be managed carefully to avoid unintended data exposure or access denial.\n - Some Salesforce editions or configurations may have limitations or specific behaviors regarding attachment privacy\n - |-\n Indicates whether the attachment is designated as private,\n restricting its visibility exclusively to users who have been explicitly\n authorized to access it.\n This setting is critical for maintaining confidentiality and ensuring that\n sensitive or proprietary information contained within attachments is accessible\n only to appropriate personnel.\n When marked as private,\n the attachment’s visibility is constrained based on the user’s permissions and\n the sharing settings of the parent record,\n thereby reinforcing data security within the Salesforce environment.\n This field directly controls the scope of access to the attachment,\n dynamically influencing who can view or interact with it,\n and must be managed carefully to align with organizational privacy policies and\n compliance requirements.\n description:\n type: string\n description: Free-text note describing the attachment's content or purpose, shown alongside the attachment details in Salesforce.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Optional and editable field used to describe the attachment’s nature or relevance.\n - Supports plain text input including special characters, punctuation, and Unicode.\n - Displayed prominently in user interfaces where attachment details appear.\n - Utilized in search, filtering, sorting, and reporting functionalities to enhance data retrieval.\n - Does not contain or affect the actual attachment content or file storage.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Encourage concise yet informative descriptions to aid quick identification.\n - Validate and sanitize input to prevent injection attacks and ensure data integrity.\n - Support full Unicode to accommodate international characters and symbols.\n - Enforce a reasonable maximum length (commonly 255 characters) to maintain performance and UI consistency.\n - Avoid inclusion of sensitive or confidential information unless appropriate security controls are applied.\n - Implement trimming or sanitization routines to maintain clean and consistent data.\n - |-\n ## EXAMPLES:\n - \"Quarterly financial report for Q1 2024.\"\n - \"Signed contract agreement with client XYZ.\"\n - \"Product brochure for the new model launch.\"\n - \"Meeting notes and action items from April 15, 2024.\"\n - \"Technical specification document for project Alpha.\"\n - |-\n ## IMPORTANT NOTES:\n - This field contains descriptive metadata only, not the attachment content.\n - Sensitive information should be excluded or protected according to security policies.\n - Changes to this field do not impact the attachment file or its storage.\n - Consistent and accurate descriptions improve attachment management and user experience.\n - Plays a key role in enhancing searchability and filtering within Salesforce.\n - |-\n ## DEPENDENCY CHAIN:\n - Directly associated with the attachment entity in Salesforce.\n - Often used in conjunction with other metadata fields such as attachment name, type, owner, size, and creation date.\n - Supports comprehensive context when combined with related records and metadata.\n - |-\n A textual description providing additional context, detailed information,\n and clarifying the content, purpose,\n or relevance of the attachment within Salesforce.\n This field enhances user understanding and facilitates easier identification,\n organization,\n and management of attachments by serving as descriptive metadata that\n complements the attachment’s core data without containing the attachment content\n itself.\n It supports plain text with special characters, punctuation, and full Unicode,\n enabling expressive, internationalized, and accessible descriptions.\n Typically displayed in user interfaces alongside attachment details,\n this description also plays a crucial role in search, filtering, sorting,\n and reporting operations, improving overall data discoverability and usability.\n - |-\n ## TECHNICAL DETAILS:\n - Data type: string.\n - Maximum length: typically 255 characters, configurable\n description: |-\n Salesforce Attachment record written by the import — a file linked to a parent record via `parentId`, with metadata such as name and content type.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Stores the binary content or a reference to a file linked to a specific Salesforce record.\n - Supports multiple file formats including but not limited to documents, images, PDFs, spreadsheets, and email files.\n - Enables attaching additional context, documentation, or evidence to Salesforce objects.\n - Allows full lifecycle management: create, read, update, and delete operations on attachments.\n - Contains metadata such as file name, MIME content type, file size, and the ID of the parent Salesforce record.\n - Reflects changes immediately in the associated Salesforce record’s file attachments.\n - Maintains a direct association with the parent record through the ParentId field to ensure accurate linkage.\n - Supports integration with Salesforce Files (ContentVersion and ContentDocument) for advanced file management features.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Encode file content using Base64 encoding when transmitting via Salesforce APIs to ensure data integrity.\n - Validate file size against Salesforce limits (commonly up to 25 MB per attachment) and confirm supported file types before upload.\n - Use the ParentId field to correctly associate the attachment with the intended Salesforce record.\n - Implement robust permission checks to ensure only authorized users can upload, view, or modify attachments.\n - Consider leveraging Salesforce Files (ContentVersion and ContentDocument objects) for enhanced file management capabilities, versioning, and sharing features, especially for new implementations.\n - Handle error responses gracefully, including those related to file size limits, unsupported formats, or permission issues.\n - Ensure metadata fields such as file name and content type are accurately populated to facilitate proper handling and display.\n - When updating attachments, confirm that the new content or metadata aligns with organizational policies and compliance requirements.\n - |-\n The attachment property represents a file or document linked to a Salesforce record,\n encompassing both the binary content of the file and its associated metadata.\n It serves as a versatile container for a wide variety of file types—including documents,\n images, PDFs, spreadsheets,\n and emails—allowing users to enrich Salesforce records such as emails, cases,\n opportunities,\n or any standard or custom objects with supplementary information or evidence.\n This property supports comprehensive lifecycle management,\n enabling operations such as uploading new files,\n retrieving existing attachments, updating file details,\n and deleting attachments as needed.\n By maintaining a direct association with the parent Salesforce record through\n identifiers like ParentId,\n it ensures accurate linkage and seamless integration within the Salesforce ecosystem.\n The attachment property facilitates enhanced collaboration, documentation,\n and data completeness by providing a centralized and manageable way to handle\n files related to Salesforce data.\n contentVersion:\n type: object\n properties:\n contentDocumentId:\n type: string\n description: |-\n ContentDocument this version belongs to; all versions of the same document share this ID. Assigned by Salesforce when the content version is created and immutable afterward.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Represents the unique ID of the parent Content Document to which the Content Version belongs.\n - Acts as the primary link connecting multiple versions of the same document.\n - Immutable after the Content Version record is created, preserving data integrity.\n - Enables navigation from a specific Content Version to its parent Content Document.\n - Automatically assigned and managed by Salesforce during Content Version creation.\n - Essential for maintaining consistent version histories and document relationships.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Use this ID to reference or retrieve the entire document across all its versions.\n - Verify that the ID corresponds to an existing Content Document before usage.\n - Utilize in SOQL queries and API calls to associate Content Versions with their parent document.\n - Avoid modifying this field directly; it is managed internally by Salesforce.\n - Employ this ID to support document lifecycle management, version tracking, and collaboration features.\n - Leverage this field to ensure accurate linkage in integrations and custom development.\n - |-\n ## EXAMPLES:\n - \"0691t00000XXXXXXAAA\"\n - \"0692a00000YYYYYYBBB\"\n - \"0693b00000ZZZZZZCCC\"\n - |-\n ## IMPORTANT NOTES:\n - Distinct from the Content Version ID, which identifies a specific version rather than the entire document.\n - Mandatory for any valid Content Version record to ensure proper document association.\n - Cannot be null or empty for Content Version records.\n - Critical for preserving document integrity, version tracking, and enabling collaborative document management.\n - Changes to this ID are not permitted post-creation to avoid breaking version linkages.\n - Plays a key role in Salesforce’s document sharing, access control, and audit capabilities.\n - |-\n ## DEPENDENCY CHAIN:\n - Requires the existence of a valid Content Document record in Salesforce.\n - Directly related to Content Version records representing different iterations of the same document.\n - |-\n contentDocumentId is the unique identifier for the parent Content Document\n associated with a specific Content Version in Salesforce.\n It serves as the fundamental reference that links all versions of the same document,\n enabling comprehensive version control and efficient document management within\n the Salesforce ecosystem.\n This ID is immutable once the Content Version is created,\n ensuring a stable and consistent connection across all iterations of the document.\n By maintaining this linkage,\n Salesforce facilitates seamless navigation between individual document versions\n and their overarching document entity,\n supporting streamlined retrieval, updates, collaboration,\n and organizational workflows.\n This field is automatically assigned by Salesforce during the creation of a\n Content Version and is critical for maintaining the integrity and traceability\n of document versions throughout their lifecycle.\n title:\n type: string\n description: Display title of this content version, shown in content libraries, search results, and version history.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Functions as the main identifier or label for the content version in user interfaces, listings, and reports.\n - Enables users to distinguish between different versions of the same content efficiently.\n - Reflects the content’s subject, purpose, key attributes, or version status.\n - Should be concise but informative to facilitate quick recognition and comprehension.\n - Commonly displayed in search results, filters, version histories, and content libraries.\n - Supports sorting and filtering operations to streamline content management.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Use clear, descriptive titles that accurately represent the content version’s focus or changes.\n - Avoid special characters or formatting that could cause display, parsing, or processing issues.\n - Keep titles within a reasonable length (typically up to 255 characters) to ensure readability across various UI components and devices.\n - Update the title appropriately when creating new versions to reflect significant updates or revisions.\n - Consider including version numbers, dates, or status indicators to enhance clarity and version tracking.\n - Follow organizational naming conventions and standards to maintain consistency.\n - |-\n ## EXAMPLES:\n - \"Q2 Financial Report 2024\"\n - \"Product Launch Presentation v3\"\n - \"Employee Handbook - Updated March 2024\"\n - \"Marketing Strategy Overview\"\n - \"Website Redesign Proposal - Final Draft\"\n - |-\n ## IMPORTANT NOTES:\n - Titles do not need to be globally unique but should be meaningful and contextually relevant.\n - Changing the title does not modify the underlying content data or its integrity.\n - Titles are frequently used in search, filtering, sorting, and reporting operations within Salesforce.\n - Consistent naming conventions and formatting improve content management efficiency and user experience.\n - Consider organizational standards or guidelines when defining title formats.\n - Titles should avoid ambiguity to prevent confusion between similar content versions.\n - |-\n ## DEPENDENCY CHAIN:\n - Part of the ContentVersion object within Salesforce’s content management framework.\n - Often associated with other metadata fields such as description\n - |-\n The title of the content version serves as the primary name or label assigned to\n this specific iteration of content within Salesforce.\n It acts as a key identifier that enables users to quickly recognize,\n differentiate, and manage various versions of content.\n The title typically summarizes the subject matter, purpose,\n or significant details about the content,\n making it easier to locate and understand at a glance.\n It should be concise yet sufficiently descriptive to effectively convey the\n essence and context of the content version.\n Well-crafted titles improve navigation, searchability,\n and version tracking within content management workflows,\n enhancing overall user experience and operational efficiency.\n pathOnClient:\n type: string\n description: |-\n Original file path on the client machine before upload. Used for reference and display only; it does not affect how the file is stored or accessed in Salesforce.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Represents the full absolute or relative file path on the client device where the file was stored prior to upload.\n - Automatically populated during file upload processes when the client environment provides this information.\n - Used mainly for reference, auditing, and user interface display to provide context about the file’s origin.\n - May be empty, null, or omitted if the file was not uploaded from a client device or if the path information is unavailable or restricted.\n - Does not influence file storage, retrieval, or access permissions within Salesforce.\n - Retains the original formatting and syntax as provided by the client system at upload time.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Capture the file path exactly as provided by the client system at the time of upload, preserving the original format and case sensitivity.\n - Handle differences in file path syntax across operating systems (e.g., backslashes for Windows, forward slashes for Unix/Linux/macOS).\n - Treat this property strictly as metadata for informational purposes; avoid using it for security, access control, or file retrieval logic.\n - Sanitize or mask sensitive directory or user information when displaying or logging this path to protect user privacy and comply with data protection regulations.\n - Ensure compliance with relevant privacy laws and organizational policies when storing or exposing client file paths.\n - Consider truncating, normalizing, or encoding paths if necessary to conform to platform length limits, display constraints, or to prevent injection vulnerabilities.\n - Provide clear documentation to users and administrators about the non-security nature of this property to prevent misuse.\n - |-\n ## EXAMPLES:\n - \"C:\\\\Users\\\\JohnDoe\\\\Documents\\\\Report.pdf\"\n - \"/home/janedoe/projects/presentation.pptx\"\n - \"Documents/Invoices/Invoice123.pdf\"\n - \"Desktop/Project Files/DesignMockup.png\"\n - \"../relative/path/to/file.txt\"\n - |-\n The pathOnClient property specifies the original file path of the content as it\n exists on the client machine before being uploaded to Salesforce.\n This path serves as a precise reference to the source location of the file on\n the user's local system,\n facilitating identification, auditing, and tracking of the file's origin.\n It captures either the full absolute path or a relative path depending on the\n client environment and upload context,\n accurately reflecting the exact location from which the file was sourced.\n This information is primarily used for informational, diagnostic,\n and user interface purposes within Salesforce and does not influence how the file is stored,\n accessed, or secured in the system.\n tagCsv:\n type: string\n description: |-\n Comma-separated tags applied to the content version for categorization and search. Updating this field replaces the entire tag set, so include every tag the version should keep.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Accepts multiple tags separated by commas, typically without spaces, though trimming whitespace is recommended.\n - Tags are used to categorize, organize, and improve the discoverability of content versions.\n - Tags can be added, updated, or removed by modifying the entire string value.\n - Duplicate tags within the string should be avoided to maintain clarity and effectiveness.\n - Tags are generally treated as case-insensitive for search purposes but stored exactly as entered.\n - Changes to this field immediately affect how content versions are indexed and retrieved in search operations.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Ensure individual tags do not contain commas to prevent parsing errors.\n - Validate the tagCsv string for invalid characters, excessive length, and proper formatting.\n - When updating tags, replace the entire string to accurately reflect the current tag set.\n - Trim leading and trailing whitespace from each tag before processing or storage.\n - Utilize this field to enhance search filters, categorization, and content management workflows.\n - Implement application-level checks to prevent duplicate or meaningless tags.\n - Consider standardizing tag formats (e.g., lowercase) to maintain consistency across records.\n - |-\n ## EXAMPLES:\n - \"marketing,Q2,presentation\"\n - \"urgent,clientA,confidential\"\n - \"releaseNotes,v1.2,approved\"\n - \"finance,yearEnd,reviewed\"\n - |-\n ## IMPORTANT NOTES:\n - The maximum length of the tagCsv string is subject to Salesforce field size limitations.\n - Tags should be meaningful, consistent, and standardized to maximize their utility.\n - Modifications to tags can impact content visibility and access if used in filtering or permission logic.\n - This field does not enforce uniqueness of tags; application-level logic should handle duplicate prevention.\n - Tags are stored as-is; any normalization or case conversion must be handled externally.\n - Improper formatting or invalid characters may cause errors or unexpected behavior in search and filtering.\n - |-\n ## DEPENDENCY CHAIN:\n - Closely related to contentVersion metadata and search/filtering functionality.\n - |-\n A comma-separated string representing the tags associated with the content\n version in Salesforce.\n This field enables the assignment of multiple descriptive tags within a single string,\n facilitating efficient categorization, organization,\n and enhanced searchability of the content.\n Tags serve as metadata labels that help users filter, locate,\n and manage content versions effectively across the platform by grouping related\n items and supporting advanced search and filtering capabilities.\n Proper use of this field improves content discoverability,\n streamlines content management workflows,\n and supports dynamic content classification.\n contentLocation:\n type: string\n description: Specifies where the content file for this ContentVersion record is stored — \"S\" for Salesforce internal storage or \"E\" for an external system (the two values the import form offers). Salesforce also defines \"L\" for content shared via a link, but the form does not set it. Typically assigned automatically based on the upload method.\n x-celigo-ai-guidance:\n - |-\n ## FIELD BEHAVIOR:\n - Determines the origin and storage context of the content file linked to the ContentVersion record.\n - Influences the mechanisms and protocols used for accessing, retrieving, and managing the content.\n - Typically assigned automatically by Salesforce based on the upload method or integration configuration.\n - Common values include 'S' for Salesforce internal storage, 'E' for external storage systems, and 'L' for linked external locations.\n - Often read-only to preserve data integrity and prevent unauthorized or unintended modifications.\n - Alterations to this field can impact content visibility, sharing settings, and delivery workflows.\n - |-\n ## IMPLEMENTATION GUIDANCE:\n - Utilize this field to programmatically distinguish between internally stored content and externally managed files.\n - When integrating with external repositories, ensure contentLocation accurately reflects the external storage to enable correct content handling and retrieval.\n - Avoid manual updates unless supported by Salesforce documentation and accompanied by a comprehensive understanding of the consequences.\n - Validate the contentLocation value prior to processing or delivering content to guarantee proper routing and access control.\n - Confirm that external storage systems are correctly configured, authenticated, and authorized to maintain uninterrupted access.\n - Monitor this field closely during content migration or integration activities to prevent data inconsistencies or access disruptions.\n - |-\n ## EXAMPLES:\n - 'S' — Content physically stored within Salesforce’s internal storage infrastructure.\n - 'E' — Content residing in an external system integrated with Salesforce, such as a third-party content repository.\n - 'L' — Content located in a linked external location, representing specialized or less common storage configurations.\n - |-\n ## IMPORTANT NOTES:\n - Manual modifications to contentLocation can lead to broken links, access failures, or data inconsistencies.\n - Salesforce generally manages this field automatically to uphold consistency and reliability.\n - The contentLocation value directly influences content delivery methods and access control policies.\n - External content locations require proper configuration, authentication, and permissions to function correctly\n - |-\n contentLocation specifies the precise storage location of the content file\n associated with the Salesforce ContentVersion record.\n It identifies where the actual file data resides—whether within Salesforce's\n internal storage infrastructure,\n an external content management system,\n or a linked external repository—and governs how the content is accessed,\n retrieved, and managed throughout its lifecycle.\n This field plays a crucial role in content delivery, access permissions,\n and integration workflows by clearly indicating the source location of the file data.\n Properly setting and interpreting contentLocation ensures that content is\n handled correctly according to its storage context,\n enabling seamless access when stored internally or facilitating robust\n integration and synchronization with third-party external systems.\n description: |-\n Salesforce ContentVersion record written by the import — a specific version of a content item in Salesforce Files. Each modification produces a new version, and all versions of a document are linked through `contentDocumentId`.\n # Shape → requireds. Nested if/else-if (no allOf), stack-style.\n # Branch 1: blob configs (attachment / contentVersion) — operation only; no api, and\n # the operation→idLookup/upsert rules do not apply (the blob record carries its own\n # identifiers; the current server accepts blob upserts without them).\n # Branch 2: server-written bookkeeping stubs — no requireds. Stub keys are {lookups}\n # plus `api`, which the server defaults to \"rest\" on some stubs, so its presence\n # alone does not indicate a user-authored config.\n # Branch 3: real record configs — operation + api, then operation → matching config\n # (update/delete/addupdate share one branch because their requirements are identical).\n if:\n anyOf:\n - required: [attachment]\n - required: [contentVersion]\n then:\n required:\n - operation\n else:\n if:\n not:\n propertyNames:\n enum: [lookups, api]\n then:\n required:\n - operation\n - api\n if:\n properties:\n operation:\n const: upsert\n required:\n - operation\n then:\n required:\n - idLookup\n - upsert\n properties:\n idLookup:\n required:\n - extract\n upsert:\n required:\n - externalIdField\n else:\n if:\n properties:\n operation:\n enum: [update, delete, addupdate]\n required:\n - operation\n then:\n required:\n - idLookup\n properties:\n idLookup:\n required:\n - whereClause\n else:\n if:\n properties:\n operation:\n const: upsertpicklistvalues\n required:\n - operation\n then:\n required:\n - upsertpicklistvalues\n properties:\n upsertpicklistvalues:\n required:\n - fullName\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":138767,"content_sha256":"7b4b1f7d64499bba900151f1ecd9e34895d79e8236b7f0244bbc9e9a472241f3"},{"filename":"references/schemas/tool.yml","content":"Tool:\n type: object\n description: |-\n Configuration for a `ToolImport`, which executes a reusable Celigo Tool resource as the\n import action. The referenced tool defines its own input contract, processing pipeline, and\n output; this object binds the tool to the import, along with the connections the tool's steps\n use for this invocation.\n x-celigo-ai-guidance:\n - |-\n A ToolImport is the **wrapper** that lets a flow / API / another Tool consume a\n Celigo Tool resource as a step.\n It carries a pointer to the underlying Tool plus per-consumer connection bindings.\n - |-\n On the wire,\n the wrapper IS a regular import — the pageProcessor entry that references it is just `{type:\n \"import\", _importId: \u003cwrapper>}`.\n The `adaptorType:\n \"ToolImport\"` discriminator and this `tool` block are how the platform knows to\n invoke the underlying Tool at runtime instead of running a normal import.\n - |-\n The Celigo Tool itself is a separate resource (lives at `/v1/tools/{_toolId}`)\n and can be reused by many wrappers — the wrapper is consumer-specific (which\n connections to bind,\n which upstream-record-to-input mappings to apply); the Tool is shared.\n Modify the Tool through `/v1/tools` directly;\n modify the wrapper through `/v1/imports`.\n required:\n - _toolId\n properties:\n _toolId:\n type: string\n format: objectId\n x-celigo-refModel: tools\n description: |-\n Tool resource this import executes. The tool supplies the input schema, routing, page\n processors, and output mapping; the import runs that tool when records reach it. Required —\n a `ToolImport` has no behavior without a tool to run.\n x-celigo-ai-guidance:\n - Reference to the Celigo Tool resource this wrapper invokes.\n - |-\n Required.\n The Tool must already exist;\n the wrapper has nothing to do without a real `_toolId` to point at.\n To change which Tool the wrapper invokes,\n update this field — the wrapper itself stays the same shape,\n just points at a different Tool.\n - |-\n Writing the wrapper does not create or change the Tool itself;\n author the Tool as its own resource through `/v1/tools`.\n examples: [\"69ce8ae7cbbb51405ae42ae5\"]\n overrides:\n type: object\n description: |-\n Per-import overrides applied to the referenced tool at execution time. Tools use a\n bring-your-own-keys connection model: the tool declares abstract connection slots, and the\n consumer binds concrete connections here so one tool definition can run against different\n systems without being modified. Bind every connection the tool's steps use; to keep the\n connection a step was authored against, bind it to itself.\n x-celigo-ai-guidance:\n - |-\n Per-consumer overrides applied when the wrapper invokes the underlying Tool.\n Currently only carries connection bindings — the underlying Tool's lookups and\n imports each reference a `_connectionId`,\n and consumers bind real account connections to those abstract requirements here.\n - |-\n The save accepts a wrapper without bindings, but when the Tool's steps use connections\n its run then fails with `connection_not_overridden`. Write one `connections` entry per\n connection the Tool's steps use, even when the wrapper keeps the Tool's authored\n connection (`_id` equal to `_abstractId`); a Tool whose steps use no connection needs none.\n properties:\n connections:\n type: array\n description: |-\n Connection bindings that replace the tool's abstract connection slots for this import.\n Each entry maps one of the tool's abstract connections to the concrete connection the\n import should use.\n x-celigo-ai-guidance:\n - |-\n Connection-binding overrides.\n One entry per abstract connection requirement the Tool exposes.\n - |-\n **Discovering abstract requirements.** The Tool resource does not carry an\n explicit \"abstract requirements\" block.\n Walk the Tool's `routers[].branches[].pageProcessors[]` array,\n fetch each underlying export / import resource,\n and collect the unique set of `_connectionId` values.\n Each unique connection id is one abstract requirement;\n the connection's `type` field is its adaptor type (`salesforce`, `netsuite`,\n `http`, etc.).\n - |-\n **Resolving each binding.** For each abstract id,\n find a connection in the consumer's account whose adaptor type matches.\n When multiple match,\n the user prompt / context disambiguates (production vs. sandbox, etc.).\n When no match is found,\n fall back to reusing the abstract id as the bound id — that means \"use the\n connection the Tool was authored against,\n as-is\" and is the right fallback when the Tool author and the consumer share a connection.\n items:\n type: object\n required:\n - _id\n - _abstractId\n properties:\n _id:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: Concrete connection bound to the abstract slot for this\n import's tool run.\n x-celigo-ai-guidance:\n - |-\n The real account connection the wrapper binds to this abstract requirement.\n May equal `_abstractId` when the wrapper inherits the Tool's authored connection.\n examples: [\"698af29c387b80d0b91659ad\"]\n _abstractId:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: |-\n Abstract connection slot, declared by the tool, that this binding replaces. Matches\n the abstract connection id defined on the referenced tool.\n x-celigo-ai-guidance:\n - |-\n The abstract connection id the Tool's underlying exports / imports reference.\n Discovered by walking the Tool's pageProcessors as described above.\n examples: [\"698af29c387b80d0b91659ad\"]\n _borrowConcurrencyFromConnectionId:\n type: string\n format: objectId\n x-celigo-refModel: connections\n description: |\n Optional pointer to another connection whose\n concurrency budget this binding shares. Use\n when several wrappers / steps should share rate\n limits against the same downstream system but\n use different credentials. Mutually exclusive\n with `_id` on the same entry.\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":7019,"content_sha256":"c06a7e29a47893f4701543af0e13f7d54e8934a786f5eafea6acd6d57e0da7d1"},{"filename":"references/schemas/topic.yml","content":"Topic:\n type: object\n description: |-\n Publishing settings for an Event Streams publisher (`adaptorType: TopicImport`). Each\n incoming record is published as a message to the topic; the platform delivers it\n internally, so the import carries no `_connectionId`. Present when `adaptorType` is\n `TopicImport`; the server fills `batchSize` with its default when it is omitted.\n x-celigo-ai-guidance:\n - |-\n Sending this object without `adaptorType` is enough — the server infers `TopicImport`\n from it. Never add a `_connectionId`: the server then validates the document as that\n connection's adaptor instead (an http connection fails with 422 missing_required_field\n \"http subschema not defined\").\n - |-\n Without a mapping, each message is the record as the step received it. With\n `oneToMany: true` and `pathToMany`, every element of that array is published as its own\n message and the rest of the parent record rides inside each one under `_PARENT` (the parent\n without the array); a mapping reshapes the message instead.\n - |-\n Publishing is not limited to flows. Verified surfaces: a flow step; an API step (the API\n invoked at `POST /apis/{version}{relativeURI}`); a Tool reached through a `ToolImport`\n step in a flow; and `POST /v1/imports/{_id}/invoke` with `{\"data\": [record, ...]}`. A flow\n or API test run (`/test/run`) publishes nothing — import steps are mocked there.\n properties:\n _topicId:\n type: string\n format: objectId\n x-celigo-refModel: topics\n x-celigo-ui-override: >-\n Required by the Event Streams publisher form (fieldDefinitions/resources/imports/import.js:\n 'topic._topicId' is required:true, validWhen isNot ''). The API saves a TopicImport\n without it, so the requirement is encoded on the request schema to mirror the form and\n keep publishers bound to a topic.\n description: |-\n Topic the records are published to. Create or look it up with `GET /v1/topics`; the\n topic must belong to the same account. A clone of the import made in the same\n environment keeps `_topicId`.\n x-celigo-ai-guidance:\n - |-\n An id that is not a topic in the account fails the save with 422 `invalid_ref`\n (\"\u003cid> is not a valid topic._topicId\").\n examples: [\"6ab1f48d38d7733b4599875c\"]\n batchSize:\n type: integer\n minimum: 1\n default: 1000\n description: |-\n Number of records published per request to the topic. Lower it when individual\n messages are large; the default keeps publishing efficient for typical record sizes.\n x-celigo-ai-guidance:\n - |-\n Zero and negative values fail the save with 422 `min` (\"Path `topic.batchSize` (0) is\n less than minimum allowed value (1)\").\n examples: [1000, 100]\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":2881,"content_sha256":"7ec4b267b20da56c18874be97143b1fe2a40d9254f19a2500b81ff75598b12f1"},{"filename":"references/schemas/wrapper.yml","content":"Wrapper:\n type: object\n description: |-\n Configuration for Wrapper imports, which delegate writing records to custom connector code\n (typically a stack-hosted function) rather than a built-in adaptor. Required when the\n _connectionId field references a wrapper connection.\n x-celigo-ai-guidance:\n - |-\n Configuration for WrapperImport resources — imports backed by a user-supplied\n \"wrapper\" JavaScript module that invokes a custom function (typically to\n integrate with a system Celigo doesn't have a first-class connector for).\n - |-\n Wrapper imports use Mapper 1.0 for field mappings and the generic Celigo lookup shape:\n mappings live in `resource.mapping` (fields/lists) and lookups mirror at\n `resource.lookups` and `resource.wrapper.lookups`.\n # `function` is required on real configs only (see the if/then at the end of this\n # schema): the server writes stub `wrapper` objects ({lookups:[]} / {}) onto imports\n # of other adaptorTypes as bookkeeping.\n properties:\n function:\n type: string\n x-celigo-ui-override: >-\n Required by the wrapper import form (wrapper.function is required:true). Encoded\n to mirror\n the form so builders produce connectable configurations.\n description: |-\n Name of the function the wrapper invokes to process records.\n Must match a callable function in the wrapper's execution context; names are case-sensitive.\n x-celigo-ai-guidance:\n - |-\n Name of the function to invoke on the wrapper module.\n Must match an exported function on the connection's wrapper script.\n The wrapper receives the import's `configuration` object and the mapped\n record(s) and returns the response the import records downstream.\n examples: [\"importItem\", \"createFulfillmentsForPending\", \"sendMessage\"]\n configuration:\n type: object\n additionalProperties: true\n description: |-\n Free-form settings passed to the wrapper function at runtime (connector-specific keys such\n as method, apiVersion, headers, or handler). Structure is defined by the wrapper code, not\n by this schema.\n x-celigo-ai-guidance:\n - |-\n Opaque configuration object passed to the wrapper `function` at runtime.\n Shape is defined entirely by the wrapper author — Celigo does not interpret or\n validate these fields.\n Use for endpoint URLs, feature flags, batch-size hints,\n or any other settings the wrapper needs beyond the connection-level credentials.\n lookups:\n x-celigo-agent:\n - lookup\n $ref: ../common-schemas/lookups.yml#/Lookups\n description: |-\n Lookup definitions for this wrapper import. Wrapper lookups use\n the generic Celigo shape: either a static `map` (key→value\n object) or a dynamic `_lookupCacheId` + `extract` pair. See\n `common/schemas/lookups.yml` for the full item shape.\n x-celigo-ai-guidance:\n - |-\n ## Mirror invariant\n The Celigo platform persists lookups in TWO locations on a\n Wrapper import: top-level `lookups` and `wrapper.lookups`.\n Both arrays must hold byte-for-byte identical content, so write\n both on every save.\n # Requireds apply only to real configs, not server-written bookkeeping stubs whose\n # keys are all in {lookups}. Stack-style conditional (no allOf).\n if:\n not:\n propertyNames:\n enum: [lookups]\n then:\n required:\n - function\n","content_type":"application/yaml; charset=utf-8","language":"yaml","size":3502,"content_sha256":"03730a1e653b3f96c7fed4bc1f61989a81c10b73eb6dc24efbc62d18b6736b85"}],"content_json":{"type":"doc","content":[{"type":"paragraph","content":[{"text":"\u003c!-- TIER:1 -->","type":"text"}]},{"type":"heading","attrs":{"level":1},"content":[{"text":"Configuring Imports","type":"text"}]},{"type":"paragraph","content":[{"text":"An import is the ","type":"text"},{"text":"data destination","type":"text","marks":[{"type":"strong"}]},{"text":" in a Celigo integration. It takes records from an upstream step and writes them to an external system -- REST APIs, databases, ERPs, file servers, or AI models. Every import has one adaptor type and, apart from the connection-less kinds in ","type":"text"},{"text":"Minimum Required Fields","type":"text","marks":[{"type":"link","attrs":{"href":"#minimum-required-fields","title":null}}]},{"text":", is bound to exactly one connection.","type":"text"}]},{"type":"paragraph","content":[{"text":"Imports handle six concerns:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Field mapping","type":"text","marks":[{"type":"strong"}]},{"text":" -- transforming source fields into the destination system's expected format (including value resolution via static maps and lookup tables). Uses Mapper 2.0 (","type":"text"},{"text":"mappings[]","type":"text","marks":[{"type":"code_inline"}]},{"text":" array) by default, including ","type":"text"},{"text":"NetSuiteHTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":". ","type":"text"},{"text":"SalesforceImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"WrapperImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" support only Mapper 1.0 (","type":"text"},{"text":"mapping.fields[]","type":"text","marks":[{"type":"code_inline"}]},{"text":" / ","type":"text"},{"text":"mapping.lists[]","type":"text","marks":[{"type":"code_inline"}]},{"text":"), and ","type":"text"},{"text":"NetSuiteDistributedImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" uses its own Mapper 1.0 dialect at ","type":"text"},{"text":"netsuite_da.mapping","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Operation logic","type":"text","marks":[{"type":"strong"}]},{"text":" -- create, update, upsert, delete, attach/detach","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Hooks","type":"text","marks":[{"type":"strong"}]},{"text":" -- JavaScript pre/post processing at various pipeline stages (preMap, postMap, postSubmit). File-based imports that generate files from records also support postAggregate; an HTTP records import refuses it (422 ","type":"text"},{"text":"invalid_hook","type":"text","marks":[{"type":"code_inline"}]},{"text":")","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"One-to-many","type":"text","marks":[{"type":"strong"}]},{"text":" -- fan out a child array already on the input record. Set ","type":"text"},{"text":"oneToMany: true","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"pathToMany","type":"text","marks":[{"type":"code_inline"}]},{"text":" to that path (e.g., ","type":"text"},{"text":"\"lineItems\"","type":"text","marks":[{"type":"code_inline"}]},{"text":") when one source record should create one import operation per child. After fan-out the child is ","type":"text"},{"text":"record","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"{{record.sku}}","type":"text","marks":[{"type":"code_inline"}]},{"text":") and the parent (without the array) is ","type":"text"},{"text":"record._PARENT","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"{{record._PARENT.id}}","type":"text","marks":[{"type":"code_inline"}]},{"text":"). Leave ","type":"text"},{"text":"pathToMany","type":"text","marks":[{"type":"code_inline"}]},{"text":" blank when the arriving record is itself an array (grouped rows from a ","type":"text"},{"text":"groupByFields","type":"text","marks":[{"type":"code_inline"}]},{"text":" export or a file group) -- each element then becomes the record, with no ","type":"text"},{"text":"_PARENT","type":"text","marks":[{"type":"code_inline"}]},{"text":". A path on an array record processes zero children and still reports success; a blank path on an object record fails with ","type":"text"},{"text":"oneToMany_error","type":"text","marks":[{"type":"code_inline"}]},{"text":". ","type":"text"},{"text":"oneToMany","type":"text","marks":[{"type":"code_inline"}]},{"text":" doesn't batch by itself: templates see ","type":"text"},{"text":"batch_of_records","type":"text","marks":[{"type":"code_inline"}]},{"text":" only when ","type":"text"},{"text":"http.batchSize","type":"text","marks":[{"type":"code_inline"}]},{"text":" is above 1 (the default varies by API) or an RDBMS query runs ","type":"text"},{"text":"per_page","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Response mapping","type":"text","marks":[{"type":"strong"}]},{"text":" -- extract fields from the import's API response back into the record for downstream steps. Configured on the flow's ","type":"text"},{"text":"pageProcessors[]","type":"text","marks":[{"type":"code_inline"}]},{"text":" entry, but planned when building the import. Extracts read the response envelope: ","type":"text"},{"text":"id","type":"text","marks":[{"type":"code_inline"}]},{"text":" (the destination-assigned record ID), ","type":"text"},{"text":"statusCode","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"ignored","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"errors","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"_headers","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"dataURI","type":"text","marks":[{"type":"code_inline"}]},{"text":" (the record's link, when the import sets ","type":"text"},{"text":"dataURITemplate","type":"text","marks":[{"type":"code_inline"}]},{"text":"), and ","type":"text"},{"text":"_json","type":"text","marks":[{"type":"code_inline"}]},{"text":" (the response body after any response transform, whose top-level fields are also flattened onto the envelope). Use ","type":"text"},{"text":"id","type":"text","marks":[{"type":"code_inline"}]},{"text":" for a created record's ID and ","type":"text"},{"text":"_json.fieldName","type":"text","marks":[{"type":"code_inline"}]},{"text":" for other response fields (e.g., ","type":"text"},{"text":"_json.output.1.content.0.text","type":"text","marks":[{"type":"code_inline"}]},{"text":" for OpenAI responses) -- ","type":"text"},{"text":"_json.id","type":"text","marks":[{"type":"code_inline"}]},{"text":" works only when the body itself has a top-level ","type":"text"},{"text":"id","type":"text","marks":[{"type":"code_inline"}]},{"text":". Flow test runs never send imports, so ","type":"text"},{"text":"id","type":"text","marks":[{"type":"code_inline"}]},{"text":" is empty there. Response mapping uses Transformation 1.0 syntax (extract/generate pairs), not the newer expression-based transforms","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"postResponseMap hook","type":"text","marks":[{"type":"strong"}]},{"text":" -- JavaScript processing after response mapping merges the response back into the record. Configured on the flow's ","type":"text"},{"text":"pageProcessors[]","type":"text","marks":[{"type":"code_inline"}]},{"text":" entry, but planned when building the import. Use to transform or enrich the merged record before downstream steps","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Imports are used across flows, APIs, and tools.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Import Execution Pipeline","type":"text"}]},{"type":"paragraph","content":[{"text":"When records arrive at an import step, this pipeline executes in strict order:","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Input filter","type":"text","marks":[{"type":"strong"}]},{"text":" (optional) -- records that don't match skip this import (see ","type":"text"},{"text":"Gotchas","type":"text","marks":[{"type":"link","attrs":{"href":"#gotchas","title":null}}]},{"text":"). Configured as ","type":"text"},{"text":"filter","type":"text","marks":[{"type":"code_inline"}]},{"text":" on the import resource itself -- an ","type":"text"},{"text":"inputFilter","type":"text","marks":[{"type":"code_inline"}]},{"text":" key on an import is silently discarded. See ","type":"text"},{"text":"configuring-filters","type":"text","marks":[{"type":"link","attrs":{"href":"../configuring-filters/SKILL.md#import-filter-import-filter","title":null}}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"preMap hook","type":"text","marks":[{"type":"strong"}]},{"text":" (optional) -- JavaScript processing before field mapping","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Field mapping","type":"text","marks":[{"type":"strong"}]},{"text":" -- Mapper 2.0 or 1.0 maps source fields to destination fields, including lookups and hardcoded values","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"postMap hook","type":"text","marks":[{"type":"strong"}]},{"text":" (optional) -- JavaScript processing after field mapping, before submission","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Submit to destination","type":"text","marks":[{"type":"strong"}]},{"text":" -- writes the mapped record to the external system","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"postSubmit hook","type":"text","marks":[{"type":"strong"}]},{"text":" (optional) -- JavaScript processing after the destination responds (access response data, log results, trigger side effects)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Response transform","type":"text","marks":[{"type":"strong"}]},{"text":" (optional, ","type":"text"},{"text":"responseTransform","type":"text","marks":[{"type":"code_inline"}]},{"text":") -- reshapes the response body (","type":"text"},{"text":"_json","type":"text","marks":[{"type":"code_inline"}]},{"text":") after postSubmit. Its expression rules are Transformation 1.0 only (","type":"text"},{"text":"expression.version: \"1\"","type":"text","marks":[{"type":"code_inline"}]},{"text":"; ","type":"text"},{"text":"\"2\"","type":"text","marks":[{"type":"code_inline"}]},{"text":" is refused with 422), and it keeps only the fields it maps, so a field it drops is gone for response mapping. Its execution-log stage is ","type":"text"},{"text":"responseTransformation","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Response mapping","type":"text","marks":[{"type":"strong"}]},{"text":" (optional) -- carries data from the destination response back into the record for downstream steps. Configured on the flow's ","type":"text"},{"text":"pageProcessors[]","type":"text","marks":[{"type":"code_inline"}]},{"text":" entry, not on the import itself","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"postResponseMap hook","type":"text","marks":[{"type":"strong"}]},{"text":" (optional) -- JavaScript processing after response mapping merges response data back into the record","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Key distinction:","type":"text","marks":[{"type":"strong"}]},{"text":" Response mapping lives on the flow's ","type":"text"},{"text":"pageProcessors[]","type":"text","marks":[{"type":"code_inline"}]},{"text":" entry, not on the import resource. preMap, the mapping and postMap shape only the request this import sends: the next step receives the record as it arrived plus what response mapping and postResponseMap add, so a field added in steps 2-4 never reaches it. When building an import that needs to pass data downstream, plan the response mapping at flow design time.","type":"text"}]},{"type":"paragraph","content":[{"text":"What an HTTP import's templates see.","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"http.relativeURI","type":"text","marks":[{"type":"code_inline"}]},{"text":" and header values render against the record before mapping, and an undefined path fails the request: a field the record lacks in the URI -- or a ","type":"text"},{"text":"{{lookup.\u003cname>}}","type":"text","marks":[{"type":"code_inline"}]},{"text":" whose lookup found nothing -- fails the record before anything is sent, with ","type":"text"},{"text":"handlebars_template_parse_error","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"Failed to generate request url from template: … Details: \"\u003cfield>\" not defined in the model.","type":"text","marks":[{"type":"code_inline"}]},{"text":"). That is missing data, not a template bug, so a retry fails the same way; an update import that some records reach without the destination id needs a composite UPDATE/CREATE or ","type":"text"},{"text":"ignoreMissing","type":"text","marks":[{"type":"code_inline"}]},{"text":" (Common Errors). A field holding ","type":"text"},{"text":"\"\"","type":"text","marks":[{"type":"code_inline"}]},{"text":" is defined: it renders empty, and the request goes out without the id. A field holding null renders the text ","type":"text"},{"text":"null","type":"text","marks":[{"type":"code_inline"}]},{"text":" instead (the request goes to ","type":"text"},{"text":"…/null","type":"text","marks":[{"type":"code_inline"}]},{"text":"), and a field under an object that is null fails the record like a missing one (","type":"text"},{"text":"\"\u003cfield>\" not defined in null","type":"text","marks":[{"type":"code_inline"}]},{"text":") -- see writing-handlebars, gotcha 16. ","type":"text"},{"text":"http.body","type":"text","marks":[{"type":"code_inline"}]},{"text":" renders against the mapped record, where an undefined path renders empty and the record still succeeds (","type":"text"},{"text":"strictHandlebarEvaluation","type":"text","marks":[{"type":"code_inline"}]},{"text":" doesn't change that). Leave ","type":"text"},{"text":"http.body","type":"text","marks":[{"type":"code_inline"}]},{"text":" unset to send the mapped record; when you template it, use the mapped (destination) field names, and double braces for string values in a JSON body (gotcha 11).","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Categories of Import","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Record-Based Imports","type":"text"}]},{"type":"paragraph","content":[{"text":"Submit structured records to APIs, databases, or ERPs. The vast majority of imports.","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"NetSuiteDistributedImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- high-performance SuiteApp writes (add, update, addupdate, delete, attach, detach). The default for NetSuite when the Celigo SuiteApp or bundle is installed. ","type":"text"},{"text":"netsuite_da.restletVersion","type":"text","marks":[{"type":"code_inline"}]},{"text":" is fixed when the step is created, so author new steps on ","type":"text"},{"text":"suiteapp2.0","type":"text","marks":[{"type":"code_inline"}]},{"text":"; a hook's SuiteScript version must match the step's","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"NetSuiteHTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- NetSuite's REST APIs on a ","type":"text"},{"text":"netsuite","type":"text","marks":[{"type":"code_inline"}]},{"text":" connection (","type":"text"},{"text":"nsDomainType: \"suitetalk\"","type":"text","marks":[{"type":"code_inline"}]},{"text":": ","type":"text"},{"text":"POST /services/rest/record/v1/\u003ctype>","type":"text","marks":[{"type":"code_inline"}]},{"text":" to insert, ","type":"text"},{"text":"PATCH .../\u003ctype>/{id}","type":"text","marks":[{"type":"code_inline"}]},{"text":" to update; ","type":"text"},{"text":"\"restlet\"","type":"text","marks":[{"type":"code_inline"}]},{"text":" for a custom RESTlet). Use it when neither the SuiteApp nor the bundle is installed, or the user asks for REST","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"HTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- REST/GraphQL APIs (POST, PUT, PATCH, DELETE). Supports connector-assisted (","type":"text"},{"text":"formType: \"assistant\"","type":"text","marks":[{"type":"code_inline"}]},{"text":") and GraphQL (","type":"text"},{"text":"graph_ql","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"assistant_graphql","type":"text","marks":[{"type":"code_inline"}]},{"text":") modes; see ","type":"text"},{"text":"GraphQL imports","type":"text","marks":[{"type":"link","attrs":{"href":"#graphql-imports","title":null}}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"SalesforceImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- Salesforce writes through ","type":"text"},{"text":"salesforce.api","type":"text","marks":[{"type":"code_inline"}]},{"text":": ","type":"text"},{"text":"soap","type":"text","marks":[{"type":"code_inline"}]},{"text":" (the default), ","type":"text"},{"text":"rest","type":"text","marks":[{"type":"code_inline"}]},{"text":", or ","type":"text"},{"text":"compositerecord","type":"text","marks":[{"type":"code_inline"}]},{"text":" (Composite Graph, a tree of related records in one request); ","type":"text"},{"text":"metadata","type":"text","marks":[{"type":"code_inline"}]},{"text":" deploys configuration, not records. ","type":"text"},{"text":"salesforce.api","type":"text","marks":[{"type":"code_inline"}]},{"text":" has no Bulk value","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"RDBMSImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- SQL databases (Snowflake, PostgreSQL, MySQL, SQL Server, Oracle). ","type":"text"},{"text":"rdbms.queryType","type":"text","marks":[{"type":"code_inline"}]},{"text":" is a one-element array whose value fixes the companion field: ","type":"text"},{"text":"per_record","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"per_page","type":"text","marks":[{"type":"code_inline"}]},{"text":" (renders against ","type":"text"},{"text":"batch_of_records","type":"text","marks":[{"type":"code_inline"}]},{"text":") and ","type":"text"},{"text":"first_page","type":"text","marks":[{"type":"code_inline"}]},{"text":" (runs once, on the first page) take ","type":"text"},{"text":"rdbms.query","type":"text","marks":[{"type":"code_inline"}]},{"text":", an array of SQL strings; ","type":"text"},{"text":"bulk_insert","type":"text","marks":[{"type":"code_inline"}]},{"text":" takes ","type":"text"},{"text":"rdbms.bulkInsert","type":"text","marks":[{"type":"code_inline"}]},{"text":" (pure INSERT, with no upsert or ignore-existing logic); ","type":"text"},{"text":"bulk_load","type":"text","marks":[{"type":"code_inline"}]},{"text":" takes ","type":"text"},{"text":"rdbms.bulkLoad","type":"text","marks":[{"type":"code_inline"}]},{"text":" (a staged load that becomes a MERGE with ","type":"text"},{"text":"primaryKeys","type":"text","marks":[{"type":"code_inline"}]},{"text":", or runs your own SQL with ","type":"text"},{"text":"overrideMergeQuery","type":"text","marks":[{"type":"code_inline"}]},{"text":"). Prefer ","type":"text"},{"text":"bulk_load","type":"text","marks":[{"type":"code_inline"}]},{"text":" where the database supports it, then ","type":"text"},{"text":"bulk_insert","type":"text","marks":[{"type":"code_inline"}]},{"text":" for pure INSERTs, then ","type":"text"},{"text":"per_page","type":"text","marks":[{"type":"code_inline"}]},{"text":", and ","type":"text"},{"text":"per_record","type":"text","marks":[{"type":"code_inline"}]},{"text":" only when the logic can't be batched. The legacy uppercase ","type":"text"},{"text":"INSERT","type":"text","marks":[{"type":"code_inline"}]},{"text":" (and the retired ","type":"text"},{"text":"MERGE","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"COPY","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"BULK INSERT","type":"text","marks":[{"type":"code_inline"}]},{"text":") fail any save with 422 \"Import query types … are deprecated\"; ","type":"text"},{"text":"UPDATE","type":"text","marks":[{"type":"code_inline"}]},{"text":" is legacy too, so don't use it. Other strings, such as lowercase ","type":"text"},{"text":"insert","type":"text","marks":[{"type":"code_inline"}]},{"text":", fail as an invalid enum value. A ","type":"text"},{"text":"bulk_load","type":"text","marks":[{"type":"code_inline"}]},{"text":" import's GET also returns a ","type":"text"},{"text":"file","type":"text","marks":[{"type":"code_inline"}]},{"text":" block describing its staging file; the platform derives it on every read, so a save that changes it is accepted and discarded. The SQL itself: ","type":"text"},{"text":"writing-sql","type":"text","marks":[{"type":"link","attrs":{"href":"../writing-sql/SKILL.md","title":null}}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"MongodbImport","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"DynamodbImport","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"JDBCImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- other databases","type":"text"}]}]}]},{"type":"heading","attrs":{"level":4},"content":[{"text":"GraphQL imports","type":"text"}]},{"type":"paragraph","content":[{"text":"A GraphQL import is an ","type":"text"},{"text":"HTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" with ","type":"text"},{"text":"http.formType","type":"text","marks":[{"type":"code_inline"}]},{"text":" ","type":"text"},{"text":"graph_ql","type":"text","marks":[{"type":"code_inline"}]},{"text":" (or ","type":"text"},{"text":"assistant_graphql","type":"text","marks":[{"type":"code_inline"}]},{"text":", the connector-assisted GraphQL form). Each ","type":"text"},{"text":"http.body[]","type":"text","marks":[{"type":"code_inline"}]},{"text":" entry is a string holding the ","type":"text"},{"text":"{\"query\", \"operationName\", \"variables\"}","type":"text","marks":[{"type":"code_inline"}]},{"text":" envelope, one per operation on an update/create pair -- never an object, which is stored as-is and doesn't render -- and the mutation's input comes from the record through Handlebars in ","type":"text"},{"text":"variables","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Endpoint.","type":"text","marks":[{"type":"strong"}]},{"text":" The import posts to the connection's base URI -- its form has no relative-URI field and writes ","type":"text"},{"text":"relativeURI: [\"/\"]","type":"text","marks":[{"type":"code_inline"}]},{"text":" on every save -- so the connection must supply the GraphQL endpoint: a ","type":"text"},{"text":"graph_ql","type":"text","marks":[{"type":"code_inline"}]},{"text":" connection, or a connector connection bound to the app's GraphQL API. On a connection bound to the app's REST API every record fails (HTTP 406 on Shopify); add a second connection bound to the GraphQL API, re-point the GraphQL steps to it, and leave the shared REST connection unchanged. The form type is set on the step, so a plain ","type":"text"},{"text":"http","type":"text","marks":[{"type":"code_inline"}]},{"text":" connection whose base URI is the endpoint works too.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Variables.","type":"text","marks":[{"type":"strong"}]},{"text":" Under ","type":"text"},{"text":"graph_ql","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"assistant_graphql","type":"text","marks":[{"type":"code_inline"}]},{"text":", a ","type":"text"},{"text":"variables","type":"text","marks":[{"type":"code_inline"}]},{"text":" value stored as a JSON string is sent as an object; under ","type":"text"},{"text":"http","type":"text","marks":[{"type":"code_inline"}]},{"text":" it goes out as a string. An object takes expressions inside string values only: ","type":"text"},{"text":"\"variables\": {\"input\": {\"email\": \"{{record.email}}\"}}","type":"text","marks":[{"type":"code_inline"}]},{"text":". A string can also carry bare values, but its string values sit two JSON levels deep, so give each one ","type":"text"},{"text":"jsonEncode","type":"text","marks":[{"type":"code_inline"}]},{"text":" in double braces: ","type":"text"},{"text":"\"variables\": \"{\\\"input\\\": {\\\"note\\\": \\\"{{jsonEncode record.note}}\\\", \\\"quantity\\\": {{record.qty}} } }\"","type":"text","marks":[{"type":"code_inline"}]},{"text":". An object or array goes in as ","type":"text"},{"text":"{{jsonSerialize record.items}}","type":"text","marks":[{"type":"code_inline"}]},{"text":", also in double braces. With ","type":"text"},{"text":"{{record.note}}","type":"text","marks":[{"type":"code_inline"}]},{"text":", or triple braces on either helper, a value holding a quote fails its record with 422 ","type":"text"},{"text":"invalid_json","type":"text","marks":[{"type":"code_inline"}]},{"text":" before anything is sent.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Errors come back as HTTP 200.","type":"text","marks":[{"type":"strong"}]},{"text":" A GraphQL server usually answers a failed mutation with 200 and an ","type":"text"},{"text":"errors","type":"text","marks":[{"type":"code_inline"}]},{"text":" array, and with no response settings the import counts the record as written. Set ","type":"text"},{"text":"http.response.resourceIdPath","type":"text","marks":[{"type":"code_inline"}]},{"text":" to the id only a successful mutation returns (","type":"text"},{"text":"[\"data.customerCreate.customer.id\"]","type":"text","marks":[{"type":"code_inline"}]},{"text":"): any other answer then fails the record with ","type":"text"},{"text":"missing_id","type":"text","marks":[{"type":"code_inline"}]},{"text":", top-level ","type":"text"},{"text":"errors","type":"text","marks":[{"type":"code_inline"}]},{"text":" and the mutation's own ","type":"text"},{"text":"userErrors","type":"text","marks":[{"type":"code_inline"}]},{"text":" alike. For the server's message on top-level errors, also set ","type":"text"},{"text":"failPath: [\"errors[0].message\"]","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"errorPath: \"errors[0].message\"","type":"text","marks":[{"type":"code_inline"}]},{"text":"; ","type":"text"},{"text":"failPath: [\"errors\"]","type":"text","marks":[{"type":"code_inline"}]},{"text":" catches a response with exactly one error and lets two or more through.","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"File-Based Imports","type":"text"}]},{"type":"paragraph","content":[{"text":"Write or upload files to remote storage. Require the ","type":"text"},{"text":"file{}","type":"text","marks":[{"type":"code_inline"}]},{"text":" configuration block. Two modes:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Record-to-file","type":"text","marks":[{"type":"strong"}]},{"text":" -- aggregates incoming records into a file (CSV, JSON, XML, XLSX). The ","type":"text"},{"text":"file{}","type":"text","marks":[{"type":"code_inline"}]},{"text":" block defines the output format, and ","type":"text"},{"text":"file.aggregation.type","type":"text","marks":[{"type":"code_inline"}]},{"text":" how a run's records become files: ","type":"text"},{"text":"all","type":"text","marks":[{"type":"code_inline"}]},{"text":" (one file per run), ","type":"text"},{"text":"page","type":"text","marks":[{"type":"code_inline"}]},{"text":" (one per page) or ","type":"text"},{"text":"key","type":"text","marks":[{"type":"code_inline"}]},{"text":" (EDI file definitions only, with B2B Manager). Send the legacy ","type":"text"},{"text":"file.skipAggregation","type":"text","marks":[{"type":"code_inline"}]},{"text":" as its pair -- ","type":"text"},{"text":"true","type":"text","marks":[{"type":"code_inline"}]},{"text":" with ","type":"text"},{"text":"page","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"false","type":"text","marks":[{"type":"code_inline"}]},{"text":" with ","type":"text"},{"text":"all","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"key","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- because validation reads the value you send: an EDI file definition on ","type":"text"},{"text":"page","type":"text","marks":[{"type":"code_inline"}]},{"text":" without ","type":"text"},{"text":"true","type":"text","marks":[{"type":"code_inline"}]},{"text":", or a ","type":"text"},{"text":"file.batchSize","type":"text","marks":[{"type":"code_inline"}]},{"text":" without it, fails with 422 ","type":"text"},{"text":"not_supported","type":"text","marks":[{"type":"code_inline"}]},{"text":". The format is ","type":"text"},{"text":"file.type","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"csv","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"json","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"xml","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"xlsx","type":"text","marks":[{"type":"code_inline"}]},{"text":"), with its settings under that name: a CSV's ","type":"text"},{"text":"file.csv.includeHeader","type":"text","marks":[{"type":"code_inline"}]},{"text":" defaults to ","type":"text"},{"text":"true","type":"text","marks":[{"type":"code_inline"}]},{"text":", so set it ","type":"text"},{"text":"false","type":"text","marks":[{"type":"code_inline"}]},{"text":" for a file with no header row. The columns are the import's ","type":"text"},{"text":"mappings","type":"text","marks":[{"type":"code_inline"}]},{"text":" rows, in order, each row's ","type":"text"},{"text":"generate","type":"text","marks":[{"type":"code_inline"}]},{"text":" naming its column; there is no ","type":"text"},{"text":"file.columns","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Blob passthrough","type":"text","marks":[{"type":"strong"}]},{"text":" -- transfers a file as-is from an upstream blob export or blob lookup. Set ","type":"text"},{"text":"blob: true","type":"text","marks":[{"type":"code_inline"}]},{"text":", and ","type":"text"},{"text":"blobKeyPath","type":"text","marks":[{"type":"code_inline"}]},{"text":" to the record field holding the ","type":"text"},{"text":"blobKey","type":"text","marks":[{"type":"code_inline"}]},{"text":" that step put there (FTP and Google Drive imports fall back to the record's root ","type":"text"},{"text":"blobKey","type":"text","marks":[{"type":"code_inline"}]},{"text":" when it's unset). Base64 content in a record is record data, not a blobKey. AI agent, guardrail and tool imports reject both fields with 422 ","type":"text"},{"text":"not_supported","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Adaptor types:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"HTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" with ","type":"text"},{"text":"http.type: \"file\"","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- the folder mode of the cloud file providers (Google Drive, Box, Dropbox, Azure Blob Storage, Google Cloud Storage, Azure Data Lake Storage Gen2, Celigo Storage). Address the folder by path in ","type":"text"},{"text":"http.relativeURI","type":"text","marks":[{"type":"code_inline"}]},{"text":" (a folder, never an API endpoint) or by the provider's folder id in ","type":"text"},{"text":"file.directory","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"pathMode: \"directoryId\"","type":"text","marks":[{"type":"code_inline"}]},{"text":"), and name the file in ","type":"text"},{"text":"file.fileName","type":"text","marks":[{"type":"code_inline"}]},{"text":"; provider-specific location rules are in ","type":"text"},{"text":"file.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/file.yml","title":null}}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"FTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- CSV, XML, JSON, XLSX, EDI files to FTP/SFTP. The folder goes in ","type":"text"},{"text":"ftp.directoryPath","type":"text","marks":[{"type":"code_inline"}]},{"text":" and the file's name in ","type":"text"},{"text":"ftp.fileName","type":"text","marks":[{"type":"code_inline"}]},{"text":", which takes no directory separators (the UI stores the same name in ","type":"text"},{"text":"file.fileName","type":"text","marks":[{"type":"code_inline"}]},{"text":" too); a dated name is a Handlebars template (","type":"text"},{"text":"writing-handlebars > File paths and names","type":"text","marks":[{"type":"link","attrs":{"href":"../writing-handlebars/SKILL.md#file-paths-and-names","title":null}}]},{"text":")","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"S3Import","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- objects to Amazon S3","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"AS2Import","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- AS2 EDI file transmission","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"FileSystemImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- local/on-premise filesystem writes","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"AI Imports","type":"text"}]},{"type":"paragraph","content":[{"text":"Invoke AI models for classification, extraction, or safety checks. No ","type":"text"},{"text":"_connectionId","type":"text","marks":[{"type":"code_inline"}]},{"text":" required unless using BYOK.","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"AiAgentImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- LLM invocations (any provider in the ","type":"text"},{"text":"aiAgent.provider","type":"text","marks":[{"type":"code_inline"}]},{"text":" enum) with structured output, tool use, and reasoning","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"GuardrailImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- PII detection, content moderation, or custom AI-based validation","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Stack and Tool Imports","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"WrapperImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- custom connector code, typically a function on a stack (","type":"text"},{"text":"wrapper.function","type":"text","marks":[{"type":"code_inline"}]},{"text":")","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"ToolImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- runs the Celigo Tool named by ","type":"text"},{"text":"tool._toolId","type":"text","marks":[{"type":"code_inline"}]},{"text":". It needs no ","type":"text"},{"text":"_connectionId","type":"text","marks":[{"type":"code_inline"}]},{"text":"; instead it must bind each connection the tool's steps use in ","type":"text"},{"text":"tool.overrides.connections","type":"text","marks":[{"type":"code_inline"}]},{"text":" as ","type":"text"},{"text":"{_abstractId, _id}","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- set ","type":"text"},{"text":"_id","type":"text","marks":[{"type":"code_inline"}]},{"text":" equal to ","type":"text"},{"text":"_abstractId","type":"text","marks":[{"type":"code_inline"}]},{"text":" to keep the tool's own connection. A connection left unbound fails the run with ","type":"text"},{"text":"connection_not_overridden","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Topic Imports (Event Streams)","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"TopicImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- publishes each record as a message to an Event Streams topic, which topic listeners (","type":"text"},{"text":"TopicExport","type":"text","marks":[{"type":"code_inline"}]},{"text":") read. Set ","type":"text"},{"text":"topic._topicId","type":"text","marks":[{"type":"code_inline"}]},{"text":" to the topic's ","type":"text"},{"text":"_id","type":"text","marks":[{"type":"code_inline"}]},{"text":", never its name (","type":"text"},{"text":"celigo topics list","type":"text","marks":[{"type":"code_inline"}]},{"text":"; an unknown id fails the save with 422 ","type":"text"},{"text":"invalid_ref","type":"text","marks":[{"type":"code_inline"}]},{"text":"), and optionally ","type":"text"},{"text":"topic.batchSize","type":"text","marks":[{"type":"code_inline"}]},{"text":" (default 1000). There's no connection and no create/update logic: each record becomes one message, reshaped by the mapping if there is one; with one-to-many, each element is its own message carrying the parent under ","type":"text"},{"text":"_PARENT","type":"text","marks":[{"type":"code_inline"}]},{"text":". Register the topic on the flow's integration so its flows can bind to it (","type":"text"},{"text":"celigo integrations register-topics \u003cintegrationId> \u003ctopicId>","type":"text","marks":[{"type":"code_inline"}]},{"text":"); the account needs the Event Streams entitlement. Test runs publish nothing; an invoke publishes for real","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Import Matching (create / update / upsert)","type":"text"}]},{"type":"paragraph","content":[{"text":"The core decision on most record-based imports is the ","type":"text"},{"text":"operation","type":"text","marks":[{"type":"strong"}]},{"text":" -- what happens to each record at the destination. Create requires no match key; update and delete require one; upsert checks first and does whichever applies. Users describe the intent in business terms (\"look up the customer and update them\", \"match by email and upsert\", \"skip the ones that already exist\") that resolve into five behaviors:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Always create","type":"text","marks":[{"type":"strong"}]},{"text":" -- every record is submitted as new. No matching, no checks.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Create only if missing","type":"text","marks":[{"type":"strong"}]},{"text":" -- match-check first; submit as create if not found, skip silently if found.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Update only if exists","type":"text","marks":[{"type":"strong"}]},{"text":" -- match-check first; submit as update if found, skip silently if not.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Update only if exists, fail if missing","type":"text","marks":[{"type":"strong"}]},{"text":" -- strict variant; no-match records error out instead of skipping.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Upsert","type":"text","marks":[{"type":"strong"}]},{"text":" -- submit a create when no match is found, an update when matched. The catch-all, and the right default when the user is vague.","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"When matching applies, decide four things: the ","type":"text"},{"text":"matching behavior","type":"text","marks":[{"type":"strong"}]},{"text":", the ","type":"text"},{"text":"match key field(s)","type":"text","marks":[{"type":"strong"}]},{"text":" (","type":"text"},{"text":"email","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"external_id","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"customer.id","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- required for anything other than always-create), the ","type":"text"},{"text":"on-match action","type":"text","marks":[{"type":"strong"}]},{"text":" (update, skip, fail), and the ","type":"text"},{"text":"on-no-match action","type":"text","marks":[{"type":"strong"}]},{"text":" (create, skip, fail). A lookup on a cross-reference field (NetSuite ","type":"text"},{"text":"externalid","type":"text","marks":[{"type":"code_inline"}]},{"text":", or a custom field that holds the source id) finds only records whose field holds that id, so the same import must write the source id into the field when it creates them; without that mapping no later run finds them, and each run adds the record again.","type":"text"}]},{"type":"paragraph","content":[{"text":"How a destination implements matching is adaptor-specific -- there is no single \"matching mode\" field. A destination might expose: a native ","type":"text"},{"text":"upsert","type":"text","marks":[{"type":"strong"}]},{"text":" keyed off an external ID (Salesforce upsert, NetSuite upsert, RDBMS ","type":"text"},{"text":"ON CONFLICT","type":"text","marks":[{"type":"code_inline"}]},{"text":"); a distinct ","type":"text"},{"text":"addupdate","type":"text","marks":[{"type":"code_inline"}]},{"text":" operation that handles both paths in one call; an ","type":"text"},{"text":"ignoreExisting","type":"text","marks":[{"type":"code_inline"}]},{"text":" flag paired with a lookup that probes before writing (lock it when repeats can arrive concurrently -- see ","type":"text"},{"text":"Writing Exactly Once","type":"text","marks":[{"type":"link","attrs":{"href":"#writing-exactly-once","title":null}}]},{"text":"); two separate create and update endpoints with no upsert variant (see composite imports below); or a lookup endpoint plus separate create and update endpoints, where the lookup runs pre-write to drive the create-vs-update decision. Some destinations have no matching concept at all -- writing a CSV to FTP, sending an email, posting to a webhook -- so every record goes out as-is.","type":"text"}]},{"type":"paragraph","content":[{"text":"Prefer import-level matching over a separate lookup step.","type":"text","marks":[{"type":"strong"}]},{"text":" Imports natively support this pre-write check, so a single import that does the matching and the write together means fewer steps, fewer round-trips, and no glue logic to maintain. A standalone lookup earns its place only when the looked-up data has a consumer ","type":"text"},{"text":"beyond","type":"text","marks":[{"type":"em"}]},{"text":" the write -- a router branching on something other than \"does this exist\", an AI agent reasoning over the result, or multiple downstream steps reading different fields. If the only consumer is the destination call itself, the work belongs inside the import. It's also the only check that runs under the import's concurrency lock, which matters whenever the same record can arrive twice (","type":"text"},{"text":"Writing Exactly Once","type":"text","marks":[{"type":"link","attrs":{"href":"#writing-exactly-once","title":null}}]},{"text":"). An import's own lookups refuse an ambiguous match only when told to: an HTTP lookup that finds more than one match fails the record only with ","type":"text"},{"text":"allowFailures: false","type":"text","marks":[{"type":"code_inline"}]},{"text":", and otherwise sends the first match with no error (gotcha 14); a NetSuite DA lookup with ","type":"text"},{"text":"allowFailures: true","type":"text","marks":[{"type":"code_inline"}]},{"text":" also resolves to one of the matching records unless ","type":"text"},{"text":"useDefaultOnMultipleMatches","type":"text","marks":[{"type":"code_inline"}]},{"text":" sends it to the ","type":"text"},{"text":"default","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"netsuitedistributed.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/netsuitedistributed.yml","title":null}}]},{"text":"). So search on a key that identifies one record. When an update's key must come from a separate lookup that can match several records, guard it -- fail or skip on multiple matches, or check for exactly one (","type":"text"},{"text":"configuring-lookup-caches > Safe Lookup-to-Update","type":"text","marks":[{"type":"link","attrs":{"href":"../configuring-lookup-caches/SKILL.md#safe-lookup-to-update----never-write-off-an-unguarded-first-match","title":null}}]},{"text":").","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Composite (two-endpoint) imports","type":"text"}]},{"type":"paragraph","content":[{"text":"When an HTTP destination has no native upsert but exposes separate create and update endpoints, a ","type":"text"},{"text":"composite import","type":"text","marks":[{"type":"strong"}]},{"text":" pins both endpoints on a single import node, role-tagged create and update. The runtime picks per record via a match-key check -- the same way a native upsert would -- so the flow stays one step. Prefer this over two separate imports driven by an upstream lookup or router; reach for separate imports only when the create and update paths must diverge beyond endpoint selection (different mappings, different downstream consumers, or different hook chains).","type":"text"}]},{"type":"paragraph","content":[{"text":"The operations sit in positional arrays, one entry per operation: ","type":"text"},{"text":"http.requestType","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"[\"UPDATE\", \"CREATE\"]","type":"text","marks":[{"type":"code_inline"}]},{"text":"), ","type":"text"},{"text":"http.method","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"http.relativeURI","type":"text","marks":[{"type":"code_inline"}]},{"text":". Exactly one decider picks per record: ","type":"text"},{"text":"http.existingExtract","type":"text","marks":[{"type":"code_inline"}]},{"text":", a field the record already carries (non-empty means UPDATE, whose URI reads it as ","type":"text"},{"text":"{{{data.0.\u003cfield>}}}","type":"text","marks":[{"type":"code_inline"}]},{"text":"), or ","type":"text"},{"text":"http.existingLookupName","type":"text","marks":[{"type":"code_inline"}]},{"text":", a lookup in ","type":"text"},{"text":"http.lookups","type":"text","marks":[{"type":"code_inline"}]},{"text":" that probes the destination (the UPDATE URI reads ","type":"text"},{"text":"{{lookup.\u003cname>}}","type":"text","marks":[{"type":"code_inline"}]},{"text":"). Never add ","type":"text"},{"text":"ignoreExisting","type":"text","marks":[{"type":"code_inline"}]},{"text":" to a composite import, and give every ","type":"text"},{"text":"http.response.*","type":"text","marks":[{"type":"code_inline"}]},{"text":" array one entry per operation.","type":"text"}]},{"type":"paragraph","content":[{"text":"existingExtract","type":"text","marks":[{"type":"code_inline"}]},{"text":" must hold the destination's id (put there by an upstream step's response mapping, or kept as a cross-reference), never the source record's own id: every record carries one, so every record would go to UPDATE. With ","type":"text"},{"text":"existingLookupName","type":"text","marks":[{"type":"code_inline"}]},{"text":", a lookup that comes back empty sends the record to CREATE, with no error -- and so does one whose ","type":"text"},{"text":"extract","type":"text","marks":[{"type":"code_inline"}]},{"text":" resolves nothing, so a ","type":"text"},{"text":"$.","type":"text","marks":[{"type":"code_inline"}]},{"text":"-prefixed ","type":"text"},{"text":"extract","type":"text","marks":[{"type":"code_inline"}]},{"text":" sends every record to CREATE. A templated ","type":"text"},{"text":"http.body","type":"text","marks":[{"type":"code_inline"}]},{"text":" has one entry per operation too; left unset, both requests send the mapped record. A search-driven upsert against an API that finds contacts by email:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"json"},"content":[{"text":"\"http\": {\n \"requestType\": [\"UPDATE\", \"CREATE\"],\n \"method\": [\"PATCH\", \"POST\"],\n \"relativeURI\": [\"/contacts/{{lookup.findContact}}\", \"/contacts\"],\n \"existingLookupName\": \"findContact\",\n \"lookups\": [{\n \"name\": \"findContact\",\n \"method\": \"POST\",\n \"relativeURI\": \"/contacts/search\",\n \"body\": \"{\\\"email\\\": \\\"{{email}}\\\"}\",\n \"extract\": \"results[0].id\"\n }]\n}","type":"text"}]},{"type":"paragraph","content":[{"text":"The lookup sits in ","type":"text"},{"text":"http.lookups","type":"text","marks":[{"type":"code_inline"}]},{"text":" (a ","type":"text"},{"text":"lookups","type":"text","marks":[{"type":"code_inline"}]},{"text":" array sent only at the top level is dropped on save), its request template is in ","type":"text"},{"text":"body","type":"text","marks":[{"type":"code_inline"}]},{"text":", and the template names the record's fields at the top level (","type":"text"},{"text":"{{email}}","type":"text","marks":[{"type":"code_inline"}]},{"text":", never ","type":"text"},{"text":"{{record.email}}","type":"text","marks":[{"type":"code_inline"}]},{"text":", which renders empty, so every search misses and every record is created again; gotchas 10 and 11). Its ","type":"text"},{"text":"extract","type":"text","marks":[{"type":"code_inline"}]},{"text":" is the id itself, because that is what the UPDATE URI receives. An HTTP lookup's ","type":"text"},{"text":"extract","type":"text","marks":[{"type":"code_inline"}]},{"text":" is a path into the response written without ","type":"text"},{"text":"$.","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- ","type":"text"},{"text":"results[0].id","type":"text","marks":[{"type":"code_inline"}]},{"text":", or ","type":"text"},{"text":"ids","type":"text","marks":[{"type":"code_inline"}]},{"text":" for an array of ids -- unlike a Mapper 2.0 row's ","type":"text"},{"text":"extract","type":"text","marks":[{"type":"code_inline"}]},{"text":", which needs ","type":"text"},{"text":"$.","type":"text","marks":[{"type":"code_inline"}]},{"text":". A ","type":"text"},{"text":"$.","type":"text","marks":[{"type":"code_inline"}]},{"text":" prefix, ","type":"text"},{"text":"[*]","type":"text","marks":[{"type":"code_inline"}]},{"text":", or a dot path through an array (","type":"text"},{"text":"results.id","type":"text","marks":[{"type":"code_inline"}]},{"text":") resolves nothing, so the lookup always comes back empty.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Writing Exactly Once","type":"text"}]},{"type":"paragraph","content":[{"text":"Every source can hand an import the same record twice: a sender fires a webhook again, a failed or canceled run is re-run over the same delta window, an error is retried, a record that wrote this step but failed a later one comes back on the next run, or another flow writes the same destination. ","type":"text"},{"text":"A trace key never de-duplicates","type":"text","marks":[{"type":"strong"}]},{"text":" -- it correlates a record's errors and is the default lock key below; a record that arrives twice is written twice. Make the write itself idempotent, in this order of preference:","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Rank","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Pattern","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Why it holds under concurrency","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"1","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A key the destination enforces","type":"text","marks":[{"type":"strong"}]},{"text":" -- an upsert or insert on a primary key or unique constraint (","type":"text"},{"text":"ON CONFLICT","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"MERGE","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- see ","type":"text"},{"text":"writing-sql","type":"text","marks":[{"type":"link","attrs":{"href":"../writing-sql/SKILL.md","title":null}}]},{"text":"), a unique external ID, an API that rejects a duplicate reference","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The destination refuses the second copy","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"2","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"ignoreExisting: true","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" + the existence check on this import + ","type":"text","marks":[{"type":"strong"}]},{"text":"idLockTemplate","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" on the same business key.","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"ignoreExisting","type":"text","marks":[{"type":"code_inline"}]},{"text":" (like ","type":"text"},{"text":"ignoreMissing","type":"text","marks":[{"type":"code_inline"}]},{"text":") is a top-level import field, not part of ","type":"text"},{"text":"http","type":"text","marks":[{"type":"code_inline"}]},{"text":". The check is ","type":"text"},{"text":"http.ignoreLookupName","type":"text","marks":[{"type":"code_inline"}]},{"text":" (a lookup on the key) or ","type":"text"},{"text":"http.ignoreExtract","type":"text","marks":[{"type":"code_inline"}]},{"text":" (the record already carries the destination ID) -- exactly one -- or the adaptor's equivalent (NetSuite ","type":"text"},{"text":"internalIdLookup","type":"text","marks":[{"type":"code_inline"}]},{"text":", Salesforce ","type":"text"},{"text":"idLookup.whereClause","type":"text","marks":[{"type":"code_inline"}]},{"text":")","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The existence check runs inside the per-key lock","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"3","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A ledger step","type":"text","marks":[{"type":"strong"}]},{"text":" in front of a destination that can't be queried -- ","type":"text"},{"text":"configuring-lookup-caches > Deduplication Ledger","type":"text","marks":[{"type":"link","attrs":{"href":"../configuring-lookup-caches/SKILL.md#deduplication-ledger","title":null}}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The ledger's claim is itself rank 2","type":"text"}]}]}]}]},{"type":"paragraph","content":[{"text":"An insert on a unique key refuses a repeat as an error (Salesforce ","type":"text"},{"text":"DUPLICATE_VALUE","type":"text","marks":[{"type":"code_inline"}]},{"text":", a unique-constraint violation). That keeps a repeated record out, but it refuses a changed one too: when changes to existing records should arrive, upsert on that key.","type":"text"}]},{"type":"paragraph","content":[{"text":"A match that is really a search run before the write -- a lookup-driven composite create/update, an ","type":"text"},{"text":"addupdate","type":"text","marks":[{"type":"code_inline"}]},{"text":" found by search -- is a check-then-write, not rank 1, unless the destination also rejects the duplicate key. A natural id is unique only within its source: when two sources can produce the same one (two stores, two accounts), namespace the key per source (","type":"text"},{"text":"{{record.storeId}}-{{record.orderId}}","type":"text","marks":[{"type":"code_inline"}]},{"text":"), or one source's record collides with or overwrites the other's. Rank 2 is only as good as its lookup: a lookup that fails counts as \"not found\", so the record is written (gotcha 12).","type":"text"}]},{"type":"paragraph","content":[{"text":"The concurrency ID lock.","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"idLockTemplate","type":"text","marks":[{"type":"code_inline"}]},{"text":" renders a key per record; records that render the same key are never submitted at the same time, and different keys run in parallel.","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Default:","type":"text","marks":[{"type":"strong"}]},{"text":" the record's trace key (a source's ","type":"text"},{"text":"traceKeyTemplate","type":"text","marks":[{"type":"code_inline"}]},{"text":" arrives on the record as ","type":"text"},{"text":"io_custom_trace_key","type":"text","marks":[{"type":"code_inline"}]},{"text":"). Without a template, the trace key is the first top-level id field the record has -- ","type":"text"},{"text":"id","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"internalId","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"_id","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"internal_id","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"identifier","type":"text","marks":[{"type":"code_inline"}]},{"text":", in a few spellings each -- and a nested id is never used. A blank trace key means ","type":"text"},{"text":"no lock","type":"text","marks":[{"type":"strong"}]},{"text":", so set ","type":"text"},{"text":"idLockTemplate","type":"text","marks":[{"type":"code_inline"}]},{"text":" when the identity is nested or composite. A template replaces the default rather than adding to it.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Render time:","type":"text","marks":[{"type":"strong"}]},{"text":" from the record as it arrives, before this import's own lookups and mapping. A key built from ","type":"text"},{"text":"{{lookup.\u003cname>}}","type":"text","marks":[{"type":"code_inline"}]},{"text":" or a mapped field renders empty -- and an empty key is no lock at all. Key it on a field the record carries (","type":"text"},{"text":"{{{record.orderId}}}","type":"text","marks":[{"type":"code_inline"}]},{"text":"), or one an upstream lookup step added through response mapping.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Scope:","type":"text","marks":[{"type":"strong"}]},{"text":" this import across every page, every concurrent run, and every flow that uses it (","type":"text"},{"text":"celigo imports dependencies \u003cid>","type":"text","marks":[{"type":"code_inline"}]},{"text":") -- not the connection: two imports rendering the same key still run in parallel. So two imports that both create the same entity (an order flow that creates missing customers, and a separate customer sync) can each miss the other's write and create it twice: create the entity through one import, or schedule the flows so they don't overlap. Records within one page are sent one at a time anyway, so a one-page test never shows the race.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Coverage:","type":"text","marks":[{"type":"strong"}]},{"text":" the ","type":"text"},{"text":"ignoreExisting","type":"text","marks":[{"type":"code_inline"}]},{"text":" existence check runs inside the lock. Nothing documents the same for ","type":"text"},{"text":"ignoreMissing","type":"text","marks":[{"type":"code_inline"}]},{"text":", the composite create/update decision, or one-to-many children -- where those must write once, connection ","type":"text"},{"text":"concurrencyLevel: 1","type":"text","marks":[{"type":"code_inline"}]},{"text":" closes the check-then-write race (the import's pages run one at a time), at the cost of serializing everything on that connection.","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Quick Reference","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Adaptor Decision Matrix","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Your data goes to...","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Use adaptorType","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Category","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Read schema","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"REST or GraphQL API","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"HTTPImport","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Record-based","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"http.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/http.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"NetSuite, Celigo SuiteApp or bundle installed","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"NetSuiteDistributedImport","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Record-based","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"netsuitedistributed.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/netsuitedistributed.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"NetSuite, neither installed (or REST requested)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"NetSuiteHTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" + ","type":"text"},{"text":"nsDomainType","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Record-based","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"http.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/http.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Salesforce objects","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"SalesforceImport","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Record-based","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"salesforce.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/salesforce.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"SQL database (Snowflake, PostgreSQL, etc.)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"RDBMSImport","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Record-based","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"rdbms.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/rdbms.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"MongoDB","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"MongodbImport","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Record-based","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"mongodb.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/mongodb.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"DynamoDB","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"DynamodbImport","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Record-based","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"dynamodb.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/dynamodb.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"JDBC database (non-built-in)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"JDBCImport","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Record-based","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"jdbc.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/jdbc.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Files to cloud storage (Google Drive, Box, Dropbox, Azure Blob, Google Cloud Storage, Celigo Storage)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"HTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" with ","type":"text"},{"text":"http.type: \"file\"","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"File-based","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"http.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/http.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Files to FTP/SFTP","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"FTPImport","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"File-based","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"ftp.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/ftp.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Files to S3","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"S3Import","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"File-based","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"s3.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/s3.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"AS2 EDI transmission","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"AS2Import","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"File-based","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"as2.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/as2.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Local filesystem","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"FileSystemImport","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"File-based","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"filesystem.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/filesystem.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"An LLM (any provider in the ","type":"text"},{"text":"aiAgent.provider","type":"text","marks":[{"type":"code_inline"}]},{"text":" enum)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"AiAgentImport","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"AI","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"aiagent.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/aiagent.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"PII detection / content moderation","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"GuardrailImport","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"AI","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"guardrail.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/guardrail.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Celigo Tool","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"ToolImport","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Tool","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"tool.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/tool.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Custom connector code on a stack","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"WrapperImport","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Stack","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"wrapper.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/wrapper.yml","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"An Event Streams topic","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"TopicImport","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Topic","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"topic.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/topic.yml","title":null}}]}]}]}]}]},{"type":"paragraph","content":[{"text":"Raw HTTP is the fallback, not the default.","type":"text","marks":[{"type":"strong"}]},{"text":" Pick the most specific match, in order:","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Native adaptor","type":"text","marks":[{"type":"strong"}]},{"text":" -- if the application has its own row (NetSuite, Salesforce, databases, FTP/S3), use it. Do not build an ","type":"text"},{"text":"HTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" against that app's REST API.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Pre-built HTTP connector","type":"text","marks":[{"type":"strong"}]},{"text":" -- for any other REST/GraphQL app, check the 550+ connector catalog before writing HTTP config (see ","type":"text"},{"text":"Check for a pre-built connector","type":"text","marks":[{"type":"link","attrs":{"href":"#3-check-for-a-pre-built-connector","title":null}}]},{"text":"). The step is still an ","type":"text"},{"text":"HTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":", but it runs on a connector-backed connection and takes its endpoint config from the connector.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Manual HTTP","type":"text","marks":[{"type":"strong"}]},{"text":" -- hand-write the config from public API docs only when no connector exists or it doesn't cover the operation you need.","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"For NetSuite, run ","type":"text"},{"text":"celigo connections suiteapp-status \u003cconnectionId>","type":"text","marks":[{"type":"code_inline"}]},{"text":": ","type":"text"},{"text":"suiteapp.success","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"bundle.success","type":"text","marks":[{"type":"code_inline"}]},{"text":" true means ","type":"text"},{"text":"NetSuiteDistributedImport","type":"text","marks":[{"type":"code_inline"}]},{"text":"; both false, each with ","type":"text"},{"text":"statusCode","type":"text","marks":[{"type":"code_inline"}]},{"text":" 404 and an install ","type":"text"},{"text":"URL","type":"text","marks":[{"type":"code_inline"}]},{"text":", means neither is installed, so use ","type":"text"},{"text":"NetSuiteHTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":". A different ","type":"text"},{"text":"statusCode","type":"text","marks":[{"type":"code_inline"}]},{"text":" means the check itself failed, not that the components are missing: fix the connection and re-run it before choosing. ","type":"text"},{"text":"NetSuiteImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" is legacy -- don't use it unless asked.","type":"text"}]},{"type":"paragraph","content":[{"text":"adaptorType","type":"text","marks":[{"type":"code_inline"}]},{"text":" is ","type":"text"},{"text":"case-sensitive","type":"text","marks":[{"type":"strong"}]},{"text":": ","type":"text"},{"text":"NetSuiteDistributedImport","type":"text","marks":[{"type":"code_inline"}]},{"text":", not ","type":"text"},{"text":"netsuitedistributedimport","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Minimum Required Fields","type":"text"}]},{"type":"paragraph","content":[{"text":"Every import needs at minimum: ","type":"text"},{"text":"name","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"adaptorType","type":"text","marks":[{"type":"code_inline"}]},{"text":", and the adaptor config block (","type":"text"},{"text":"http{}","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"netsuite_da{}","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"salesforce{}","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"tool{}","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"topic{}","type":"text","marks":[{"type":"code_inline"}]},{"text":", etc.). ","type":"text"},{"text":"_connectionId","type":"text","marks":[{"type":"code_inline"}]},{"text":" is required except on the connection-less imports: ","type":"text"},{"text":"ToolImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" (connections are bound per tool step in ","type":"text"},{"text":"tool.overrides","type":"text","marks":[{"type":"code_inline"}]},{"text":"), ","type":"text"},{"text":"TopicImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" (must not carry one -- with a connection, the save is validated as that connection's adaptor and fails), and ","type":"text"},{"text":"AiAgentImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" / ","type":"text"},{"text":"GuardrailImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" unless they use a BYOK connection.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Which Schemas to Read","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Always: ","type":"text"},{"text":"request.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/request.yml","title":null}}]},{"text":" (base fields)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Plus: the adaptor-specific file from the decision matrix","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"If file-based: also ","type":"text"},{"text":"file.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/file.yml","title":null}}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"If cloning: ","type":"text"},{"text":"clone-request.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/clone-request.yml","title":null}}]},{"text":", ","type":"text"},{"text":"clone-response.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/clone-response.yml","title":null}}]}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Schema Index","type":"text"}]},{"type":"paragraph","content":[{"text":"All schemas are in ","type":"text"},{"text":"references/schemas/","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/","title":null}}]},{"text":":","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Base fields (all imports):","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"request.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/request.yml","title":null}}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Response shape:","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"response.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/response.yml","title":null}}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Adaptor-specific config:","type":"text","marks":[{"type":"strong"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"http.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/http.yml","title":null}}]},{"text":" -- HTTP/REST/GraphQL (methods, URIs, headers, response parsing, lookups, composite upsert via ","type":"text"},{"text":"existingExtract","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"existingLookupName","type":"text","marks":[{"type":"code_inline"}]},{"text":")","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"netsuitedistributed.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/netsuitedistributed.yml","title":null}}]},{"text":" -- NetSuite SuiteApp (operation, recordType, internalIdLookup, mapping, lookups)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"netsuite.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/netsuite.yml","title":null}}]},{"text":" -- NetSuite legacy","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"salesforce.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/salesforce.yml","title":null}}]},{"text":" -- Salesforce (sObjectType, operation, api, idLookup)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"rdbms.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/rdbms.yml","title":null}}]},{"text":" -- SQL databases (queryType, query, bulkInsert, bulkLoad)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"ftp.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/ftp.yml","title":null}}]},{"text":" -- FTP/SFTP","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"s3.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/s3.yml","title":null}}]},{"text":" -- Amazon S3","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"mongodb.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/mongodb.yml","title":null}}]},{"text":" -- MongoDB (method, collection, filter, upsert)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"dynamodb.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/dynamodb.yml","title":null}}]},{"text":" -- DynamoDB","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"jdbc.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/jdbc.yml","title":null}}]},{"text":" -- JDBC databases","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"as2.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/as2.yml","title":null}}]},{"text":" -- AS2 EDI","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"wrapper.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/wrapper.yml","title":null}}]},{"text":" -- custom stack connectors","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"tool.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/tool.yml","title":null}}]},{"text":" -- Tool imports (","type":"text"},{"text":"_toolId","type":"text","marks":[{"type":"code_inline"}]},{"text":", connection overrides)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"topic.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/topic.yml","title":null}}]},{"text":" -- Event Streams publisher (","type":"text"},{"text":"_topicId","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"batchSize","type":"text","marks":[{"type":"code_inline"}]},{"text":")","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"filesystem.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/filesystem.yml","title":null}}]},{"text":" -- local filesystem","type":"text"}]}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"AI config:","type":"text","marks":[{"type":"strong"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"aiagent.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/aiagent.yml","title":null}}]},{"text":" -- AI agent (provider, model, instructions, tools, structured output)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"guardrail.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/guardrail.yml","title":null}}]},{"text":" -- guardrails (PII, moderation, AI-agent validation)","type":"text"}]}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"File output:","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"file.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/file.yml","title":null}}]},{"text":" (CSV, XML, JSON, XLSX config for file-based imports)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Clone:","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"clone-request.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/clone-request.yml","title":null}}]},{"text":", ","type":"text"},{"text":"clone-response.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/clone-response.yml","title":null}}]}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Related Skills","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"configuring-connections > Quick Reference","type":"text","marks":[{"type":"link","attrs":{"href":"../configuring-connections/SKILL.md#quick-reference","title":null}}]},{"text":" -- connection types and auth for import destinations","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"managing-file-storage > Use stored files in a flow","type":"text","marks":[{"type":"link","attrs":{"href":"../managing-file-storage/SKILL.md#8-use-stored-files-in-a-flow","title":null}}]},{"text":" -- Celigo Storage folders as a file destination","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"writing-mappings > Mapper 2.0 Workflow","type":"text","marks":[{"type":"link","attrs":{"href":"../writing-mappings/SKILL.md#mapper-20-workflow","title":null}}]},{"text":" -- field mappings on imports","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"writing-scripts > Data Pipeline Hooks","type":"text","marks":[{"type":"link","attrs":{"href":"../writing-scripts/SKILL.md#data-pipeline-hooks","title":null}}]},{"text":" -- preMap, postMap, postSubmit, postAggregate hooks","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"writing-handlebars > Quick Reference","type":"text","marks":[{"type":"link","attrs":{"href":"../writing-handlebars/SKILL.md#quick-reference","title":null}}]},{"text":" -- dynamic values in URIs, HTTP bodies, SQL queries","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"building-flows > How to Build a Flow","type":"text","marks":[{"type":"link","attrs":{"href":"../building-flows/SKILL.md#how-to-build-a-flow","title":null}}]},{"text":" -- wiring imports into flow pipelines as page processors","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"configuring-topics > How to Build an Event Stream","type":"text","marks":[{"type":"link","attrs":{"href":"../configuring-topics/SKILL.md#how-to-build-an-event-stream","title":null}}]},{"text":" -- the topic a publisher writes to: registration, retention, messages","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"troubleshooting-flows > Diagnostic Workflow","type":"text","marks":[{"type":"link","attrs":{"href":"../troubleshooting-flows/SKILL.md#diagnostic-workflow","title":null}}]},{"text":" -- diagnosing import-related failures","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"\u003c!-- TIER:2 -->","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"How to Build an Import","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"1. Identify the target application","type":"text"}]},{"type":"paragraph","content":[{"text":"What system are you writing data to? This determines adaptor type, connection type, and configuration shape.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"2. Check for existing patterns","type":"text"}]},{"type":"paragraph","content":[{"text":"Before building from scratch, look at what already exists:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"# Search your account (fast, uses local index)\nceligo account search \"\u003ckeyword>\"\n\n# Show what an existing import uses (connection) and what uses it (flows)\nceligo account dependencies import \u003cid>\n\n# Find orphaned imports not referenced by any flow\nceligo account lint\n\n# Search marketplace templates\nceligo templates marketplace\n\n# Extract just imports from a template\nceligo templates preview \u003cid> --model Import\nceligo templates preview \u003cid> --summary","type":"text"}]},{"type":"paragraph","content":[{"text":"The account index auto-refreshes when stale (older than 15 minutes by default). Force a fresh snapshot with ","type":"text"},{"text":"celigo account snapshot","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"3. Check for a pre-built connector","type":"text"}]},{"type":"paragraph","content":[{"text":"Always run this check before writing any HTTP config.","type":"text","marks":[{"type":"strong"}]},{"text":" Celigo maintains 550+ HTTP connectors with pre-configured auth, endpoints, and resources. Hand-write a manual ","type":"text"},{"text":"HTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" from public API docs only when this search comes up empty or the connector doesn't cover the operation you need.","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"# Search HTTP connectors\nceligo http-connectors list | grep -i \"\u003capplication-name>\"\nceligo http-connectors get \u003cid> --full # see endpoints, resources, auth config\n\n# Drill into the endpoints the connector defines for imports\nceligo http-connectors catalog \u003cid> --resource-type import --published-only\nceligo http-connectors endpoint-detail \u003cid> --resource-type import --resource-id \u003crid> --endpoint-id \u003cepid>\nceligo http-connectors endpoint-detail \u003cid> --resource-type import --resource-id \u003crid> --endpoint-ids \u003ccreateId>,\u003cupdateId> # a create/update pair\n\n# Search trading partner connectors (EDI, AS2)\nceligo tp-connectors list","type":"text"}]},{"type":"paragraph","content":[{"text":"If a connector exists, create the connection from it (","type":"text"},{"text":"http._httpConnectorId","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- see ","type":"text"},{"text":"configuring-connections > Check for a pre-built connector and global iClient","type":"text","marks":[{"type":"link","attrs":{"href":"../configuring-connections/SKILL.md#4-check-for-a-pre-built-connector-and-global-iclient","title":null}}]},{"text":") and take the import's ","type":"text"},{"text":"relativeURI","type":"text","marks":[{"type":"code_inline"}]},{"text":", method, and body/response shapes from the connector's endpoint metadata rather than reconstructing them from public API docs. Write each operation's ","type":"text"},{"text":"relativeURI","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"method","type":"text","marks":[{"type":"code_inline"}]},{"text":" exactly as the endpoint defines them: saving links the import to those endpoints by setting ","type":"text"},{"text":"http._httpConnectorVersionId","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"http._httpConnectorResourceId","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"http._httpConnectorEndpointIds[]","type":"text","marks":[{"type":"code_inline"}]},{"text":" (one per operation; imports drop the singular ","type":"text"},{"text":"_httpConnectorEndpointId","type":"text","marks":[{"type":"code_inline"}]},{"text":"), but pointers you send for the same connector are kept as sent. Read the import back: missing pointers mean a URI matched no endpoint. When retargeting an import, update or drop the old pointers in the same save -- a GET-modify-PUT that changes only the URI leaves the import naming the old endpoint.","type":"text"}]},{"type":"paragraph","content":[{"text":"endpoint-detail","type":"text","marks":[{"type":"code_inline"}]},{"text":" on the import plane returns the resource, whose ","type":"text"},{"text":"resourceFields","type":"text","marks":[{"type":"code_inline"}]},{"text":" catalog the record's fields, and each endpoint's ","type":"text"},{"text":"supportedBy","type":"text","marks":[{"type":"code_inline"}]},{"text":": what the connector's own form applies. ","type":"text"},{"text":"preConfiguredFields","type":"text","marks":[{"type":"code_inline"}]},{"text":" are values it sets, ","type":"text"},{"text":"fieldsToUnset","type":"text","marks":[{"type":"code_inline"}]},{"text":" are fields it clears, and ","type":"text"},{"text":"fieldsUserMustSet","type":"text","marks":[{"type":"code_inline"}]},{"text":" is what it asks for -- on Shopify's create-customer endpoint, ","type":"text"},{"text":"ignoreExisting","type":"text","marks":[{"type":"code_inline"}]},{"text":" and the mapping targets ","type":"text"},{"text":"mapping.fields.generate.customer.email","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"…first_name","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"…last_name","type":"text","marks":[{"type":"code_inline"}]},{"text":". The paths are the form's, not the import's. On create and update endpoints, ","type":"text"},{"text":"lookupToIdentifyExisting","type":"text","marks":[{"type":"code_inline"}]},{"text":" is the existence lookup to build (the endpoint to call and the ","type":"text"},{"text":"extract","type":"text","marks":[{"type":"code_inline"}]},{"text":" that reads the existing id, such as ","type":"text"},{"text":"customers[0].id","type":"text","marks":[{"type":"code_inline"}]},{"text":"), and ","type":"text"},{"text":"pathParameterToIdentifyExisting","type":"text","marks":[{"type":"code_inline"}]},{"text":" names the update URI's id parameter.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"4. Query metadata for the target system","type":"text"}]},{"type":"paragraph","content":[{"text":"For NetSuite, Salesforce, and RDBMS connections, discover available record types and fields:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"celigo metadata types \u003cconnectionId> # List record types / sObjects / tables\nceligo metadata fields \u003cconnectionId> \u003ctype> # List fields for an entity type\nceligo metadata fields \u003cconnectionId> \u003ctype> --refresh # Bypass the cache after a field is created in the app","type":"text"}]},{"type":"paragraph","content":[{"text":"Metadata is cached on the server, so a field or record type created since the last read stays missing until you re-read with ","type":"text"},{"text":"--refresh","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"NetSuite:","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"metadata fields","type":"text","marks":[{"type":"code_inline"}]},{"text":" returns field IDs, types, and groups. On ","type":"text"},{"text":"NetSuiteDistributedImport","type":"text","marks":[{"type":"code_inline"}]},{"text":", use the IDs in ","type":"text"},{"text":"netsuite_da.mapping.fields[].generate","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"netsuite_da.mapping.lists[].fields[].generate","type":"text","marks":[{"type":"code_inline"}]},{"text":"; sublist names (groups such as ","type":"text"},{"text":"\"item\"","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"\"addressbook\"","type":"text","marks":[{"type":"code_inline"}]},{"text":") go in ","type":"text"},{"text":"netsuite_da.mapping.lists[].generate","type":"text","marks":[{"type":"code_inline"}]},{"text":". ","type":"text"},{"text":"NetSuiteHTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" maps with Mapper 2.0 ","type":"text"},{"text":"mappings[]","type":"text","marks":[{"type":"code_inline"}]},{"text":". Lookup field IDs here are the ","type":"text"},{"text":"searchField","type":"text","marks":[{"type":"code_inline"}]},{"text":"/","type":"text"},{"text":"resultField","type":"text","marks":[{"type":"code_inline"}]},{"text":" values for ","type":"text"},{"text":"netsuite_da.lookups[]","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Salesforce:","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"metadata fields","type":"text","marks":[{"type":"code_inline"}]},{"text":" returns field API names, types, and relationship info. Use field API names for ","type":"text"},{"text":"salesforce.sObjectType","type":"text","marks":[{"type":"code_inline"}]},{"text":" lookups and for discovering which fields are createable/updateable.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"RDBMS:","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"metadata fields","type":"text","marks":[{"type":"code_inline"}]},{"text":" returns column names and types for a table — use these to write SQL queries (see ","type":"text"},{"text":"writing-sql","type":"text","marks":[{"type":"code_inline"}]},{"text":") and verify column names before building ","type":"text"},{"text":"bulkInsert.tableName","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"bulkLoad.tableName","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"5. Determine the category","type":"text"}]},{"type":"paragraph","content":[{"text":"Is this a ","type":"text"},{"text":"record-based import","type":"text","marks":[{"type":"strong"}]},{"text":" (submit records to an API/database/ERP), a ","type":"text"},{"text":"file-based import","type":"text","marks":[{"type":"strong"}]},{"text":" (write files to storage), or an ","type":"text"},{"text":"AI import","type":"text","marks":[{"type":"strong"}]},{"text":" (invoke a model)?","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"6. Choose the right adaptor type","type":"text"}]},{"type":"paragraph","content":[{"text":"Refer to the ","type":"text"},{"text":"Adaptor Decision Matrix","type":"text","marks":[{"type":"link","attrs":{"href":"#adaptor-decision-matrix","title":null}}]},{"text":" in the Quick Reference above. For NetSuite, ","type":"text"},{"text":"Diagnosing NetSuite Steps","type":"text","marks":[{"type":"link","attrs":{"href":"../configuring-exports/references/netsuite.md","title":null}}]},{"text":" has the account-side checks (the SuiteApp and bundle, the account's concurrency limit) and the errors NetSuite steps return.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"7. Build the import JSON","type":"text"}]},{"type":"paragraph","content":[{"text":"Reference the ","type":"text"},{"text":"Schema Index","type":"text","marks":[{"type":"link","attrs":{"href":"#schema-index","title":null}}]},{"text":" for the exact fields needed. Use the ","type":"text"},{"text":"Which Schemas to Read","type":"text","marks":[{"type":"link","attrs":{"href":"#which-schemas-to-read","title":null}}]},{"text":" decision rule to determine which files to consult.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"File Uploads over HTTP (multipart/form-data)","type":"text"}]},{"type":"paragraph","content":[{"text":"Some destination APIs accept files only as ","type":"text"},{"text":"multipart/form-data","type":"text","marks":[{"type":"code_inline"}]},{"text":" POSTs (Jira attachments, QuickBooks attachables, OpenAI file uploads). This is an ","type":"text"},{"text":"HTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" with ","type":"text"},{"text":"blob: true","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"blobKeyPath","type":"text","marks":[{"type":"code_inline"}]},{"text":", posting to the API's upload endpoint, and it works nothing like a JSON record write. Do ","type":"text"},{"text":"not","type":"text","marks":[{"type":"strong"}]},{"text":" set ","type":"text"},{"text":"http.type: \"file\"","type":"text","marks":[{"type":"code_inline"}]},{"text":": that is the folder mode of file-storage connectors (Google Drive, Box, Dropbox, Azure Blob), which treats ","type":"text"},{"text":"relativeURI","type":"text","marks":[{"type":"code_inline"}]},{"text":" as a folder path -- combined with an API endpoint URL it saves cleanly and fails every record with \"Directory is not found.\" Four pieces have to line up:","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Where the bytes come from.","type":"text","marks":[{"type":"strong"}]},{"text":" The import never carries the file itself. A preceding blob export or blob lookup pulls the file into blob storage, and that step's response mapping puts the reference onto the record -- the idiom is ","type":"text"},{"text":"{\"extract\": \"data.0.blobKey\", \"generate\": \"blobKey\"}","type":"text","marks":[{"type":"code_inline"}]},{"text":". The record then carries a ","type":"text"},{"text":"blobKey","type":"text","marks":[{"type":"code_inline"}]},{"text":" (a storage pointer, not content).","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"The media type.","type":"text","marks":[{"type":"strong"}]},{"text":" The connection's media type (or the import's request media type override) is ","type":"text"},{"text":"multipart/form-data","type":"text","marks":[{"type":"code_inline"}]},{"text":"; success/error response media types usually override to JSON so replies parse normally.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"The request body.","type":"text","marks":[{"type":"strong"}]},{"text":" Not raw MIME and not a normal handlebars payload -- a JSON array of parts, each ","type":"text"},{"text":"{name, value, type}","type":"text","marks":[{"type":"code_inline"}]},{"text":" plus optional ","type":"text"},{"text":"filename","type":"text","marks":[{"type":"code_inline"}]},{"text":" (include the extension) and ","type":"text"},{"text":"mime-headers","type":"text","marks":[{"type":"code_inline"}]},{"text":". The file part is ","type":"text"},{"text":"\"type\": \"attachment\"","type":"text","marks":[{"type":"code_inline"}]},{"text":" with ","type":"text"},{"text":"\"value\": \"{{blob}}\"","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- ","type":"text"},{"text":"{{blob}}","type":"text","marks":[{"type":"code_inline"}]},{"text":" is the only accepted value for an attachment (anything else 422s). Fields the API wants alongside the file ride as ","type":"text"},{"text":"\"type\": \"inline\"","type":"text","marks":[{"type":"code_inline"}]},{"text":" parts; an inline part whose value is a JSON object must be serialized. One file reference per import.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"blob: true","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" + ","type":"text","marks":[{"type":"strong"}]},{"text":"blobKeyPath","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" (Advanced settings) -- ","type":"text"},{"text":"blobKeyPath","type":"text","marks":[{"type":"code_inline"}]},{"text":" is the JSON path in the record where the blobKey lives (","type":"text"},{"text":"blobKey","type":"text","marks":[{"type":"code_inline"}]},{"text":", or ","type":"text"},{"text":"file.blobKey","type":"text","marks":[{"type":"code_inline"}]},{"text":" if nested). At send time the platform follows it into blob storage and streams the real bytes into the attachment part.","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"The platform assembles the final MIME body itself -- it generates the ","type":"text"},{"text":"boundary","type":"text","marks":[{"type":"code_inline"}]},{"text":" (never hardcode one), writes each part's ","type":"text"},{"text":"Content-Disposition","type":"text","marks":[{"type":"code_inline"}]},{"text":", and substitutes the attachment part with the raw file bytes. The parts array is a build recipe, not the payload.","type":"text"}]},{"type":"paragraph","content":[{"text":"Not every multipart API is form-data.","type":"text","marks":[{"type":"strong"}]},{"text":" Some upload endpoints expect ","type":"text"},{"text":"multipart/related","type":"text","marks":[{"type":"code_inline"}]},{"text":" instead (e.g. Google Drive's ","type":"text"},{"text":"/upload/drive/v3/files?uploadType=multipart","type":"text","marks":[{"type":"code_inline"}]},{"text":"), and the parts-array machinery does NOT apply. There the request body is the literal MIME document: explicit boundary, a JSON metadata part, and a content part referencing ","type":"text"},{"text":"{{blob}}","type":"text","marks":[{"type":"code_inline"}]},{"text":" (double braces). Check which flavor the destination API documents before building -- mixing them produces an import that saves cleanly and fails at runtime.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Async Destinations (submit, poll, confirm)","type":"text"}]},{"type":"paragraph","content":[{"text":"Some destination APIs only acknowledge a write (an HTTP 202, a job ticket) and finish it in the background -- bulk loads, file ingestion, document conversion. Attach an ","type":"text"},{"text":"async helper","type":"text","marks":[{"type":"strong"}]},{"text":" to the import (","type":"text"},{"text":"http._asyncHelperId","type":"text","marks":[{"type":"code_inline"}]},{"text":") so the step submits, polls a status export until the external work completes, and only then resolves. The mechanics and constraints (a status export with done/error value lists and poll intervals; no transform, output filter, or hook on the async-configured step; helpers cannot nest) are identical to the export side -- see ","type":"text"},{"text":"configuring-exports > Async APIs (submit, poll, fetch)","type":"text","marks":[{"type":"link","attrs":{"href":"../configuring-exports/SKILL.md#async-apis-submit-poll-fetch","title":null}}]},{"text":". Only add one when the API genuinely cannot confirm the write synchronously.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Testing an Import Without Writing","type":"text"}]},{"type":"paragraph","content":[{"text":"Most ways of trying an import send the request to the destination for real. Know which is which before you point one at a production system:","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"What you run","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Reaches the destination?","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"celigo imports invoke \u003cid>","type":"text","marks":[{"type":"code_inline"}]},{"text":" with records on stdin","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Yes -- a real write.","type":"text","marks":[{"type":"strong"}]},{"text":" One result per record, every adaptor; no job is created","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"celigo imports invoke","type":"text","marks":[{"type":"code_inline"}]},{"text":" with an import document on stdin (the preview route)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Yes -- a real write","type":"text","marks":[{"type":"strong"}]},{"text":", despite the name. Only the first record of ","type":"text"},{"text":"data","type":"text","marks":[{"type":"code_inline"}]},{"text":" runs, so try one record per call. NetSuite, Salesforce, RDBMS, and FTP imports are rejected with 422 ","type":"text"},{"text":"invalid_adaptor","type":"text","marks":[{"type":"code_inline"}]},{"text":"; a bare document piped without ","type":"text"},{"text":"data","type":"text","marks":[{"type":"code_inline"}]},{"text":" runs against one empty record","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"celigo imports invoke [\u003cid>] --connection \u003cconnectionId>","type":"text","marks":[{"type":"code_inline"}]},{"text":" (a virtual import)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Yes -- a real write.","type":"text","marks":[{"type":"strong"}]},{"text":" Runs the saved import, or a piped ","type":"text"},{"text":"{\"import\": {...}, \"data\": [...]}","type":"text","marks":[{"type":"code_inline"}]},{"text":" document, through that connection, NetSuite and RDBMS imports included; one result per record","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"celigo flows test-run \u003cflowId> --export \u003cexportId>","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"No.","type":"text","marks":[{"type":"strong"}]},{"text":" Imports never execute in a test run -- each returns its saved ","type":"text"},{"text":"mockResponse","type":"text","marks":[{"type":"code_inline"}]},{"text":". A NetSuite DA import is the exception: the page still goes to NetSuite, where the SuiteApp or bundle runs the lookups and SuiteScript hooks and builds each record without saving it. Its stages are ","type":"text"},{"text":"mapping","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"lookup","type":"text","marks":[{"type":"code_inline"}]},{"text":" (plus one per SuiteScript hook), not ","type":"text"},{"text":"request","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"response","type":"text","marks":[{"type":"code_inline"}]},{"text":", and an error NetSuite raises while building a record is real and counts on the job","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A real flow run, or ","type":"text"},{"text":"celigo flows retry-errors","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Yes","type":"text"}]}]}]}]},{"type":"paragraph","content":[{"text":"Invoking by id or as a virtual import runs the step with the import's own settings only: hooks get no integration or flow settings, and ","type":"text"},{"text":"{{settings.integration.*}}","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"{{settings.flow.*}}","type":"text","marks":[{"type":"code_inline"}]},{"text":" render empty. To run it as a step of its flow, tool or API, add ","type":"text"},{"text":"--integration \u003cid>","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"--flow","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"--tool","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"--api \u003cid>","type":"text","marks":[{"type":"code_inline"}]},{"text":" to the ","type":"text"},{"text":"--connection","type":"text","marks":[{"type":"code_inline"}]},{"text":" form.","type":"text"}]},{"type":"paragraph","content":[{"text":"In the flow builder's import editor, ","type":"text"},{"text":"Preview","type":"text","marks":[{"type":"strong"}]},{"text":" builds the request without sending it (HTTP imports only), while ","type":"text"},{"text":"Send","type":"text","marks":[{"type":"strong"}]},{"text":" is a real write. No CLI command offers the compose-only preview. From the CLI, test-run the flow and read the import's stages with ","type":"text"},{"text":"celigo flows test-run-step-results","type":"text","marks":[{"type":"code_inline"}]},{"text":": the ","type":"text"},{"text":"mapping","type":"text","marks":[{"type":"code_inline"}]},{"text":" stage's output is the record after mapping, while the ","type":"text"},{"text":"request","type":"text","marks":[{"type":"code_inline"}]},{"text":" stage shows the body template unrendered, and its input is the record before mapping. Database imports have no compose-only surface at all; the SQL editor's evaluation is their only preview.","type":"text"}]},{"type":"paragraph","content":[{"text":"Save a realistic ","type":"text","marks":[{"type":"strong"}]},{"text":"mockResponse","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":".","type":"text","marks":[{"type":"strong"}]},{"text":" In test runs and flow-builder previews, an import with no saved mock answers with the placeholder ","type":"text"},{"text":"{\"id\": \"1234567890\", \"mockResponse\": \"replace me\"}","type":"text","marks":[{"type":"code_inline"}]},{"text":" (numbered per record on a page: ","type":"text"},{"text":"1234567890-1","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"1234567890-2","type":"text","marks":[{"type":"code_inline"}]},{"text":"), so response mappings extract nothing real and downstream steps see empty data. Save an array of records shaped like the destination's answer -- ","type":"text"},{"text":"statusCode","type":"text","marks":[{"type":"code_inline"}]},{"text":", the created record's ","type":"text"},{"text":"id","type":"text","marks":[{"type":"code_inline"}]},{"text":", and the response body under ","type":"text"},{"text":"_json","type":"text","marks":[{"type":"code_inline"}]},{"text":":","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"celigo imports set \u003cimportId> 'mockResponse=[{\"statusCode\": 200, \"id\": \"SO-1001\", \"_json\": {\"id\": \"SO-1001\", \"status\": \"created\"}}]'","type":"text"}]},{"type":"paragraph","content":[{"text":"A saved ","type":"text"},{"text":"mockResponse","type":"text","marks":[{"type":"code_inline"}]},{"text":" substitutes only in test runs and flow-builder previews -- ","type":"text"},{"text":"imports invoke","type":"text","marks":[{"type":"code_inline"}]},{"text":", real runs, and retries always send. ","type":"text"},{"text":"sampleResponseData","type":"text","marks":[{"type":"code_inline"}]},{"text":" only seeds the response-mapping editor; test runs and previews never use it.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"CLI Commands","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"# CRUD\nceligo imports list\nceligo imports get \u003cid>\nceligo imports create \u003c import.json\nceligo imports update \u003cid> \u003c import.json\nceligo imports set \u003cid> key=value [key2=value2 ...]\nceligo imports delete \u003cid> [-y]\nceligo imports audit \u003cid> # change history\nceligo imports dependencies \u003cid> # the flows, APIs and tools that use it\n\n# Invoke -- a real write: each record is sent to the destination (no job is created).\n# HTTP 200 even when records fail; check each result's statusCode and errors\necho '[{\"name\":\"test\"}]' | celigo imports invoke \u003cid>\n# Virtual import: an unsaved or modified document through a connection (also a real write)\necho '{\"import\": {...}, \"data\": [{\"name\":\"test\"}]}' | celigo imports invoke --connection \u003cconnectionId>\n\n# Mock response for test runs and flow-builder previews (never used by invoke or real runs)\nceligo imports set \u003cid> mockResponse=file://./mock-response.json\n\n# Clone and connection management\necho '{\"connectionMap\":{\"oldConnId\":\"newConnId\"}}' | celigo imports clone \u003cid>\nceligo imports replace-connection \u003cid> \u003cnewConnectionId>\n\n# Discovery\nceligo templates marketplace\nceligo http-connectors list\nceligo http-connectors catalog \u003cid> --resource-type import --published-only\nceligo http-connectors endpoint-detail \u003cid> --resource-type import --resource-id \u003crid> --endpoint-id \u003cepid>\nceligo tp-connectors list\nceligo metadata types \u003cconnectionId>\nceligo metadata fields \u003cconnectionId> \u003centityType>\n\n# Debug -- a window of at most 60 minutes (debugUntil, set by PATCH; a PUT doesn't set it).\n# It records each request the import sends while open: not earlier ones, and nothing from test runs\nceligo imports enable-debug \u003cid> [--duration \u003cminutes>]\nceligo imports disable-debug \u003cid>\nceligo flows debug-requests \u003cflowId> \u003cimportId> # read them from the consumer (or apis / tools debug-requests)","type":"text"}]},{"type":"paragraph","content":[{"text":"\u003c!-- TIER:3 -->","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Pre-Submit Checklist","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Required (all imports)","type":"text"}]},{"type":"checkbox_list","attrs":{"id":null},"content":[{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"name","type":"text","marks":[{"type":"code_inline"}]},{"text":" is set","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"adaptorType","type":"text","marks":[{"type":"code_inline"}]},{"text":" exact case matches connection type (request.yml > adaptorType)","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"_connectionId","type":"text","marks":[{"type":"code_inline"}]},{"text":" references a valid, online connection (skip for ","type":"text"},{"text":"ToolImport","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"TopicImport","type":"text","marks":[{"type":"code_inline"}]},{"text":", and AI imports without BYOK)","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"Adaptor config block name matches adaptorType (","type":"text"},{"text":"http{}","type":"text","marks":[{"type":"code_inline"}]},{"text":" for HTTPImport, ","type":"text"},{"text":"netsuite_da{}","type":"text","marks":[{"type":"code_inline"}]},{"text":" for NetSuiteDistributedImport, etc.)","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Adaptor-specific","type":"text"}]},{"type":"checkbox_list","attrs":{"id":null},"content":[{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"HTTP: pre-built connector was checked (","type":"text"},{"text":"celigo http-connectors list","type":"text","marks":[{"type":"code_inline"}]},{"text":") -- hand-written config only because no connector covers the app or operation","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"HTTP: ","type":"text"},{"text":"http.method","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"http.relativeURI","type":"text","marks":[{"type":"code_inline"}]},{"text":" are set (http.yml)","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"HTTP: string values in a JSON ","type":"text"},{"text":"http.body","type":"text","marks":[{"type":"code_inline"}]},{"text":" or lookup ","type":"text"},{"text":"body","type":"text","marks":[{"type":"code_inline"}]},{"text":" are double-braced inside quotes; triple braces only for numbers, booleans and serialized JSON (gotcha 11)","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"HTTP lookups: every lookup is in ","type":"text"},{"text":"http.lookups","type":"text","marks":[{"type":"code_inline"}]},{"text":" (the save keeps that array and mirrors it to the top-level ","type":"text"},{"text":"lookups","type":"text","marks":[{"type":"code_inline"}]},{"text":"; one sent only at the top level is dropped), each request template is in ","type":"text"},{"text":"body","type":"text","marks":[{"type":"code_inline"}]},{"text":", not ","type":"text"},{"text":"postBody","type":"text","marks":[{"type":"code_inline"}]},{"text":", and names the record's fields without ","type":"text"},{"text":"record.","type":"text","marks":[{"type":"code_inline"}]},{"text":" (gotcha 10), and each ","type":"text"},{"text":"extract","type":"text","marks":[{"type":"code_inline"}]},{"text":" is a path without ","type":"text"},{"text":"$.","type":"text","marks":[{"type":"code_inline"}]},{"text":" to the id itself or an array of ids, from a search on a key that matches one record (gotcha 14)","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"NetSuite DA: ","type":"text"},{"text":"netsuite_da.operation","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"netsuite_da.recordType","type":"text","marks":[{"type":"code_inline"}]},{"text":" are set, and ","type":"text"},{"text":"update","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"addupdate","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"delete","type":"text","marks":[{"type":"code_inline"}]},{"text":" carry ","type":"text"},{"text":"netsuite_da.internalIdLookup","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- the lookup, not a mapped ","type":"text"},{"text":"internalid","type":"text","marks":[{"type":"code_inline"}]},{"text":", picks the record (on ","type":"text"},{"text":"update","type":"text","marks":[{"type":"code_inline"}]},{"text":" it overrides a mapped one). When the record already carries the internal id, use the expression form ","type":"text"},{"text":"[\"internalid\",\"is\",\"{{{id}}}\"]","type":"text","marks":[{"type":"code_inline"}]},{"text":" rather than dropping the lookup (netsuitedistributed.yml)","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"NetSuite DA fields set at record initialization (NetSuite preloads them when it creates or transforms the record, before the other fields): one body row ","type":"text"},{"text":"{\"generate\": \"celigo_initializeValues\", \"hardCodedValue\": \"entity,subsidiary\"}","type":"text","marks":[{"type":"code_inline"}]},{"text":" listing their ids, comma-separated and without ","type":"text"},{"text":".internalid","type":"text","marks":[{"type":"code_inline"}]},{"text":", next to the rows that map those fields. It's what the mapping editor's \"Use this field for NetSuite record initialization\" saves; a ","type":"text"},{"text":"useAsAnInitializeValue","type":"text","marks":[{"type":"code_inline"}]},{"text":" key on a row isn't stored","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"NetSuite DA serialized items: the SuiteApp and bundle take a line's ","type":"text"},{"text":"serialnumbers","type":"text","marks":[{"type":"code_inline"}]},{"text":" as ","type":"text"},{"text":"\u003cserial>(\u003cquantity>)","type":"text","marks":[{"type":"code_inline"}]},{"text":" text when the account's Advanced Bin / Numbered Inventory Management feature is off; with it on, give the serials through the line's inventory detail subrecord (","type":"text"},{"text":"celigo_inventorydetail","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"netsuitedistributed.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/netsuitedistributed.yml","title":null}}]},{"text":")","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"NetSuite DA keyed on ","type":"text"},{"text":"externalid","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"[\"externalid\",\"is\",\"{{{id}}}\"]","type":"text","marks":[{"type":"code_inline"}]},{"text":"): a mapping writes that same source id to ","type":"text"},{"text":"externalid","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"{\"extract\": \"id\", \"generate\": \"externalid\"}","type":"text","marks":[{"type":"code_inline"}]},{"text":"). The lookup finds only records whose ","type":"text"},{"text":"externalid","type":"text","marks":[{"type":"code_inline"}]},{"text":" was set, so without the mapping an ","type":"text"},{"text":"addupdate","type":"text","marks":[{"type":"code_inline"}]},{"text":" adds the record again on every run","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"RDBMS: ","type":"text"},{"text":"rdbms.queryType","type":"text","marks":[{"type":"code_inline"}]},{"text":" is one current value (","type":"text"},{"text":"bulk_load","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"bulk_insert","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"per_page","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"per_record","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"first_page","type":"text","marks":[{"type":"code_inline"}]},{"text":") with its companion field set -- never the legacy uppercase ","type":"text"},{"text":"INSERT","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"UPDATE","type":"text","marks":[{"type":"code_inline"}]},{"text":" (rdbms.yml)","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"Salesforce: ","type":"text"},{"text":"salesforce.sObjectType","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"salesforce.operation","type":"text","marks":[{"type":"code_inline"}]},{"text":" are set. ","type":"text"},{"text":"upsert","type":"text","marks":[{"type":"code_inline"}]},{"text":" also needs ","type":"text"},{"text":"salesforce.upsert.externalIdField","type":"text","marks":[{"type":"code_inline"}]},{"text":" (the Salesforce external id field) and ","type":"text"},{"text":"salesforce.idLookup.extract","type":"text","marks":[{"type":"code_inline"}]},{"text":" (the record field matched against it); ","type":"text"},{"text":"update","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"delete","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"addupdate","type":"text","marks":[{"type":"code_inline"}]},{"text":" and an ","type":"text"},{"text":"insert","type":"text","marks":[{"type":"code_inline"}]},{"text":" with ","type":"text"},{"text":"ignoreExisting","type":"text","marks":[{"type":"code_inline"}]},{"text":" need ","type":"text"},{"text":"idLookup.whereClause","type":"text","marks":[{"type":"code_inline"}]},{"text":" instead (salesforce.yml). The mapping is Mapper 1.0: ","type":"text"},{"text":"mapping.fields[]","type":"text","marks":[{"type":"code_inline"}]},{"text":" rows of ","type":"text"},{"text":"{extract, generate}","type":"text","marks":[{"type":"code_inline"}]},{"text":" (or ","type":"text"},{"text":"hardCodedValue","type":"text","marks":[{"type":"code_inline"}]},{"text":"), never ","type":"text"},{"text":"mappings[]","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Cross-resource consistency","type":"text"}]},{"type":"checkbox_list","attrs":{"id":null},"content":[{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"Connection ","type":"text"},{"text":"type","type":"text","marks":[{"type":"code_inline"}]},{"text":" matches the import's ","type":"text"},{"text":"adaptorType","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"If response mapping needed: configured on the flow's ","type":"text"},{"text":"pageProcessors[]","type":"text","marks":[{"type":"code_inline"}]},{"text":" entry, not on the import itself","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"If one-to-many: ","type":"text"},{"text":"oneToMany: true","type":"text","marks":[{"type":"code_inline"}]},{"text":", with ","type":"text"},{"text":"pathToMany","type":"text","marks":[{"type":"code_inline"}]},{"text":" set to the child array path for an object record (child fields then use ","type":"text"},{"text":"record.\u003cfield>","type":"text","marks":[{"type":"code_inline"}]},{"text":" and parent fields ","type":"text"},{"text":"record._PARENT.\u003cfield>","type":"text","marks":[{"type":"code_inline"}]},{"text":") and left blank for an array (grouped) record","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"If downstream steps read this import's response: a realistic ","type":"text"},{"text":"mockResponse","type":"text","marks":[{"type":"code_inline"}]},{"text":" is saved, so test runs carry real-looking data past this step","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"If a repeated record would duplicate data: the write is keyed (a destination-enforced key, or ","type":"text"},{"text":"ignoreExisting","type":"text","marks":[{"type":"code_inline"}]},{"text":" + an existence check) and ","type":"text"},{"text":"idLockTemplate","type":"text","marks":[{"type":"code_inline"}]},{"text":" renders a non-empty key from a field the record carries on arrival -- never ","type":"text"},{"text":"{{lookup.*}}","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"Writing Exactly Once","type":"text","marks":[{"type":"link","attrs":{"href":"#writing-exactly-once","title":null}}]},{"text":"), and its existence lookup has been checked against a record that exists (gotcha 12)","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"Mapper 1.0 only where required: ","type":"text"},{"text":"mapping.fields[]","type":"text","marks":[{"type":"code_inline"}]},{"text":" / ","type":"text"},{"text":"mapping.lists[]","type":"text","marks":[{"type":"code_inline"}]},{"text":" on Salesforce and wrapper imports, ","type":"text"},{"text":"netsuite_da.mapping","type":"text","marks":[{"type":"code_inline"}]},{"text":" on NetSuite DA -- never ","type":"text"},{"text":"mappings[]","type":"text","marks":[{"type":"code_inline"}]},{"text":" there. ","type":"text"},{"text":"NetSuiteHTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" uses ","type":"text"},{"text":"mappings[]","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Gotchas","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"PUT erases omitted fields.","type":"text","marks":[{"type":"strong"}]},{"text":" Always GET first, modify, then PUT. The ","type":"text"},{"text":"set","type":"text","marks":[{"type":"code_inline"}]},{"text":" command handles this. The import PATCH route accepts only ","type":"text"},{"text":"debugUntil","type":"text","marks":[{"type":"code_inline"}]},{"text":" (any other path is refused with 422), so a change to a mapping, body or lookup is always this full PUT, never a reason to send the user to the UI. Leave the read-only fields a GET adds (","type":"text"},{"text":"_id","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"lastModified","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"createdAt","type":"text","marks":[{"type":"code_inline"}]},{"text":") out of the body.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Including a ","type":"text","marks":[{"type":"strong"}]},{"text":"rest:","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" block creates a legacy RESTImport.","type":"text","marks":[{"type":"strong"}]},{"text":" Use only ","type":"text"},{"text":"http:","type":"text","marks":[{"type":"code_inline"}]},{"text":" for new imports.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Input filter skips that import, not the record.","type":"text","marks":[{"type":"strong"}]},{"text":" Filtered records skip the current import step but continue to subsequent steps in the flow, without this import's response-mapped fields, so a later step that writes those fields back needs its own filter on them. Check ","type":"text"},{"text":"numIgnore","type":"text","marks":[{"type":"code_inline"}]},{"text":" on the job if records seem to bypass a step.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Multipart file parts only accept ","type":"text","marks":[{"type":"strong"}]},{"text":"{{blob}}","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":".","type":"text","marks":[{"type":"strong"}]},{"text":" In a ","type":"text"},{"text":"multipart/form-data","type":"text","marks":[{"type":"code_inline"}]},{"text":" parts array, the file part must be ","type":"text"},{"text":"\"type\": \"attachment\"","type":"text","marks":[{"type":"code_inline"}]},{"text":" with ","type":"text"},{"text":"\"value\": \"{{blob}}\"","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- any other value 422s. Never hardcode the MIME ","type":"text"},{"text":"boundary","type":"text","marks":[{"type":"code_inline"}]},{"text":"; the platform generates it. Fields the API wants alongside the file ride as ","type":"text"},{"text":"\"type\": \"inline\"","type":"text","marks":[{"type":"code_inline"}]},{"text":" parts.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"bodyKey","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":"/","type":"text","marks":[{"type":"strong"}]},{"text":"blobKey","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" in logs is an artifact, not a payload field.","type":"text","marks":[{"type":"strong"}]},{"text":" Audit and debug logs never show the assembled multipart body -- an internal storage pointer appears where the body would be. But if the destination actually received the literal string ","type":"text"},{"text":"bodyKey","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"blobKey","type":"text","marks":[{"type":"code_inline"}]},{"text":", the file part is misconfigured (an ","type":"text"},{"text":"inline","type":"text","marks":[{"type":"code_inline"}]},{"text":" part where an ","type":"text"},{"text":"attachment","type":"text","marks":[{"type":"code_inline"}]},{"text":" belongs, or a ","type":"text"},{"text":"blobKeyPath","type":"text","marks":[{"type":"code_inline"}]},{"text":" that doesn't resolve).","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"imports invoke","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" is a real write.","type":"text","marks":[{"type":"strong"}]},{"text":" \"No job created\" doesn't mean nothing is sent: every form executes the request against the live destination, and a saved ","type":"text"},{"text":"mockResponse","type":"text","marks":[{"type":"code_inline"}]},{"text":" doesn't stop them. It also runs the import's hooks, postSubmit included, and its response transform (the editor's ","type":"text"},{"text":"Send","type":"text","marks":[{"type":"strong"}]},{"text":" stops before postSubmit). A 200 can still carry per-record failures in each result's ","type":"text"},{"text":"errors","type":"text","marks":[{"type":"code_inline"}]},{"text":", and on a file import (FTP, S3, file-mode HTTP) it means the file was composed, not delivered: an FTP import to an unreachable host still answers ","type":"text"},{"text":"{name, fileLocation}","type":"text","marks":[{"type":"code_inline"}]},{"text":". Point it at a sandbox, or test-run the flow when you only need to check mapping output.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"A trace key is not duplicate detection.","type":"text","marks":[{"type":"strong"}]},{"text":" Repeated keys are normal (delta re-emits, retries) and never skip, merge, or reject a record. Only a keyed write stops the second copy -- see ","type":"text"},{"text":"Writing Exactly Once","type":"text","marks":[{"type":"link","attrs":{"href":"#writing-exactly-once","title":null}}]},{"text":".","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"A lock keyed on this import's own lookup is no lock.","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"idLockTemplate","type":"text","marks":[{"type":"code_inline"}]},{"text":" renders before the import's lookups and mapping run, so ","type":"text"},{"text":"{{lookup.\u003cname>}}","type":"text","marks":[{"type":"code_inline"}]},{"text":" renders empty. Check the key against a real record (an error's retry data: ","type":"text"},{"text":"celigo flows error \u003cflowId> \u003cimportId> \u003cerrorId> --retry-data","type":"text","marks":[{"type":"code_inline"}]},{"text":"), not a sample that may lack the field.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Lookups can't chain.","type":"text","marks":[{"type":"strong"}]},{"text":" A lookup's own URI, body and headers render before any lookup has run, so ","type":"text"},{"text":"{{lookup.\u003cname>}}","type":"text","marks":[{"type":"code_inline"}]},{"text":" there is undefined: it either fails the render (","type":"text"},{"text":"handlebars_template_parse_error","type":"text","marks":[{"type":"code_inline"}]},{"text":": ","type":"text"},{"text":"Unable to read \"\u003cname>\" from undefined","type":"text","marks":[{"type":"code_inline"}]},{"text":") or renders empty, and either way that lookup fails or asks the wrong question for every record. Test runs never execute imports, so the first real run is where it shows. Resolve the value in one query, or add an upstream lookup step and reference the field it adds to the record.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"An HTTP import's lookups send ","type":"text","marks":[{"type":"strong"}]},{"text":"body","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":", not ","type":"text","marks":[{"type":"strong"}]},{"text":"postBody","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":", and read the record's fields at the top level.","type":"text","marks":[{"type":"strong"}]},{"text":" Put a lookup's request template in ","type":"text"},{"text":"http.lookups[].body","type":"text","marks":[{"type":"code_inline"}]},{"text":". A template saved as ","type":"text"},{"text":"postBody","type":"text","marks":[{"type":"code_inline"}]},{"text":", the field ","type":"text"},{"text":"http.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/http.yml","title":null}}]},{"text":" documents, saves without complaint, but the lookup goes out with an empty body, so it asks the wrong question for every record and nothing errors. The CLI's schema check calls ","type":"text"},{"text":"body","type":"text","marks":[{"type":"code_inline"}]},{"text":" undocumented; it's still the field that's sent. Inside that template the record's fields sit at the top level: ","type":"text"},{"text":"{{email}}","type":"text","marks":[{"type":"code_inline"}]},{"text":" renders the record's value, and ","type":"text"},{"text":"{{record.email}}","type":"text","marks":[{"type":"code_inline"}]},{"text":" renders empty.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"In a JSON body, string values take double braces.","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"{\"name\": \"{{record.name}}\"}","type":"text","marks":[{"type":"code_inline"}]},{"text":" is JSON-escaped at runtime, so a quote, backslash, newline or tab in the value arrives intact. Triple braces insert the raw text: such a value fails its record with 422 ","type":"text"},{"text":"invalid_json","type":"text","marks":[{"type":"code_inline"}]},{"text":" before anything is sent, and in a lookup's ","type":"text"},{"text":"body","type":"text","marks":[{"type":"code_inline"}]},{"text":" the lookup quietly finds nothing (gotcha 12). Keep triple braces for numbers, booleans and serialized JSON (","type":"text"},{"text":"{{{jsonSerialize record}}}","type":"text","marks":[{"type":"code_inline"}]},{"text":").","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"A failing existence lookup counts as \"not found\".","type":"text","marks":[{"type":"strong"}]},{"text":" With ","type":"text"},{"text":"ignoreExisting","type":"text","marks":[{"type":"code_inline"}]},{"text":", an ","type":"text"},{"text":"http.ignoreLookupName","type":"text","marks":[{"type":"code_inline"}]},{"text":" lookup that fails -- a wrong URI answering 404, a body broken by a raw quote -- extracts nothing, so the record is written as new and no error is raised. Before trusting the check, invoke the import in a sandbox with a record that already exists there and confirm the result comes back ","type":"text"},{"text":"ignored","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"A 200 isn't always success.","type":"text","marks":[{"type":"strong"}]},{"text":" When an API answers 200 for a failed write, set ","type":"text"},{"text":"http.response.successPath","type":"text","marks":[{"type":"code_inline"}]},{"text":" + ","type":"text"},{"text":"successValues","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"failPath","type":"text","marks":[{"type":"code_inline"}]},{"text":" + ","type":"text"},{"text":"failValues","type":"text","marks":[{"type":"code_inline"}]},{"text":" (one entry per operation; values compare as strings). A record that trips them fails: an invoke reports ","type":"text"},{"text":"statusCode","type":"text","marks":[{"type":"code_inline"}]},{"text":" 422, and a run records an error whose code is the destination's status (","type":"text"},{"text":"200","type":"text","marks":[{"type":"code_inline"}]},{"text":"), source ","type":"text"},{"text":"application","type":"text","marks":[{"type":"code_inline"}]},{"text":", with the response body as its message -- or ","type":"text"},{"text":"response_failure","type":"text","marks":[{"type":"code_inline"}]},{"text":" (\"response failed using path\") when a ","type":"text"},{"text":"successPath","type":"text","marks":[{"type":"code_inline"}]},{"text":" value doesn't match.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"An ambiguous HTTP lookup fails only with ","type":"text","marks":[{"type":"strong"}]},{"text":"allowFailures: false","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":".","type":"text","marks":[{"type":"strong"}]},{"text":" When a lookup's ","type":"text"},{"text":"extract","type":"text","marks":[{"type":"code_inline"}]},{"text":" resolves to two or more values (an array of ids or of result objects), the record fails with ","type":"text"},{"text":"lookup_conflict","type":"text","marks":[{"type":"code_inline"}]},{"text":" (\"Multiple matches found for lookup.\") only if the lookup sets ","type":"text"},{"text":"allowFailures: false","type":"text","marks":[{"type":"code_inline"}]},{"text":". With it left out or ","type":"text"},{"text":"true","type":"text","marks":[{"type":"code_inline"}]},{"text":", the first value is sent and nothing errors -- from an array of objects, the literal text ","type":"text"},{"text":"[object Object]","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- and ","type":"text"},{"text":"useDefaultOnMultipleMatches","type":"text","marks":[{"type":"code_inline"}]},{"text":" hasn't been seen to send it to the ","type":"text"},{"text":"default","type":"text","marks":[{"type":"code_inline"}]},{"text":" instead. An indexed ","type":"text"},{"text":"extract","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"results[0].id","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"ids[0]","type":"text","marks":[{"type":"code_inline"}]},{"text":") takes the first match silently, whatever ","type":"text"},{"text":"allowFailures","type":"text","marks":[{"type":"code_inline"}]},{"text":" says: right for a search that can match only one record, and never a fix for ","type":"text"},{"text":"lookup_conflict","type":"text","marks":[{"type":"code_inline"}]},{"text":", which it turns into writes against whichever record comes first. Fix ambiguity in the search: query on a key that identifies one record (an email, an external id), taken from a field the record carries -- a failed record's retry data shows them -- and keep ","type":"text"},{"text":"allowFailures: false","type":"text","marks":[{"type":"code_inline"}]},{"text":", so a search that still finds two fails rather than writing against the wrong record.","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Common Errors","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Error","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Cause","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Fix","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"422 \"Invalid value '\u003cvalue>' for field 'adaptorType'\"","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Wrong case (","type":"text"},{"text":"httpimport","type":"text","marks":[{"type":"code_inline"}]},{"text":") or an unknown adaptor","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Use the exact case from the decision matrix: ","type":"text"},{"text":"HTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"NetSuiteDistributedImport","type":"text","marks":[{"type":"code_inline"}]},{"text":", etc.","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"422 \"Expected field: _connectionId to be present\"","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"No connection on an adaptor that needs one","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Set ","type":"text"},{"text":"_connectionId","type":"text","marks":[{"type":"code_inline"}]},{"text":"; only the connection-less imports in ","type":"text"},{"text":"Minimum Required Fields","type":"text","marks":[{"type":"link","attrs":{"href":"#minimum-required-fields","title":null}}]},{"text":" omit it","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"422 ","type":"text"},{"text":"invalid_ref","type":"text","marks":[{"type":"code_inline"}]},{"text":" (\"connection not found.; \u003cid> is not a valid _connectionId\")","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The connection id doesn't exist in this environment","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Use an id from ","type":"text"},{"text":"celigo connections list","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Copying an Integration App's import fails: 403 ","type":"text"},{"text":"create_not_allowed","type":"text","marks":[{"type":"code_inline"}]},{"text":" (\"Cannot create an Integration App resource\"), 422 ","type":"text"},{"text":"invalid_ref","type":"text","marks":[{"type":"code_inline"}]},{"text":" (\"The referenced Connection document ... belongs to different Integration App installation\"), or 400 from ","type":"text"},{"text":"imports clone","type":"text","marks":[{"type":"code_inline"}]},{"text":" (\"Cloning can't be performed for ... that were installed from an Integration App.\")","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The body still carries the app's ","type":"text"},{"text":"_connectorId","type":"text","marks":[{"type":"code_inline"}]},{"text":", or the copy keeps the app's connection, which only the app's own resources can use; a clone of an app's import is refused outright","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Build the copy as your own import, on a connection of your own, with ","type":"text"},{"text":"_connectorId","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"_id","type":"text","marks":[{"type":"code_inline"}]},{"text":" and the timestamps left out","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"422 \"http subschema not defined\" (or another connection type's name)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The ","type":"text"},{"text":"adaptorType","type":"text","marks":[{"type":"code_inline"}]},{"text":" doesn't match the connection: the save validates the import as the connection's adaptor","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Pick the adaptor for the connection's type from the decision matrix, with its config block","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"422 \"","type":"text"},{"text":"\u003cvalue>","type":"text","marks":[{"type":"code_inline"}]},{"text":" is not a valid enum value for path ","type":"text"},{"text":"\u003cfield>","type":"text","marks":[{"type":"code_inline"}]},{"text":"\"","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A value outside the field's enum, such as ","type":"text"},{"text":"http.method: [\"PUSH\"]","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Use a value the adaptor's schema lists","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"422 ","type":"text"},{"text":"Import query types … are deprecated","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"rdbms.queryType","type":"text","marks":[{"type":"code_inline"}]},{"text":" holds the legacy uppercase ","type":"text"},{"text":"INSERT","type":"text","marks":[{"type":"code_inline"}]},{"text":" (or a retired ","type":"text"},{"text":"MERGE","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"COPY","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"BULK INSERT","type":"text","marks":[{"type":"code_inline"}]},{"text":")","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Use one current value -- ","type":"text"},{"text":"bulk_load","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"bulk_insert","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"per_page","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"per_record","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"first_page","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- and set its companion field","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"422 ","type":"text"},{"text":"… is not a valid enum value for path rdbms.queryType.0","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A ","type":"text"},{"text":"queryType","type":"text","marks":[{"type":"code_inline"}]},{"text":" string outside the enum, such as lowercase ","type":"text"},{"text":"insert","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Use one of the current values above","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"422 ","type":"text"},{"text":"distributed required","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Missing NetSuite flag","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Use ","type":"text"},{"text":"NetSuiteDistributedImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" with ","type":"text"},{"text":"distributed: true","type":"text","marks":[{"type":"code_inline"}]},{"text":" on the import (there's no such connection flag); check the SuiteApp with ","type":"text"},{"text":"celigo connections suiteapp-status \u003cconnectionId>","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"422 ","type":"text"},{"text":"mapping invalid","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Wrong mapper version","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Salesforce uses Mapper 1.0 (","type":"text"},{"text":"mapping.fields[]","type":"text","marks":[{"type":"code_inline"}]},{"text":") and NetSuite DA uses ","type":"text"},{"text":"netsuite_da.mapping","type":"text","marks":[{"type":"code_inline"}]},{"text":", not Mapper 2.0 (","type":"text"},{"text":"mappings[]","type":"text","marks":[{"type":"code_inline"}]},{"text":"); ","type":"text"},{"text":"NetSuiteHTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":" uses ","type":"text"},{"text":"mappings[]","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Unable to find lookup in mapping configs - \u003cname>","type":"text","marks":[{"type":"code_inline"}]},{"text":" on every record (source ","type":"text"},{"text":"mapping","type":"text","marks":[{"type":"code_inline"}]},{"text":")","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A mapping row's ","type":"text"},{"text":"lookupName","type":"text","marks":[{"type":"code_inline"}]},{"text":" names a lookup the import doesn't define -- on an HTTP import, often one sent only in the top-level ","type":"text"},{"text":"lookups","type":"text","marks":[{"type":"code_inline"}]},{"text":", which the save drops; the save doesn't check it","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Define the lookup under that exact name (in ","type":"text"},{"text":"http.lookups","type":"text","marks":[{"type":"code_inline"}]},{"text":" on an HTTP import), or fix the row","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Every record fails with ","type":"text"},{"text":"Cannot set properties of undefined (setting '_hasStarNotation')","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A NetSuite DA mapping condition (","type":"text"},{"text":"conditional.lookupName","type":"text","marks":[{"type":"code_inline"}]},{"text":") names a lookup the import doesn't define; on a sublist row the job can stall instead","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Fix the name or restore the lookup -- changing the SuiteScript version doesn't help","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"INVALID_HANDLE_BAR_EXPRESSION","type":"text","marks":[{"type":"code_inline"}]},{"text":" on a NetSuite DA sublist lookup","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The lookup's ","type":"text"},{"text":"expression","type":"text","marks":[{"type":"code_inline"}]},{"text":" reads a bare line field, which is undefined at the record root","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Address the line as ","type":"text"},{"text":"\u003carray>[*].\u003cfield>","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"{{{line_items[*].sku}}}","type":"text","marks":[{"type":"code_inline"}]},{"text":")","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"VALUE_LOOKUP_FAILED","type":"text","marks":[{"type":"code_inline"}]},{"text":" \"Could not find a match for …\"","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The rendered criteria, quoted in the message, matched no NetSuite record; inactive records don't count unless the lookup sets ","type":"text"},{"text":"includeInactive","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Fix the criteria or the data, or set ","type":"text"},{"text":"allowFailures: true","type":"text","marks":[{"type":"code_inline"}]},{"text":" with a ","type":"text"},{"text":"default","type":"text","marks":[{"type":"code_inline"}]},{"text":" to fall back","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"VALUE_LOOKUP_FAILED","type":"text","marks":[{"type":"code_inline"}]},{"text":" \"More than one match for …\" (","type":"text"},{"text":"MORE_THAN_ONE_MATCH_FOUND","type":"text","marks":[{"type":"code_inline"}]},{"text":" on a ","type":"text"},{"text":"suiteapp2.0","type":"text","marks":[{"type":"code_inline"}]},{"text":" step)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The criteria matched several records; an empty source value searches ","type":"text"},{"text":"[\"\u003cfield>\",\"isempty\"]","type":"text","marks":[{"type":"code_inline"}]},{"text":", which matches every blank one","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Keep an empty key out of the search (branch in the ","type":"text"},{"text":"expression","type":"text","marks":[{"type":"code_inline"}]},{"text":"), or set ","type":"text"},{"text":"useDefaultOnMultipleMatches","type":"text","marks":[{"type":"code_inline"}]},{"text":" with ","type":"text"},{"text":"allowFailures: true","type":"text","marks":[{"type":"code_inline"}]},{"text":" and a ","type":"text"},{"text":"default","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"NetSuite ","type":"text"},{"text":"INVALID_KEY_OR_REF","type":"text","marks":[{"type":"code_inline"}]},{"text":" (\"Invalid \u003cfield> reference key \u003cvalue>.\") on some records","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The field references another NetSuite record, and the value NetSuite received isn't a key it accepts. When the quoted value is a name or code and the row writing that field sets ","type":"text"},{"text":"internalId: true","type":"text","marks":[{"type":"code_inline"}]},{"text":" (its value must be the referenced record's internal id), those records carry the name where the id belongs (","type":"text"},{"text":"Diagnosing NetSuite Steps","type":"text","marks":[{"type":"link","attrs":{"href":"../configuring-exports/references/netsuite.md","title":null}}]},{"text":" has the other causes)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Resolve the name to the id on that row with a lookup: a static ","type":"text"},{"text":"map","type":"text","marks":[{"type":"code_inline"}]},{"text":" from each name to its internal id (the ids are the user's to supply), or a NetSuite lookup that searches the name field and returns ","type":"text"},{"text":"resultField: \"internalid\"","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"netsuitedistributed.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/netsuitedistributed.yml","title":null}}]},{"text":"). Then retry the failed records; a retry before the change fails again","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"NetSuite ","type":"text"},{"text":"USER_ERROR","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"source: application","type":"text","marks":[{"type":"code_inline"}]},{"text":") whose message states a business rule (\"PO # is required for customers in the Distributor category\")","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A rule in the NetSuite account -- often its own user event script or workflow -- refused the record. Celigo's hooks report their own ","type":"text"},{"text":"source","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"pre_map_hook","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"post_submit_hook_ss","type":"text","marks":[{"type":"code_inline"}]},{"text":" and the like), so ","type":"text"},{"text":"application","type":"text","marks":[{"type":"code_inline"}]},{"text":" means NetSuite raised it","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Read a failing record's retry data for the value the rule asks for and check whether a mapping row writes it: map it, or have a NetSuite administrator change the rule. Don't edit a Celigo hook for it (","type":"text"},{"text":"Diagnosing NetSuite Steps","type":"text","marks":[{"type":"link","attrs":{"href":"../configuring-exports/references/netsuite.md","title":null}}]},{"text":")","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"NetSuite ","type":"text"},{"text":"INVALID_FLD_VALUE","type":"text","marks":[{"type":"code_inline"}]},{"text":" \"You have entered an Invalid Field Value \u003cvalue> for the following field: \u003cfield>\" on some records, on a list or select field","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The row sends text, and a NetSuite DA import sets a select field's text by the option's display name: the list has no option named exactly that value -- often a new value the source started sending. On a date field the same code means a value the import can't read as a date (","type":"text"},{"text":"Diagnosing NetSuite Steps","type":"text","marks":[{"type":"link","attrs":{"href":"../configuring-exports/references/netsuite.md","title":null}}]},{"text":")","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Compare the quoted value with what the records that succeed send. Add the option to the NetSuite list with that exact name, or translate the values with a static lookup on the row (to the options' internal ids, with ","type":"text"},{"text":"internalId: true","type":"text","marks":[{"type":"code_inline"}]},{"text":"). Then retry the failed records","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"NetSuite ","type":"text"},{"text":"USER_ERROR","type":"text","marks":[{"type":"code_inline"}]},{"text":" \"Please enter value(s) for: \u003cfield label>\" on some records only","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The field is mandatory for those records and the mapping leaves it empty for them: their source value is blank, or another field makes NetSuite require it -- a customer whose ","type":"text"},{"text":"isperson","type":"text","marks":[{"type":"code_inline"}]},{"text":" is ","type":"text"},{"text":"F","type":"text","marks":[{"type":"code_inline"}]},{"text":" (a company) needs Company Name, while an individual (","type":"text"},{"text":"isperson","type":"text","marks":[{"type":"code_inline"}]},{"text":" ","type":"text"},{"text":"T","type":"text","marks":[{"type":"code_inline"}]},{"text":") needs first and last name instead","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Read a failing record's retry data next to one that succeeded. Map the field from a value those records have, give the row a default, or set the deciding field from the data (","type":"text"},{"text":"isperson","type":"text","marks":[{"type":"code_inline"}]},{"text":" ","type":"text"},{"text":"T","type":"text","marks":[{"type":"code_inline"}]},{"text":" when the company is empty). Then retry the failed records","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"NetSuite ","type":"text"},{"text":"SSS_INVALID_SUBLIST_OPERATION","type":"text","marks":[{"type":"code_inline"}]},{"text":" (\"…trying to add or remove lines from a static sublist\", often ending \"Tip: Please make sure that all the line items that you're trying to use exist on the record.\") on an item fulfillment","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A mapped line matched none of the fulfillment's lines, so the import tried to add one, and a fulfillment can't add item lines: its lines are its order's. Lines match on the sublist's ","type":"text"},{"text":"isKey","type":"text","marks":[{"type":"code_inline"}]},{"text":" columns, or on the item when none is marked (","type":"text"},{"text":"writing-mappings > Mapper 1.0","type":"text","marks":[{"type":"link","attrs":{"href":"../writing-mappings/SKILL.md#mapper-10-reference-netsuite-and-salesforce","title":null}}]},{"text":")","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Find the line whose key value none of the order's lines holds -- in its retry data, or from a change to the row or lookup that resolves the item -- and fix the mapping or the data. A retry before that fails the same way","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"NetSuite ","type":"text"},{"text":"MULTIPLE_LINES_WITH_SAME_KEYS","type":"text","marks":[{"type":"code_inline"}]},{"text":" \"Multiple lines found with keys item=\u003cid> …\"","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Two of the record's lines hold the mapped line's key values: on a fulfillment, receipt or invoice keyed on the item (no column is ","type":"text"},{"text":"isKey","type":"text","marks":[{"type":"code_inline"}]},{"text":"), an order with that item on two lines","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Key each line on what only it holds: map the order line's id (its ","type":"text"},{"text":"line","type":"text","marks":[{"type":"code_inline"}]},{"text":" on the order) to ","type":"text"},{"text":"orderline","type":"text","marks":[{"type":"code_inline"}]},{"text":" and mark that row ","type":"text"},{"text":"isKey","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- a mapped ","type":"text"},{"text":"orderline","type":"text","marks":[{"type":"code_inline"}]},{"text":" without ","type":"text"},{"text":"isKey","type":"text","marks":[{"type":"code_inline"}]},{"text":" isn't a key, so the item still is. Then retry the failed records; a retry before the change fails the same way","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A NetSuite update repeats the record's lines on every run, with no error","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The sublist has no ","type":"text"},{"text":"isKey","type":"text","marks":[{"type":"code_inline"}]},{"text":" column, so no mapped line matches an existing line and each is added; only sublists with a default key fall back to it -- the item on fulfillment-style transactions, the applied transaction on a payment's or credit's","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Mark the row that identifies a line ","type":"text"},{"text":"isKey","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- a line id the record's create wrote to a line field -- so each mapped line updates its own. To also drop lines the source no longer has, map ","type":"text"},{"text":"celigo_replaceAllLines_\u003csublist id>","type":"text","marks":[{"type":"code_inline"}]},{"text":" to ","type":"text"},{"text":"true","type":"text","marks":[{"type":"code_inline"}]},{"text":". Records already updated keep their repeated lines, and since the repeats carry the same id, their next update with the key set fails with ","type":"text"},{"text":"MULTIPLE_LINES_WITH_SAME_KEYS","type":"text","marks":[{"type":"code_inline"}]},{"text":": clean those up first","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Records the destination already has show as ignored, and their changes never arrive","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"ignoreExisting","type":"text","marks":[{"type":"code_inline"}]},{"text":" skips each record its existence check finds; it never updates one","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"To update them, make the import a composite UPDATE/CREATE that decides on the same lookup (","type":"text"},{"text":"http.existingLookupName","type":"text","marks":[{"type":"code_inline"}]},{"text":"; ","type":"text"},{"text":"Composite imports","type":"text","marks":[{"type":"link","attrs":{"href":"#composite-two-endpoint-imports","title":null}}]},{"text":") and drop ","type":"text"},{"text":"ignoreExisting","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The mapping condition:\"record_updated\" not supported for the operation","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A ","type":"text"},{"text":"record_updated","type":"text","marks":[{"type":"code_inline"}]},{"text":" condition on a NetSuite DA ","type":"text"},{"text":"add","type":"text","marks":[{"type":"code_inline"}]},{"text":" import","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Use ","type":"text"},{"text":"addupdate","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"update","type":"text","marks":[{"type":"code_inline"}]},{"text":", or drop the condition","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A record fails with code ","type":"text"},{"text":"200","type":"text","marks":[{"type":"code_inline"}]},{"text":", source ","type":"text"},{"text":"application","type":"text","marks":[{"type":"code_inline"}]},{"text":", and the response body as its message (or ","type":"text"},{"text":"response_failure","type":"text","marks":[{"type":"code_inline"}]},{"text":")","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The import's ","type":"text"},{"text":"failPath","type":"text","marks":[{"type":"code_inline"}]},{"text":" / ","type":"text"},{"text":"successPath","type":"text","marks":[{"type":"code_inline"}]},{"text":" rules judged a 200 a failure","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Expected when the API reports failures in a 200 body (gotcha 13); fix the record, or the rule if it misfires","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"422 attachment ","type":"text"},{"text":"value","type":"text","marks":[{"type":"code_inline"}]},{"text":" invalid","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Multipart file part is not ","type":"text"},{"text":"{{blob}}","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Set the file part to ","type":"text"},{"text":"\"type\": \"attachment\"","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"\"value\": \"{{blob}}\"","type":"text","marks":[{"type":"code_inline"}]},{"text":"; let the platform generate the ","type":"text"},{"text":"boundary","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"422 ","type":"text"},{"text":"invalid_json","type":"text","marks":[{"type":"code_inline"}]},{"text":" (\"An invalid JSON is sent in the request.body\") on some records only","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A triple-braced string in a JSON ","type":"text"},{"text":"http.body","type":"text","marks":[{"type":"code_inline"}]},{"text":" met a value holding a quote, backslash, newline or tab; in a GraphQL ","type":"text"},{"text":"variables","type":"text","marks":[{"type":"code_inline"}]},{"text":" string, any string value without ","type":"text"},{"text":"jsonEncode","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Double-brace string values inside quotes (gotcha 11); inside a GraphQL ","type":"text"},{"text":"variables","type":"text","marks":[{"type":"code_inline"}]},{"text":" string, ","type":"text"},{"text":"{{jsonEncode record.x}}","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"GraphQL imports","type":"text","marks":[{"type":"link","attrs":{"href":"#graphql-imports","title":null}}]},{"text":")","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A GraphQL import reports every record written, and the destination has none","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The server answered 200 with an ","type":"text"},{"text":"errors","type":"text","marks":[{"type":"code_inline"}]},{"text":" array or the mutation's ","type":"text"},{"text":"userErrors","type":"text","marks":[{"type":"code_inline"}]},{"text":", and the import sets no response rules","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Set ","type":"text"},{"text":"resourceIdPath","type":"text","marks":[{"type":"code_inline"}]},{"text":" to the created object's id, and ","type":"text"},{"text":"failPath","type":"text","marks":[{"type":"code_inline"}]},{"text":"/","type":"text"},{"text":"errorPath","type":"text","marks":[{"type":"code_inline"}]},{"text":" to ","type":"text"},{"text":"errors[0].message","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"GraphQL imports","type":"text","marks":[{"type":"link","attrs":{"href":"#graphql-imports","title":null}}]},{"text":")","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"missing_id","type":"text","marks":[{"type":"code_inline"}]},{"text":" (\"could not extract id using path: …\")","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"No id at ","type":"text"},{"text":"resourceIdPath","type":"text","marks":[{"type":"code_inline"}]},{"text":": the write failed (the response, quoted in the message, carries the server's errors) or the path is wrong","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Read the quoted response; fix the data, or the path","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"handlebars_template_parse_error","type":"text","marks":[{"type":"code_inline"}]},{"text":": \"Failed to generate request url from template: … Details: \"\u003cfield>\" not defined in the model.\" (source ","type":"text"},{"text":"application","type":"text","marks":[{"type":"code_inline"}]},{"text":", classification ","type":"text"},{"text":"parse","type":"text","marks":[{"type":"code_inline"}]},{"text":") on some records","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Those records lack the field the ","type":"text"},{"text":"relativeURI","type":"text","marks":[{"type":"code_inline"}]},{"text":" names, or its ","type":"text"},{"text":"{{lookup.\u003cname>}}","type":"text","marks":[{"type":"code_inline"}]},{"text":" found nothing -- missing data, not template syntax. Nothing was sent, and a retry fails the same way","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"When the records may not exist at the destination yet (new items, no destination id), make the import a composite UPDATE/CREATE that decides on the field (","type":"text"},{"text":"http.existingExtract","type":"text","marks":[{"type":"code_inline"}]},{"text":"; ","type":"text"},{"text":"Composite imports","type":"text","marks":[{"type":"link","attrs":{"href":"#composite-two-endpoint-imports","title":null}}]},{"text":"), or skip them with ","type":"text"},{"text":"ignoreMissing: true","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"http.ignoreExtract","type":"text","marks":[{"type":"code_inline"}]},{"text":" set to the field. Otherwise get the id onto the record upstream, then retry","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"422 ","type":"text"},{"text":"unsupported_type","type":"text","marks":[{"type":"code_inline"}]},{"text":" (\"NetSuiteHTTPImport does not support type: file\")","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"http.type: \"file\"","type":"text","marks":[{"type":"code_inline"}]},{"text":" on a ","type":"text"},{"text":"NetSuiteHTTPImport","type":"text","marks":[{"type":"code_inline"}]},{"text":": the NetSuite REST import writes records, and the save refuses file mode","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Remove ","type":"text"},{"text":"http.type","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Every record fails with \"Directory is not found\"","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"http.type: \"file\"","type":"text","marks":[{"type":"code_inline"}]},{"text":" (file-storage folder mode) combined with an API upload endpoint","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Remove ","type":"text"},{"text":"http.type: \"file\"","type":"text","marks":[{"type":"code_inline"}]},{"text":"; keep ","type":"text"},{"text":"blob: true","type":"text","marks":[{"type":"code_inline"}]},{"text":" + ","type":"text"},{"text":"blobKeyPath","type":"text","marks":[{"type":"code_inline"}]},{"text":" and post to the endpoint","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"422 ","type":"text"},{"text":"invalid_adaptor","type":"text","marks":[{"type":"code_inline"}]},{"text":" (\"Import preview is not supported for adaptor type\")","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"celigo imports invoke","type":"text","marks":[{"type":"code_inline"}]},{"text":" with a document on stdin, for a NetSuite, Salesforce, RDBMS, or FTP import","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Save the import and invoke it by ID, or run the document as a virtual import with ","type":"text"},{"text":"--connection","type":"text","marks":[{"type":"code_inline"}]},{"text":" (both real writes); or test-run the flow to check mapping output without sending","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"422 ","type":"text"},{"text":"Connection is offline","type":"text","marks":[{"type":"code_inline"}]},{"text":" from a mapping preview or a NetSuite field check","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The destination connection is marked offline (its ","type":"text"},{"text":"offline","type":"text","marks":[{"type":"code_inline"}]},{"text":" flag), so the platform doesn't call it; nothing is wrong with the import or its mapping","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Fix the connection (test it, re-authorize), then retry","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A Salesforce import fails with ","type":"text"},{"text":"response stream exceeded limit of 5242880 bytes.","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The platform reads at most 5 MB of Salesforce's answer to a request, and this batch's results were larger","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Send fewer records per request","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Salesforce ","type":"text"},{"text":"UNABLE_TO_LOCK_ROW","type":"text","marks":[{"type":"code_inline"}]},{"text":" on some records -- on the ","type":"text"},{"text":"rest","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"compositerecord","type":"text","marks":[{"type":"code_inline"}]},{"text":" APIs the platform replaces Salesforce's message with \"Salesforce creates a lock on a record while updating it. If another request is trying to update the same record which is already locked, Salesforce would throw this error.\" and a help link","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Another write held the Salesforce record this one needed: this import's own pages or runs submitting in parallel, or another import or flow writing the same records","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Set ","type":"text"},{"text":"idLockTemplate","type":"text","marks":[{"type":"code_inline"}]},{"text":" to the key of the record the writes contend for -- the record's own id, or the parent id they share when they update children of one record, taken from a field the incoming record carries (","type":"text"},{"text":"{{{record.AccountId}}}","type":"text","marks":[{"type":"code_inline"}]},{"text":"; gotcha 8) -- so pages carrying the same key never submit together through this import. The lock doesn't reach other imports: when another flow writes the same records, lower the connection's ","type":"text"},{"text":"concurrencyLevel","type":"text","marks":[{"type":"code_inline"}]},{"text":" instead. Then retry the failed records","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A Salesforce import reports a field error for a record that Salesforce saved","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"On the ","type":"text"},{"text":"soap","type":"text","marks":[{"type":"code_inline"}]},{"text":" API (the default), when Salesforce refuses a value in a field the import mapped (the error names the field), the platform drops that field and sends the record again, up to ","type":"text"},{"text":"maxAttempts","type":"text","marks":[{"type":"code_inline"}]},{"text":" more times (default and maximum 3; ","type":"text"},{"text":"maxAttempts: 1","type":"text","marks":[{"type":"code_inline"}]},{"text":" turns it off). A resend that succeeds saves the record without the field, and its error still counts against the job","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Look the record up in Salesforce before retrying: a retry of an ","type":"text"},{"text":"insert","type":"text","marks":[{"type":"code_inline"}]},{"text":" creates it a second time. Fix the value at the source, update the saved record, and resolve the error","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A ","type":"text"},{"text":"compositerecord","type":"text","marks":[{"type":"code_inline"}]},{"text":" import fails a record with ","type":"text"},{"text":"Composite Operation failed due to the following error(s)","type":"text","marks":[{"type":"code_inline"}]},{"text":", then one line per failure (","type":"text"},{"text":"\u003cmessage> for \u003cobject>[\u003cindex>]","type":"text","marks":[{"type":"code_inline"}]},{"text":")","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"One object in the record's tree failed. The platform sends a record and its related records as one all-or-none Composite API request, so Salesforce saved none of them; the rollback notices it returns for the other objects (","type":"text"},{"text":"PROCESSING_HALTED","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"ALL_OR_NONE_OPERATION_ROLLED_BACK","type":"text","marks":[{"type":"code_inline"}]},{"text":") are left out, and the error's code is the first real failure's","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Fix the field each line names on that object, then retry the whole record","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Salesforce ","type":"text"},{"text":"ENTITY_IS_DELETED","type":"text","marks":[{"type":"code_inline"}]},{"text":" on an ","type":"text"},{"text":"update","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"delete","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The record was deleted in Salesforce","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"With ","type":"text"},{"text":"ignoreMissing: true","type":"text","marks":[{"type":"code_inline"}]},{"text":" the import counts such records as ignored instead of failing them; without it, resolve the errors once the deletions are confirmed as intended","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Salesforce ","type":"text"},{"text":"DUPLICATE_VALUE","type":"text","marks":[{"type":"code_inline"}]},{"text":" (\"duplicate value found: \u003cfield> duplicates value on record with id: \u003cid>\") on some records","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Salesforce's meaning: the record set a unique or External ID field to a value another record already holds. On an ","type":"text"},{"text":"insert","type":"text","marks":[{"type":"code_inline"}]},{"text":" import those records already exist in Salesforce -- sent again by an overlapping window, a re-run, or because they changed","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"To create new records and update existing ones, make the import an ","type":"text"},{"text":"upsert","type":"text","marks":[{"type":"code_inline"}]},{"text":" on that External ID field (","type":"text"},{"text":"upsert.externalIdField","type":"text","marks":[{"type":"code_inline"}]},{"text":" with the ","type":"text"},{"text":"idLookup.extract","type":"text","marks":[{"type":"code_inline"}]},{"text":" that carries the value; ","type":"text"},{"text":"salesforce.yml","type":"text","marks":[{"type":"link","attrs":{"href":"references/schemas/salesforce.yml","title":null}}]},{"text":"), then retry the failed records. ","type":"text"},{"text":"insert","type":"text","marks":[{"type":"code_inline"}]},{"text":" with ","type":"text"},{"text":"ignoreExisting","type":"text","marks":[{"type":"code_inline"}]},{"text":" and an ","type":"text"},{"text":"idLookup.whereClause","type":"text","marks":[{"type":"code_inline"}]},{"text":" skips them instead, and their changes never arrive","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The destination refuses the text ","type":"text"},{"text":"Invalid date","type":"text","marks":[{"type":"code_inline"}]},{"text":" in a date field (its error quotes that value)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A date mapping (","type":"text"},{"text":"extractDateFormat","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"extractDateTimezone","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"generateDateFormat","type":"text","marks":[{"type":"code_inline"}]},{"text":" set, Mapper 1.0 or 2.0) couldn't read the source value with its ","type":"text"},{"text":"extractDateFormat","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- ","type":"text"},{"text":"YYYY-MM-DD HH:mm:ss","type":"text","marks":[{"type":"code_inline"}]},{"text":" when unset -- so it wrote the text ","type":"text"},{"text":"Invalid date","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Set ","type":"text"},{"text":"extractDateFormat","type":"text","marks":[{"type":"code_inline"}]},{"text":" to the format the source actually sends (a failed record's retry data shows the value), then retry the failed records","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Downstream steps see ","type":"text"},{"text":"1234567890","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"\"replace me\"","type":"text","marks":[{"type":"code_inline"}]},{"text":" in a test run","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"No ","type":"text"},{"text":"mockResponse","type":"text","marks":[{"type":"code_inline"}]},{"text":" saved on the upstream import -- test runs never call the destination","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Save a realistic ","type":"text"},{"text":"mockResponse","type":"text","marks":[{"type":"code_inline"}]},{"text":" on that import","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The destination receives the same record twice","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A bare create; ","type":"text"},{"text":"ignoreExisting","type":"text","marks":[{"type":"code_inline"}]},{"text":" with no lock on records whose trace key is blank, or with an existence lookup that fails (gotcha 12); or a lock keyed on the import's own lookup","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Key the write and lock it on the business key (","type":"text"},{"text":"Writing Exactly Once","type":"text","marks":[{"type":"link","attrs":{"href":"#writing-exactly-once","title":null}}]},{"text":"); find which runs wrote it first (","type":"text"},{"text":"building-flows > Duplicate Records","type":"text","marks":[{"type":"link","attrs":{"href":"../building-flows/SKILL.md#duplicate-records","title":null}}]},{"text":")","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A NetSuite ","type":"text"},{"text":"addupdate","type":"text","marks":[{"type":"code_inline"}]},{"text":" adds the same record again on every run","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"internalIdLookup","type":"text","marks":[{"type":"code_inline"}]},{"text":" searches ","type":"text"},{"text":"externalid","type":"text","marks":[{"type":"code_inline"}]},{"text":", but no mapping writes ","type":"text"},{"text":"externalid","type":"text","marks":[{"type":"code_inline"}]},{"text":", so the lookup never finds what earlier runs added","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Map the source id to ","type":"text"},{"text":"externalid","type":"text","marks":[{"type":"code_inline"}]},{"text":" and keep the lookup; the duplicates already added stay until someone merges them","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"lookup_error","type":"text","marks":[{"type":"code_inline"}]},{"text":": \"Could not find value for static lookup: \u003cname> {Lookup Key: \u003cvalue>} …\" on some records","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The value isn't a key of that lookup's ","type":"text"},{"text":"map","type":"text","marks":[{"type":"code_inline"}]},{"text":", and the lookup has ","type":"text"},{"text":"allowFailures: false","type":"text","marks":[{"type":"code_inline"}]},{"text":"; the other records succeed","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Add each missing value as a key, exactly as the message quotes it (case, spacing and suffixes count), with the destination's value (ask the user -- a guessed code sends the record wrong), keeping every existing key in the save, then retry the failed records. A retry runs the step as it is now, so one before the map changes fails again. A ","type":"text"},{"text":"default","type":"text","marks":[{"type":"code_inline"}]},{"text":" would instead send every unmapped value as the default, with no error","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"lookup_conflict","type":"text","marks":[{"type":"code_inline"}]},{"text":": \"Multiple matches found for lookup.\" (source ","type":"text"},{"text":"lookup","type":"text","marks":[{"type":"code_inline"}]},{"text":") on some records","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"An HTTP lookup with ","type":"text"},{"text":"allowFailures: false","type":"text","marks":[{"type":"code_inline"}]},{"text":" got two or more values at its ","type":"text"},{"text":"extract","type":"text","marks":[{"type":"code_inline"}]},{"text":": the search key isn't unique (a name several records share)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Search on a key that identifies one record and keep ","type":"text"},{"text":"allowFailures: false","type":"text","marks":[{"type":"code_inline"}]},{"text":". Don't index the ","type":"text"},{"text":"extract","type":"text","marks":[{"type":"code_inline"}]},{"text":" (","type":"text"},{"text":"ids[0]","type":"text","marks":[{"type":"code_inline"}]},{"text":") or allow failures to get past it: either one writes against whichever match comes first (gotcha 14)","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"batch_lookup_evaluation_failed","type":"text","marks":[{"type":"code_inline"}]},{"text":": \"\u003cfield>: The batch lookup could not be evaluated. Check the value in the key '\u003cvalue>', and try again.\" (source ","type":"text"},{"text":"internal","type":"text","marks":[{"type":"code_inline"}]},{"text":")","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"An HTTP lookup with ","type":"text"},{"text":"allowFailures: false","type":"text","marks":[{"type":"code_inline"}]},{"text":" got nothing usable for that value: the search matched nothing, or the ","type":"text"},{"text":"extract","type":"text","marks":[{"type":"code_inline"}]},{"text":" resolves nothing (a ","type":"text"},{"text":"$.","type":"text","marks":[{"type":"code_inline"}]},{"text":" prefix, ","type":"text"},{"text":"[*]","type":"text","marks":[{"type":"code_inline"}]},{"text":", a dot path through an array) or points at a result object rather than an id","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Check the ","type":"text"},{"text":"extract","type":"text","marks":[{"type":"code_inline"}]},{"text":" against a real response -- a path without ","type":"text"},{"text":"$.","type":"text","marks":[{"type":"code_inline"}]},{"text":" to the id (","type":"text"},{"text":"results[0].id","type":"text","marks":[{"type":"code_inline"}]},{"text":") or an array of ids; if the value truly has no match, fix the data or the search, then retry","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"422 ","type":"text"},{"text":"cannot_set_traceKeyTemplate","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"traceKeyTemplate","type":"text","marks":[{"type":"code_inline"}]},{"text":" set on a one-to-one import","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Set it on the source export or listener -- on an import it is valid only with ","type":"text"},{"text":"oneToMany: true","type":"text","marks":[{"type":"code_inline"}]},{"text":". To change the lock key, set ","type":"text"},{"text":"idLockTemplate","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"oneToMany_error","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A blank ","type":"text"},{"text":"pathToMany","type":"text","marks":[{"type":"code_inline"}]},{"text":" on an object record (\"Record at index 0 was not an array, and no PathToMany was provided\"). The editor's Preview can also show it falsely on a step fed an array, because the preview re-splits the record it displays","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Object record: set ","type":"text"},{"text":"pathToMany","type":"text","marks":[{"type":"code_inline"}]},{"text":" to the child array. Array (grouped) record: blank is right, and each row is the record, with no ","type":"text"},{"text":"_PARENT","type":"text","marks":[{"type":"code_inline"}]},{"text":" -- confirm with a test run before changing the configuration","type":"text"}]}]}]}]}]},"metadata":{"date":"2026-11-02","name":"configuring-imports","author":"@skillopedia","source":{"stars":4,"repo_name":"ai","origin_url":"https://github.com/celigo/ai/blob/HEAD/skills/configuring-imports/SKILL.md","repo_owner":"celigo","body_sha256":"aabc64046d78a27931632d048a8eff93959dbc0bf05067a5a96ea1b600bb0b44","cluster_key":"d0d9e590ce13c9b068ab5bca571ce2c8e685feb8d6d30db12e4a2002d2795309","clean_bundle":{"format":"clean-skill-bundle-v1","source":"celigo/ai/skills/configuring-imports/SKILL.md","text_bytes":851151,"source_commit":"2f3853ed93eece259b8c78ee74ef19035dfae052","skills_sh_rank":6167,"attachment_count":38,"text_attachments":38,"attachment_storage":"skill_attachments","binary_attachments":0},"cluster_size":1,"skill_md_path":"skills/configuring-imports/SKILL.md","import_metadata":{"date":"2026-11-02","author":"@skillopedia","version":"v1","category":"integrations-apis","category_label":"Integrations"},"exact_dupes_collapsed_into_this":0},"version":"v1","category":"integrations-apis","import_tag":"skills-sh-top20k-2026-11","description":"Configure Celigo imports -- the destination step that writes records to external systems. Use when creating or editing imports, choosing the adaptor type, setting up lookups, upsert or exactly-once writes, or file-based imports. For mapping rules, use writing-mappings; for AI agent steps, configuring-ai-agents."}},"renderedAt":1793666243992}

Important: agents should read /llm.txt, /llms.txt, or /.well-known/skills.json to discover the public Skillopedia API.