Data Loader

Data Loader is an application added to Console that allows users to load data from various locations (Google Cloud Storage (GCS), Microsoft Azure Blob Storage, Amazon Web Services (AWS) S3, Secure File Transfer Protocol (SFTP) or the local machine.

Data Loader is an intuitive application that enables users to visualize data, define mappings, do basic transformation, and analyze data before loading into the tenant. Currently, the following file types are supported for the input file:

  • CSV
  • Excel (.xlsx)
  • JSON
Note: Formulas and formatting are not supported, and therefore the Excel files must be in plain format.

Related Video:

While the user loads the JSON file, the content of the file is displayed with an array of JSON objects.

Example: Array of JSON objects

{
    "crosswalks_value": "609-854-7156",
    "crosswalks_type": "configuration/sources/HMS",
    "attributes_LastName_value": "Fritz",
    "attributes_FirstName_value": "An",
    "attributes_AddressLine_value": "506 S Hacienda Dr",
    "type": "configuration/entityTypes/HCP"
  },
  {
    "crosswalks_value": "203-564-1543",
    "crosswalks_type": "configuration/sources/HMS",
    "attributes_LastName_value": "Layous",
    "attributes_FirstName_value": "Ma",
    "attributes_AddressLine_value": "78112 Morris Ave",
    "type": "configuration/entityTypes/HCP"
  },
  {
    "crosswalks_value": "215-647-2158",
    "crosswalks_type": "configuration/sources/HMS",
    "attributes_LastName_value": "Rantanen",
    "attributes_FirstName_value": "My",
    "attributes_AddressLine_value": "8 Mcarthur Ln",
    "type": "configuration/entityTypes/HCP"
  },
  {
    "crosswalks_value": "973-903-4175",
    "crosswalks_type": "configuration/sources/HMS",
    "attributes_LastName_value": "Mcnease",
    "attributes_FirstName_value": "Nu",
    "attributes_AddressLine_value": "88 Sw 28th Ter",
    "type": "configuration/entityTypes/HCP"
  }
]

Data Loader API and Limitations

The following file size limitations exist:

  • The file size limit allocated for a .csv file is 33 GB. Make sure the file size does not exceed 50 MB, when you upload a file from the local file system.
  • The file size limit allocated for a JSON file is 50 MB, irrespective of whether you upload the file from the local file system or remote storage.
Important: While loading the data through a .csv file, when the CSV file is delimited by || and a column value in .csv has | (single pipe), you must wrap the values which have | with double quotation marks.

For more information on the general requirements from the specification, see CSV specification.

For example, consider that columns - col1, col2, col3, and col4 are delimited/separated by double pipes. The value corresponding to col3 has two values separated by a single pipe. So, the value for column 3 must be wrapped within double quotes as shown below.

col1 || col2||col3||col4

val1||val2||”val3|val4”||val5

Data Loader API

Use the following link to access Data Loader API documentation: Reltio Swagger.