Files Inbound Connector

About the Files Inbound Connector

You can use the files inbound connector to load customer data via a file upload. These jobs can be scheduled to repeat on a schedule that you set, or be configured to be a one-time data pull.

File inbound jobs allow you to upload data in the following formats:

• Flat delimited text files (CSV, TSV, etc)
• XLS or XLSX
• JSON
• WebVTT

Setting Up a Files Inbound Job

1. On the Jobs page, click New Job.
   
2. Select Files.
   
3. Give your job a name so you can identify it.
   
4. Choose the project to load data into.
5. Give your job a description so you know its purpose.
6. Click Next.
7. Choose an SFTP account to use, or select Add New to add a new one.
   
8. If adding a new account, enter the following information for your SFTP account:
   • Connection Name: Give the connection a name so you can recognize it later.
   • SFTP Username: Enter the username to connect to your SFTP server.
   • SFTP Password: Enter the password to connect to your SFTP server.
   • SFTP Host: Enter your SFTP server host URL.
   • SFTP Port: Enter your SFTP Port. Usually this is 22.
   • Send Expiration Notifications: If you’d like your SFTP credentials to expire after a certain amount of time, select this option.
   • Days Prior Expiration: If enabling expiration notifications, enter how many days you’d like to be notified before the expiration. You may enter a value between 1 to 100 days.
   • Expiration Date: Set the date when the credentials will expire. Click on the box to open a calendar to pick the date.
9. Click Next.
10. Choose the type of file you want to upload:
    
    • Delimiter-separated “flat” Files
    • Excel Files (XLS, XLSX)
    • JSON Files
    • WebVTT Files
11. Depending on your selected file type, there are additional settings for you to choose:
    • Delimiter-separated flat Files: For delimiter-separated files, choose the following:
      
      • File encoding: Choose the file’s encoding system (UTF-8, ASCII, etc)
      • Delimiter character: Enter the character used to delimit data entries. By default, this is a comma for CSV files.
      • Enclosure character: Enter the character that encloses data entry. Leave this field blank if No enclosure is selected.
      • No enclosure: Enable this option if your file does not contain enclosure characters.
    • JSON: Enter the JSON path that contains the document data you want to upload to XM Discover. Leave this field blank if your documents are at the root level.
      
    • Skip to the next step for Excel and WebVTT.
12. Click Browse and choose the folder path on your SFTP server that contains the file you’d like to upload.
    Qtip: When choosing a folder path, you can click on a folder icon to expand that folder. Click on the folder name to select that folder, and then click Ok when done.
13. Enter the filename regular expression that matches the files you want to upload.
14. If your files are compressed, enable Unzip file(s) and enter the Zipped Filename regular expression that matches the files that should be decompressed.
    
15. If your files are decrypted, enable Decrypt file(s) and enter the following:
    • Encrypted Filename regular expression: Enter the filename regular expression for files that should be decrypted.
    • PGP/GPG key: Enter the PGP/GPG key used for encryption.
    • Passphrase: Enter the passphrase for decryption.
16. If your files are both compressed and encrypted, you must choose which should happen first:
    • Unzip then Decrypt: Files are decompressed and then decrypted.
    • Decrypt then Unzip: Files are decrypted and then decompressed.
17. Choose the type of data you want to import:
    
    • Chat: Digital interactions with multiple lines of dialogue between 2 or more participants.
    • Call: Call transcripts with multiple lines of dialogue between 2 or more participants.
    • Feedback: Documents presented as single-row, or “flat” objects.
      Qtip: Depending on your file type, some data types are not compatible. For example, WebVTT files can only be used to upload call transcripts.
18. You must provide a sample file, which can be either saved on your computer or on your SFTP server:
    • Select Download Sample from File Storage if the sample file is saved on your SFTP server.
    • Select Upload sample from my computer if the sample is saved to your computer.
19. If needed, you can select a Template file to download. Click the here link to download your selected template. Use this file to add the data you want to import into XM Discover. See the XM Discover Data Formats support page for specific formatting information on each file and data type.
20. Click the Select Sample File button and choose the sample file on your computer or on your SFTP server.
21. A preview of the file will appear. If you see an error message or raw file contents instead of the preview, there may be a problem with the data format options you selected. See the Sample File Errors page for help troubleshooting your file.
22. Click Next.
23. If needed, adjust your data mappings. See the Data Mapping support page for detailed information on mapping fields in XM Discover. The Default Data Mapping section has information on the fields specific to this connector and the Mapping Conversational Fields section covers how to map data for conversational data.
    
24. Click Next.
25. If desired, you can add data substitution and redaction rules to hide sensitive data or automatically replace certain words and phrases in customer feedback and interactions. See the Data Substitution and Redaction support page for more information.
    
26. Click Next.
27. If desired, you can add a connector filter to filter the incoming data to limit what data is imported.
    
28. You can also limit the number of records imported in a single job by entering a number in the Specify Record Limit box. Enter “All” if you would like to import all records.
    Qtip: For conversational data, the limit is applied based on conversations rather than rows.
29. Click Next.
30. Choose when you’d like to be notified. See Job Notifications for more information.
    
31. Click Next.
32. Choose how duplicate documents are handled. See Duplicate Handling for more information.
    
33. Choose Schedule Incremental Runs if you’d like your job to run periodically on a schedule, or Set Up One-Time Pull if you’d like the job to run just once. See Job Scheduling for more information.
34. Choose a retention period for processed files. Files are deleted after the set period.
    
35. Click Next.
36. Review your setup. If you need to change a specific setting, click the Edit button to be brought to that step in the connector setup.
    
37. Click Finish to save the job.

Default Data Mapping

This section contains information on the default fields for files inbound jobs.

• feedback_type: Identifies data based on its type. This is useful for reporting when your project contains different types of data (for example, surveys and social media feedback). By default, this attribute’s value is set to “call” for call transcripts, “chat” for digital interactions or “feedback” for individual feedback. Use custom transformations to set a custom value, define an expression, or map it to another field.
• source_value: Identifies data obtained from a specific source. This can be anything that describes the origin of data, such as the name of a survey or a mobile marketing campaign. By default, this attribute’s value is set to “Files.” Use custom transformations to set a custom value, define an expression, or map it to another field.
• document_date: The primary date field associated with a document. This date is used in XM Discover reports, trends, alerts, and so on. You can use any date field in your dataset for the document date. You can also set a specific document date.
• natural_id: A unique identifier of a document. It is highly recommended to have a unique ID for each document to process duplicates correctly. For Natural ID, you can select any text or numeric field from your data. Alternatively, you can automatically generate IDs by adding a custom field.
• feedback_provider: Identifies data obtained from a specific provider. For file uploads, this attribute’s value is set to “Files” and cannot be changed.
• job_name: Identifies data based on the name of the job used to upload it. You can modify this attribute’s value during the setup via the Job Name field that is displayed at the top of each page during the setup.
• loadDate: Indicates when a document was uploaded into XM Discover. This field is set automatically and cannot be changed.

Updating Sample Configuration

Files inbound connectors require a sample file to generate data mappings. If you want to change the data mappings after creating a job, perform the following:

1. Unmap the fields that are not present in the new sample file.
2. Update sample configuration by uploading a new sample file with a new set of fields. The steps to update the sample configuration are below.
3. Update data mappings according to the new sample file.

Updating sample configuration

1. In the job options menu, select Update Sample Configuration for an existing files job.
   
2. Select Click To Select File For Upload and choose your sample file saved on your computer.
   
   Qtip: If your file contains fields that are not in your data mapping, you will receive an error. To fix this issue, edit your data mapping so it has the same fields as your sample file.
3. Double check your file settings. See the Files Inbound Connector page for more information about sample file settings.
4. Click Ok.

Updating Path and File Mask

Files inbound connectors require a path and file mask (filename regular expression) to find the right files to import. You can update the path and file mask for any existing files job.

1. In the job options menu, select Update Path and File Mask for an existing files job.
   
2. Click Browse.
   
3. Select the folder where the files are stored.
   
   Attention: Only the files in the selected folder are processed; sub-folders need to be processed separately.
4. Click Ok.
5. In the Filename regular expression field, enter the file mask (filename regular expression) that matches the files you’d like to import.
   
6. Click Save.

XM Discover Supported Key Exchange Ciphers

Connecting to an SFTP server requires a series of SSH handshakes to coordinate the encryption used between the server and the automation. In order to establish a connection, the SFTP server will need to have at least one cipher for each cipher type that matches a cipher supported by automations. Below is the list of ciphers supported by automations grouped by the cipher type.

Here’s a list of key exchange ciphers supported by XM Discover SFTP servers:

• diffie-hellman-group14-sha256
• diffie-hellman-group18-sha512
• ecdh-sha2-nistp256
• ecdh-sha2-nistp384
• ecdh-sha2-nistp521
• curve25519-sha256
• curve25519-sha256@libssh.org
• diffie-hellman-group16-sha512
• ecdh-nistp256-kyber-512r3-sha256-d00@openquantumsafe.org
• ecdh-nistp384-kyber-768r3-sha384-d00@openquantumsafe.org
• ecdh-nistp521-kyber-1024r3-sha512-d00@openquantumsafe.org
• x25519-kyber-512r3-sha256-d00@amazon.com