Bulk Data Tranfer

Securely send customer data using encrypted storage

The preferred data upload method used by fido is batch transfer over an S3 bucket.

Simple Storage Service (Amazon S3) is an object storage service offering industry-leading scalability, data availability, security, and performance.

All the buckets used to store customer data are fiscally separate and encryption is enabled both at rest and in transit.

Parameters

In order to proceed with an upload, your private bucket must first be created.

After you reach out to your account manager they will provide the following information:

  • Key
  • Secret
  • Bucket Name
  • Region

The above parameters are used to connect with the AWS S3 service.

The bucket can be created in any of the available AWS regions, you can look at the current regions here

File format

Each row must contain the data points required, in separate columns.

  • customer_id: the internal number you use to identify each customer
  • date: the date when the client made the request
  • first_name: first name of the customer/identity
  • last_name: last name of the customer/identity
  • ip: the IP address of the user, no data center/server
  • user_agent: User agent of the browser performing the
  • phone: The expected format is international. With leading country-code and without zeros or '+' symbol
  • email: email address of the customer/identity
  • outcome: Must contain only OK or KO. Please note: the % of OK/KO must be representative of reality

An example:

customer_id,date,first_name,last_name,email,phone,ip,user_agent,outcome
18590357,yyyy-mm-dd,Francesco,Galli,[email protected],393394554546,192.168.0.1,Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36,OK

Linux:

If your operating system of choice is Unix based you can then proceed with an upload using AWS CLI commands, more info can be found here:

An example using the CP command:

aws s3 cp ./a_file_name.csv s3://a_bucket_name/a_file_name.csv

Another example using the SYNC command:

aws s3 sync ./a_folder s3://a_bucket_name/a_folder

Windows

If you use Windows, just download S3 Browser, Once done, just follow the instructions:

Step 1: you’ll be asked to provide the KEY and the secret, then click to “Add new account”

546546

Step 2: you’ll be asked to add an External Bucket, please click YES

545545

Step 3: you’ll be asked to provide the Bucket Name, then click on “Add External bucket”

You should be able to see your bucket folder and upload/download any file.


Did this page help you?