Ingest Okta data into Port via Airbyte, S3 and webhook
This guide will demonstrate how to ingest Okta data into Port using Airbyte, S3 and a webhook integration.
S3 integrations lack some of the features (such as reconciliation) found in Ocean or other Port integration solutions.
As a result, if a record ingested during the initial sync is later deleted in the data source, thereβs no automatic mechanism to remove it from Port. The record simply wonβt appear in future syncs, but it will remain in Port indefinitely.
If the data includes a flag for deleted records (e.g., is_deleted: "true"), you can configure a webhook delete operation in your webhookβs mapping configuration to remove these records from Port automatically.
Prerequisitesβ
-
Ensure you have a Port account and have completed the onboarding process.
-
This feature is part of Port's limited-access offering. To obtain the required S3 bucket, please contact our team directly via chat, Slack, or e-mail, and we will create and manage the bucket on your behalf.
-
Access to an available Airbyte app (can be cloud or self-hosted) - for reference, follow the quick start guide.
-
An Okta Personal API Token used to retrieve data.
Data model setupβ
Add Blueprintsβ
Add the Okta Permission
blueprint:
-
Go to the Builder page of your portal.
-
Click on "+ Blueprint".
-
Click on the
{...}
button in the top right corner, and choose "Edit JSON". -
Paste the following JSON schema into the editor:
Okta Permission (Click to expand)
Add the Okta Role
blueprint in the same way:
Okta Role (Click to expand)
Add the Okta Role Assignment
blueprint in the same way:
Okta Role Assignment (Click to expand)
Add the Okta User
blueprint in the same way:
Okta User (Click to expand)
Create Webhook Integrationβ
Create a webhook integration to ingest the data into Port:
-
Go to the Data sources page of your portal.
-
Click on "+ Data source".
-
In the top selection bar, click on Webhook, then select
Custom Integration
. -
Enter a name for your Integration (for example: "Okta Integration"), enter a description (optional), then click on
Next
. -
Copy the Webhook URL that was generated and include set up the airbyte connection (see Below).
-
Scroll down to the section titled "Map the data from the external system into Port" and paste the following mapping:
Okta Webhook Mapping (Click to expand)
Airbyte Setupβ
Set up S3 Destinationβ
If you haven't already set up S3 Destination for Port S3, follow these steps:
- UI
- Terraform
-
Login to your Airbyte application (cloud or self-hosted).
-
In the left-side pane, click on
Destinations
. -
Click on
+ New Destination
. -
Input the S3 Credentials provided to you by Port:
- Under S3 Key ID enter your S3 Access Key ID.
- Under S3 Access Key enter your S3 Access Key Secret.
- Under S3 Bucket Name enter the bucket name (example: "org-xxx").
- Under S3 Bucket Path enter "data/".
- Under S3 Bucket Region enter the appropriate region.
- For output format, choose "JSON Lines: Newline-delimited JSON".
- For compression, choose "GZIP".
- Under Optional Fields, enter the following in S3 Path Format:
${NAMESPACE}/${STREAM_NAME}/year=${YEAR}/month=${MONTH}/${DAY}_${EPOCH}_
-
Click
Test and save
and wait for Airbyte to confirm the Destination is set up correctly.
terraform {
required_providers {
airbyte = {
source = "airbytehq/airbyte"
version = "0.6.5"
}
}
}
provider "airbyte" {
username = "<AIRBYTE_USERNAME>"
password = "<AIRBYTE_PASSWORD>"
server_url = "<AIRBYTE_API_URL>"
}
resource "airbyte_destination_s3" "port-s3-destination" {
configuration = {
access_key_id = "<S3_ACCESS_KEY>"
secret_access_key = "<S3_SECRET_KEY>"
s3_bucket_region = "<S3_REGION>"
s3_bucket_name = "<S3_BUCKET>"
s3_bucket_path = "data/"
format = {
json_lines_newline_delimited_json = {
compression = { gzip = {} }
format_type = "JSONL"
}
}
s3_path_format = `$${NAMESPACE}/$${STREAM_NAME}/year=$${YEAR}/month=$${MONTH}/$${DAY}_$${EPOCH}_`
destination_type = "s3"
}
name = "port-s3-destination"
workspace_id = var.workspace_id
}
variable "workspace_id" {
default = "<AIRBYTE_WORKSPACE_ID>"
}
Set up Okta Connectionβ
-
Follow Airbyte's guide to set up Okta connector.
-
After the Source is set up, proceed to create a "+ New Connection".
-
For Source, choose the Okta source you have set up.
-
For Destination, choose the S3 Destination you have set up.
-
In the Select Streams step, make sure only "custom_roles", "permissions", "user_role_assignments" and "users" are marked for synchronization.
-
In the Configuration step, under "Destination Namespace", choose "Custom Format" and enter the Webhook URL you have copied when setting up the webhook", for example: "wSLvwtI1LFwQzXXX".
-
Click on Finish & Sync to apply and start the Integration process!
If for any reason you have entered different values than the ones specified in this guide, inform us so we can assist to ensure the integration will run smoothly.