Ornitela

Supported technology/Animal Tracking

cpanel.glosendas.net

Direction Data provider
Data Movement data
Platform Gundi v2
Type Pull integration

Setup Gundi engineering

Ornitela sends GPS positions and sensor data from its tags to EarthRanger through Gundi. OrniTrack uploads data files to Gundi's SFTP server, Gundi checks for new files on a schedule, and the positions show up as tracks on the map.

What you'll receive

We add details here as we confirm what Ornitela sends. If you know something we're missing, let us know.

Positions Yes, from every tag whose data OrniTrack uploads, named after the device name in OrniTrack. Only GPS records become positions on the map.
Alerts and events No
Sensor readings Speed, heading, altitude, battery voltage and charge, solar current, satellite count, fix quality (HDOP), and accelerometer and magnetometer values. With Include Sensor Data on (the default), extra sensor records (temperature, light, depth, conductivity) are also sent, without a real location.
Photos and files No
Past data Gundi sends what OrniTrack uploads. Records older than Historical Limit Days in the Route settings (5 days by default) are skipped. There is no separate history fetch.

How this integration works

This is a pull integration. Gundi checks the data provider for new data on a schedule, using the account details you enter in the Route, and passes what it finds on to your destination.

Who's involved

  • You create the Route in Gundi and enter your provider account details, or give them to Gundi support when the setup badge says so.
  • The data provider keeps your account and devices active.
  • Your destination, for example EarthRanger, is where the data shows up.
  • Gundi support helps only when the setup badge at the top of this page says so.

Historical data

When you first connect, Gundi fetches a limited amount of past data. How far back it goes depends on the integration and is shown under "Past data" above.

When data doesn't show up

Check your Route Activity Logs first: wrong account details and provider outages show up there. Then check with the provider that your devices are reporting.

How often data arrives

Gundi checks on a schedule. Where we know how often, it's shown under "Data Pull Frequency" on this page. New data arrives after the next check, plus the time the device takes to upload it to the provider.

Prerequisites

Ornitela

You will need:

  • An account with them, with Admin privileges.   

Gundi

  • Gundi SFTP account details.

EarthRanger

To send movement data to EarthRanger, you will need:

  • A special EarthRanger user (e.g., “Gundi Service Account”) for this integration. Please refer to EarthRanger's documentation or contact Support.
  • A long-lived token for authentication assigned to the user created in the previous step. Please refer to EarthRanger's documentation or contact Support.

Configuration

GUIDES

1. Gundi SFTP Server Settings

Log in to the Gundi SFTP Admin Panel.

Go to Users and find an existing integration to use as a template (e.g. another Ornitela user).

Click Actions > Use as Template.

Under Users, set the username and password for the external provider to use when connecting to the SFTP server. Leave the public key field blank.

Note: The username and password set in step 5 are the credentials you will hand off to the external provider (OrniTrack) for their SFTP upload configuration (see next section).

Under File System, update the following fields:

  1. Storage: leave as GCS (default)
  2. Root directory: set to /srv/sftpgo/data/{sitename} (e.g. /srv/sftpgo/data/skyrangers)
  3. Bucket: leave unchanged from the template (sintegrate-sftpgo-storage-prod)
  4. Automatic credentials: leave toggled onKey Prefix: set to {provider}/{sitename}/ (e.g. ornitela/skyrangers/)
  5. Click Save.

2. OrniTrack Settings

2.1 Go to OrniTrack.

2.2 Go to Settings.

2.3 Enable data upload to FTP/SFTP server.

2.3 Select CSV 2 as the data file format.

2.4 Use sftp.gundiservice.org as the Host name.

2.5 Use 2022 as the Port number.

2.6 Use / as the Remote Directory.

2.7 Configure the username and password used by the Gundi SFTP account.

3. Gundi Connection Settings

3.1 Log in to Gundi.

3.2 Click Create Route.

3.3 Select Ornitela from the list of supported technologies

3.4 Configure these fields:

Workspace name

Route name (e.g., “Ornitela (Account)”)

Bucket Path: ornitela/{username}

Batch Size: 300

Chunk Size: 3000

Max Files per run: 5

Historical Limit Days: leave the default of 5

3.10 Click Next.

3.11 Click Add destination on the flow map and select a destination. Please consult our guides.

4. EarthRanger Settings

The following configurations are optional:

4.1. Select the data you want to see in the map.

4.1.1 Log in to EarthRanger as an admin (site.pamdas.org/admin).
4.1.2 Go to Source Providers.
4.1.3 Select the Source Provider created for this integration (e.g., "gundi_ornitela_UUID")

The “Source Provider” is automatically created after Gundi successfully retrieves data from this integration and pushes it to EarthRanger. If you don't find it, please ensure your Route includes a destination, and review the Route Activity Logs for additional details. You should see some records mentioning that Gundi successfully delivered data to EarthRanger.
Important Note:  The integration might take 5 minutes to run. Please verify your connection after this period.

4.1.4 Click on Subject Details Configuration.
4.1.5 Configure the additional information you would like to see on the map.

4.2 Assign your Subjects to a Subject Group.

Please refer to EarthRanger's documentation or contact Support.

Troubleshooting

Internal Documentation


Q: How does file processing work end-to-end?
Files move through three bucket folders in sequence:

  1. Root (ornitela/) — Ornitela drops raw CSV files here. The connector checks this every 5 minutes.
  2. in_progress/ — When processing starts, the connector carves a chunk from the root file, compresses
      it, and places it here. The root file is updated with remaining rows (or deleted if empty).
  3. archive/ — On success, the chunk is moved here. If the root file still has rows, the next chunk is
      triggered automatically (chaining).
  4. dead_letter/ — On failure or timeout, the chunk is moved here instead.

Files named like bird001_a3f9c1b2_2.csv.gz encode: <original_stem>_<chain_id>_<chunk_index>.csv.gz.


Q: How do I trace a single file's full processing history in logs?
Every log line for a chunk is tagged with the filename in square brackets:

   [bird001_a3f9c1b2_1.csv.gz] Starting processing for integration 97eed06a-...
   [bird001_a3f9c1b2_1.csv.gz] Processing in_progress/bird001_a3f9c1b2_1.csv.gz: 3000 rows
   [bird001_a3f9c1b2_1.csv.gz] Archived successfully
   [bird001_a3f9c1b2_1.csv.gz] Processed: extracted 3000 records, sent 3000 observations

To trace an entire chain (all chunks from one source file), filter logs by the chain_id (the 8-character
hex segment, e.g. a3f9c1b2). All chunks carved from the same source file share the same chain_id.

In Cloud Logging / GCP, use:

   textPayload=~"a3f9c1b2"


Q: A file ended up in dead_letter/. What happened?
Check the logs for the filename. Common causes:

  • “Timed out — moving to dead_letter/”. Processing exceeded MAX_ACTION_EXECUTION_TIME (default: 8 min). File may be too large, Gundi API was slow, or Cloud Run killed the container.
  • "Error: <exception>" + “Successfully moved to dead_letter/”. Unhandled exception during CSV parsing, download, or observation sending. See exception on preceding line.
  • “Could not move <file> to dead_letter/ — file remains in    in_progress/”. The dead-letter move itself failed (GCS permission or network issue). File is still in in_progress/.

Retrying dead-letter files:  Move the file back to the root folder (ornitela/). The cron job will pick it up on the next 5-minute tick.

Q: Files are piling up in in_progress/ and not moving anywhere.
This usually means the processing action was interrupted before it could archive or dead-letter the file.

Causes:

  • Cloud Run SIGTERM at 5 minutes: if PROCESS_PUBSUB_MESSAGES_IN_BACKGROUND is not "true", Cloud Run kills the container at the request timeout (300s) before the action completes. The file stays in in_progress/ indefinitely.
  • Container restart / OOM: same result — no cleanup path ran.

To diagnose:
Check if the file has been in in_progress/ longer than 10 minutes with no log activity. If so, move it to root to retry.

Permanent fix:
Set PROCESS_PUBSUB_MESSAGES_IN_BACKGROUND=true in Cloud Run.

Q: process_new_files reports new_files_found: 0 but files exist in the bucket.
The action only counts root-level files — files not under in_progress/, archive/, or dead_letter/. Files already being processed or archived are intentionally excluded.

If the root appears empty but you expect files:

  1. Check that files were uploaded to the correct bucket and bucket_path prefix (default: ornitela/).
  2. Verify the INFILE_STORAGE_BUCKET env var is set to the correct bucket.
  3. Look for log lines: "Could not get metadata for file <name>" — this indicates a permissions or network issue during listing.

Q: process_new_files returns success immediately but observations haven't arrived yet.
This is expected behavior. process_new_files returns as soon as the chunk sub-actions are triggered — it does not wait for them to finish.

The actual observations are sent by action_process_ornitela_file, which runs asynchronously.

Check logs for:
[<chunk_name>] Processed: extracted N records, sent N observations

Q: Observations are arriving but they're all filtered out (nothing appears in EarthRanger).
Check generate_gundi_observations filtering. Rows are dropped if:

  • recorded_at is older than historical_limit_days (default: 5 days). If the file contains older historical data, increase this value in the action config.
  • The UTC_datetime field is missing or malformed. The row will raise a ValueError and be skipped, with a log like:
    Error parsing CSV row in bird001_a3f9c1b2_1.csv.gz: time data '' does not match format '%Y-%m-%d %H:%M:%S'

Q: The chain stopped — only chunk 1 was processed, chunks 2+ never appeared.
The self-chaining mechanism triggers the next chunk from _trigger_next_chunk after archiving chunk N.

If the chain breaks:

  1. Lock contention: "Source file is locked — cron will handle the next chunk"
    This is normal. The 5-minute cron will pick up the root file on the next run.
  2. Source file 404: "Source file no longer exists — chain complete"
    The root file was already consumed or deleted. The chain finished normally.
  3. Trigger error: "Error triggering next chunk: <error>"
    The trigger_action call failed. The root file still exists with remaining rows, so the cron will retry.
  4. Cron picked it up independently:
    Both the self-chain and cron attempted processing. The lock prevented duplication, and the successful chain ID appears in logs.

Q: How do I check how many observations a specific run sent?
The activity log in the Gundi portal shows per-chunk results. Each process_ornitela_file action logs:

[<chunk_name>] Processed: extracted N records, sent N observations

The activity log entry includes chain_id as structured metadata, allowing filtering of all chunks from one source file.

Q: The cleanup action deleted files it shouldn't have, or isn't deleting anything.
action_cleanup_archive runs at midnight and deletes files from archive/ older than delete_after_archive_days (default: 3 days), based on GCS timeCreated.

  • Deleting too aggressively: increase delete_after_archive_days in the action config.
  • Not deleting anything: ensure the CleanupArchive action config exists and the value is set correctly. Look for:
    Deleted old archived file: archive/<name>
  • Errors during cleanup:
    "Could not process archived file archive/<name>: <error>"
    Individual file errors are logged as warnings and skipped. The rest continue.

Q: What do the log levels mean for this connector?
Level: INFO
Meaning: Normal progress such as chunk creation, row parsing, archiving, and triggering the next chunk

Level: WARNING
Meaning: Recoverable issues such as metadata fetch failures, transient GCS interruptions, or file locks

Level: ERROR
Meaning: File sent to dead letter, failure moving to dead letter, or unhandled exceptions

Level: DEBUG
Meaning: Encoding detection per file (visible only if LOGGING_LEVEL=DEBUG)

 
 

 

 

Data Pull Frequency

Approximately every 5 minutes.

Other Animal Tracking

Argos Africa Wildlife Tracking (AWT) Advanced Telemetry Systems (or ATS)
Ceres Wild digitanimal  
Back to the integration catalog Need help setting this up? Contact us