Destination Options¶

scheduler.* — Transfer Scheduler Options¶

These options affect the transfer requests created for the destination queue.

Timing & scheduling¶

Queue management & request attributes¶

Requeue & duplicate handling¶

Force overrides¶

Quick-start example — business-hours scheduling¶

scheduler.activeTimeRange = "06:00-18:00,20:00-22:00"
scheduler.delay = "15m"
scheduler.lifetime = "3d"
scheduler.version = "$destination-$date-$target"
scheduler.dateformat = "yyyyMMdd"

Quick-start example — hold larger duplicate files¶

scheduler.requeueon = "$size2 > $size1"
scheduler.requeuepattern = "(.*)\.grib"
scheduler.requeueignore = "(.*)\.tmp"

Quick-start example — force rules per filename¶

scheduler.force = "
  (== {avhrr_n.*}) standby=never;delay=2h
  (.= E1) asap=yes;transfergroup=internet
  (=. .tmp) standby=always
"

incoming.* — Destination Portal Options¶

These options control how files pushed to or pulled from this Destination behave in the Data Portal. They are destination-level settings, unlike portal.*, which applies to a specific Data User.

Upload scheduling & lifecycle¶

Temporary upload handling¶

Metadata & parsing¶

Listing & presentation¶

Transfer-rate limits & quotas¶

Quick-start example — portal uploads with metadata and eventing¶

incoming.priority = "80"
incoming.delay = "10m"
incoming.version = "$date-$target"
incoming.dateformat = "yyyyMMddHHmm"
incoming.metadata = "source=portal;target=$target;version=$date"
incoming.event = "yes"

Quick-start example — temporary uploads and quotas¶

incoming.tmpDetect = "yes"
incoming.tmpPattern = "(.*)\.partial|(.*)\.tmp"
incoming.maxUploadBytes = "100GB"
incoming.uploadPeriod = "24h"
incoming.maxBytesPerSecForOutput = "20MB"

alias.* — Destination Alias Options¶

Conditional aliasing¶

alias.pattern accepts comma-separated parameters such as:

• selectors: pattern=..., ignore=...
• alias attributes: lifetime=..., priority=..., asap=..., event=..., delay=...
• date controls for $date: dateformat=..., datedelta=..., datesource=..., datepattern=...
• target rewriting with placeholders such as $name, $path, $parent, $destination, $alias and $date

For multi-line rules, each line uses this format:

({operator} target) parameter1=value1,parameter2=value2

Supported operators are ==, !=, .= and =.. With ==, a target wrapped in {} is interpreted as a regex.

Quick-start example — alias GRIB files only¶

alias.pattern = "pattern=(.*)\.grib,ignore=(.*)\.tmp,lifetime=P2D,priority=80,asap=yes"

Quick-start example — rewrite targets per pattern¶

alias.pattern = "
  (== {(.*)\.dat}) lifetime=P2D,priority=80,asap=yes,event=no,delay=PT15M,target=/archive/$target
  (.= M) target=/mirror/$date/$target,dateformat=MMdd,datedelta=-1,datesource=$target[2..12],datepattern=yyyyMMddHH
"

mqtt.* — MQTT Publishing Options¶

These options control MQTT notifications published when destination transfer requests complete.

Quick-start example — retained JSON notifications¶

incoming.event = "yes"
mqtt.publish = "yes"
mqtt.topic = "ecpds/data/"
mqtt.qos = "1"
mqtt.contentType = "application/json"
mqtt.retain = "yes"

Building the payload with JavaScript¶

When a static mqtt.payload property is not enough, place a JavaScript snippet in the destination's JavaScript field (after the ###### END-OF-PROPERTIES ###### separator). The script is executed once per completed transfer and must return a plain object whose keys mirror the mqtt.* option names. Only the keys you return are used; everything else falls back to the properties.

Runtime variables — the following $-prefixed placeholders are substituted into the script source before execution:

Return object structure — the script must return an object with a mqtt key. Any property listed in the table above can be overridden dynamically:

return {
  mqtt: {
    topic:       "...",   // overrides mqtt.topic
    payload:     "...",   // required — the message body
    contentType: "...",   // overrides mqtt.contentType
    retain:      true,    // overrides mqtt.retain
    qos:         1,       // overrides mqtt.qos
    expiryInterval: "PT48H"  // overrides mqtt.expiryInterval
  }
};

Only mqtt.payload is strictly required; all other keys are optional overrides.

Full example — WIS2-style GeoJSON notification with multi-protocol links¶

This example generates a WIS2-compatible GeoJSON notification that includes direct download links for HTTPS, SFTP, WebDAV, and S3.

Properties (in the destination's Properties field):

incoming.event = "yes"
mqtt.contentType = "application/json"
mqtt.expiryInterval = "PT48H"
mqtt.publish = "yes"
mqtt.retain = "yes"

JavaScript (in the destination's JavaScript field):

// $-placeholders are substituted before execution
var destination = "$destination";
var filename    = "$filename";
var filesize    = $filesize;                          // integer literal
var pubtime     = new Date($timefile).toISOString();  // integer literal -> ISO string
var dataId      = destination + "/" + filename;

// Adjust host and ports for your deployment
var host      = "localhost";
var httpsPort = 7443;
var sftpPort  = 7022;

var baseUrl = "https://" + host + ":" + httpsPort + "/";
var s3Url   = "https://" + host + ":" + httpsPort + "/s3/";
var sftpUrl = "sftp://test@" + host + ":" + sftpPort + "/";

return {
  mqtt: {
    payload: JSON.stringify({
      id:       "$uuid",
      type:     "Feature",
      geometry: null,
      properties: {
        datetime:       pubtime,
        pubtime:        pubtime,
        data_id:        dataId,
        content_length: filesize
      },
      links: [
        { rel: "canonical", type: "application/octet-stream",
          length: filesize, href: baseUrl + filename,
          title: "Download via HTTPS" },
        { rel: "alternate", type: "application/octet-stream",
          length: filesize, href: sftpUrl + filename,
          title: "Download via SFTP" },
        { rel: "alternate", type: "application/octet-stream",
          length: filesize, href: s3Url + filename,
          title: "Download via S3" }
      ]
    })
  }
};

The resulting MQTT message published on topic destinationName/filename will be a GeoJSON Feature carrying three access URLs. Any subscriber can pick the protocol that suits them best.

• Data Portal use case
• Notifications (MQTT)
• OpenECPDS entities
• Acquisition options
• Host options