Data Mover: ecmwf.properties¶

The Mover's ecmwf.properties configures its Master connection, public protocol listeners, data repository, retrieval behaviour and shared runtime. This reference includes options read by the implementation even if they are absent from the supplied template.

See also the Master and Monitor references.

The optional ECaccess transfer module requires Master RMI connectivity and the existing ECproxyPlugin callback listener. Its gateway properties belong to Hosts, not a new Mover provider section.

File location and overrides¶

The source template is etc/mover/conf/ecmwf.properties. Its RPM path is /usr/local/ecpds/mover/conf/ecmwf.properties; the launcher selects it with the ecmwf.properties JVM property.

Use /etc/ecpds/mover.cnf or the container/service environment for launcher settings such as addresses, ports, operational mode and shared secrets. The template's ${...} expressions resolve JVM properties, section references or supported methods, not arbitrary environment variables.

The final #include "${mover.etc}/ecmwf.properties" loads local overrides, normally from /etc/ecpds/mover/ecmwf.properties. This is an active directive, not a commented-out example. Do not include the file into itself. See the configuration syntax and loading rules.

For example:

[HttpPlugin]
maxThreads=200
minThreads=20
webdavEnabled=false

[SshPlugin]
maxConcurrentSessions=100

Restart the affected Mover after changing service configuration unless a feature explicitly supports live updates.

Reading this reference¶

Code fallback means the reader's value when the option is absent, not the explicit shipped value. A dash means no assignment in the template. null means the reader supplies no fallback; it does not guarantee that omitting the setting is valid. Different reader fallbacks are listed separately, with source links. Java expressions are preserved for context-dependent defaults. Credentials are not reproduced.

Rows marked template entry have no direct literal reader in this process's reviewed sources. Some entries are substitutions or retained compatibility settings; named-socket and open-map readers are described separately.

Names are case-sensitive. Duration-reading options accept suffixes such as s, m, h and d; bare configured durations mean hours. Plain numeric socket timeouts are milliseconds. Rates in [TestModule] are bytes per second; file-content settings take a path, while their fallback is content.

Operational guidance¶

Public listeners, portal links and control-channel authentication are separate:

• [PluginList] selects services to start; plugin sections configure them. Advertising a protocol in [DataPortal] does not start its listener.
• [HttpPlugin] serves the Data Portal, REST control channel, optional S3 endpoint and WebDAV. Restricting public portal access does not replace control-channel authentication.
• [Security] rccSharedSecret signs REST control requests between relays and regular Movers. cliSharedSecret protects a different direct-socket connection; do not confuse the two.
• [MoverProxy] configures a control-plane HTTP tunnel. It is not an OpenTelemetry exporter or a generic outbound transfer proxy.
• [ProxySocket] plainText=yes disables TLS on that socket. Keep internal traffic on trusted networks and review TLS verification before exposing it.
• [Mover] operational affects work availability and test behaviour. The [TestModule] options are not normal production transfer-rate limits.

Movers follow the Master's transfer-tracing decision and forward completed spans through the existing control connection. They need no OpenTelemetry Collector endpoint, credentials or SDK configuration; see Event Logging.

For transfer-module settings and per-user portal policy, use the linked references in the dynamic-families section below. Those defaults can affect many transfers, so prefer a Host or User override when only one entity needs a different policy.

Option catalogue¶

Shared components and optional modules consume their settings only when used; listing an option does not activate that component. Named socket sections inherit [SocketConfig], then apply their own overrides.

[CallBack]¶

Exported callback endpoint. serverFactory and clientFactory are Java socket-factory classes. This section also accepts the named socket overrides described below.

[Command]¶

External helper commands. ecauth overrides the token-generation executable; sh selects the ECios shell command.

[DataPortal]¶

Public endpoint hints and protocol visibility in portal pages. Empty public URLs/hosts let the portal derive values from the request/listener. These switches do not enable or disable the actual FTP, SFTP or S3 listeners. Per-user portal settings can override global hints.

[ECauthModule]¶

ECauth connection and host-list cache. Durations control connection timeout, refresh polling, refresh frequency and cached-entry lifetime. Credentials here identify the ECauth service account.

[ECios]¶

ECios helper start/kill socket timeouts, in milliseconds. Also accepts named socket overrides.

[ECpdsClient]¶

ECpds socket-client read timeout in milliseconds. This is distinct from the server-side plugin timeout.

[ECpdsPlugin]¶

Master ECpds listener and request-processing behaviour. checkPort requires a privileged client source port when enabled. SocketSoTimeout applies to accepted request sockets in milliseconds. resetDataTransferSchedulesByGroup resets schedules on group completion. The section also accepts named socket options.

[ECproxyPlugin]¶

Mover ECproxy data listener and repository. Explicit TCP options override the accepted socket's existing values. Buffer sizes are bytes; socket timeouts are milliseconds. Also accepts named socket settings.

[ExecModule]¶

Defaults for the executable transfer module. script runs after file reception; dir falls back to the transfer directory. synchronous waits for command completion, returnCode is its accepted exit status, and usePipe supplies arguments on standard input. mkdirs creates parent directories and usetmp stages reception in a temporary file. Empty prefix and suffix select an effective .tmp suffix.

[ExecutorManager]¶

Opt-in virtual-thread execution for the shared executor. This is not a general scheduler concurrency limit.

[FTPActiveSocket]¶

FTP server active-data sockets. Accepts every named socket option below, in addition to these direct readers.

[FTPPassiveSocket]¶

FTP server passive-data sockets. boundAddress controls the address used for passive binding. Also accepts named socket settings, including local-port ranges.

[Filter]¶

Streaming compression/decompression command definitions. The four fly.* keys are read by the stream manager. The template also retains pre.out.* definitions, but the current source has no direct reader for those entries.

[FtpDomainList]¶

Virtual FTP/SFTP domain mappings; arbitrary domain names are supported. * is the fallback. Domain mappings select backing filesystem/domain implementations, not network listener addresses.

[FtpModuleList]¶

FTP extension-module lists. Keys select extensions; * supplies the default list. Values are comma-separated module definitions consumed by the FTP utility.

[FtpPlugin]¶

FTP control listener and behaviour. welcome, browser and help are file paths; tail is response text. passive, rest and extended affect FTP features. hidden is a list of hidden virtual domains. alwaysCheckOnRETR requests retrieval-time checks. The section also accepts named socket options.

[HttpPlugin]¶

Jetty listener, pool, protocol and portal settings. Thread limits are counts; idleThreadsTimeout and idleTimeout are milliseconds; buffer/header limits are bytes. http/https=-1 leaves that connector disabled. On Movers, S3 skew is seconds and S3 request limits are bytes; CORS and DNS mappings apply to portal paths. On the Monitor, only the shared Jetty/HTTP2 readers listed here apply; its HTTPS listener is in [MonitorPlugin].

[HttpPluginSSL]¶

Optional explicit HTTPS store configuration. If this section exists, keyStorePath and keyStorePassword must be supplied: the normal [Security] fallback branch is not used. Trust-store values default to the selected key store; store type defaults to PKCS12.

[IO]¶

Local repository I/O implementation. byteBufferSizeInMb sets the buffer size in MiB; pool idle age is seconds. Direct buffers and pooling only apply when the byte-buffer path is used. channelForce, channelForceMetadata and fdSync govern persistence/sync behaviour; changing them trades durability and performance.

[IncomingUsersManagementThread]¶

Incoming-user synchronization polling (delay, milliseconds) and maximum wait duration.

[InterruptibleRMIServerSideSocket]¶

Cleanup interval for server-side interruptible RMI socket state, in minutes.

[InterruptibleRMISocket]¶

Interruptible RMI stream-buffer pooling. Buffer size is bytes and buffer TTL is minutes.

[Login]¶

Component identity, registration credential and addresses. root defaults to the hostname in normal service readers. Several utility readers use their own hostname/service fallback, so removing identity settings is not recommended. External/internal addresses are primarily consumed through template references and named socket sections.

[LoginManagement]¶

Failed-login throttling and ban tracking. active enables enforcement; max is consecutive failures. timer, wait and the login-path delay are seconds. The cleanup thread also reads delay as a raw millisecond polling interval with a different fallback; both readers are shown. whitelist bypasses these checks for matching entries.

[MasterREST]¶

Continental/relay Master REST update interval, in milliseconds.

[MasterServer]¶

Master host and RMI registry port used by a regular Mover.

[Monitor]¶

External service-status integration, independent of the web monitoring UI. activated enables it, debug logs submissions and dontSend suppresses delivery. delay is a duration: the template's bare 120 means 120 hours, whereas the code fallback is five minutes.

[Mover]¶

Operational availability and whether successful processing removes the uncompressed working file.

[MoverProvider]¶

Mover-side provider cache and portal-session caching. Timeout/TTL values are milliseconds.

[MoverProxy]¶

HTTP control-plane tunnel endpoints and connection timeout in milliseconds. trustAllCerts=true disables certificate verification for this REST client; do not treat that default as a production recommendation.

[MqttPlugin]¶

Embedded MQTT/MQTTS broker listener ports, configuration and persistent data paths. -1 leaves a connector unconfigured. Expiry settings are durations. priority and startPriority order the embedded broker extension and its startup; they are not Java thread priorities.

[NetworkWeather]¶

Network Weather admission/control limits. Values are counts except maxConfigurationBytes and maxControlResponseBytes, which are bytes. The Monitor controls activation; these settings bound accepted work rather than enabling probes.

[Other]¶

Shared utilities. buffer is an I/O buffer size in bytes; ticketWaitDuration is a raw millisecond timeout. ecauthTokenCache enables token caching. dontCheckIp bypasses the provider's client-address check during subscription. tmp is retained in templates; the launcher separately sets the JVM temporary directory.

[PluginList]¶

Template-defined configuration; see the reader links for its consuming component.

[Ports]¶

Template port substitutions. Most keys are only referenced by listener sections and therefore have no independent code default. ecpds also has a direct client-reader fallback.

[ProxySocket]¶

Proxy data/control socket settings. plainText chooses non-TLS transport and otherwise depends on the connection's addresses. host/port select the control endpoint; dataHost/dataPort can select the data endpoint separately. cipherSuites is the TLS suite list. Also accepts named socket options.

[RESTClient]¶

REST client diagnostics. Debug logging can expose request metadata; enable only for troubleshooting.

[RMIClientSocketFactory]¶

Template-defined configuration; see the reader links for its consuming component.

[RMIServerSocketFactory]¶

Template-defined configuration; see the reader links for its consuming component.

[ReplicationHost]¶

Template-defined configuration; see the reader links for its consuming component.

[ReportCommand]¶

Executable commands for Mover and remote-Host reports. Host commands expand $host, $port and $listenAddress.

[ResourceTracker]¶

Shared runtime-resource diagnostics. defaultDebugFrequency counts resource-close events between debug reports; it is not a time interval.

[RetrievalInputStream]¶

Retrieval stream buffering and cancellation, plus progress-monitor update duration. A negative monitored fallback leaves periodic monitoring unconfigured.

[Scheduler]¶

Worker cadence, concurrency and watchdogs. *JammedTimeout, maximumDuration* and timeOut* read durations; a negative timeout leaves that execution limit unset. *TimeRanges accepts configured execution windows. max*Threads values are concurrency counts; cacheRatio scales prefetch. The legacy purgeDataBase/updateECUsers intervals are hours, with optional clock-time lists.

[ScriptManager]¶

JavaScript/Graal engine settings. runningTimeout and longRunningTime are milliseconds. resourceLimits=0 leaves the statement limit disabled. Shared engines, compilation and virtual-thread execution are independent switches.

[Security]¶

Authentication, secret protection, socket TLS and optional HTTPS store defaults. SSL store readers can fall back to JVM javax.net.ssl.* properties; HTTPS has its own fallback path. JCE settings protect stored secrets and are not HTTPS cipher settings. checkCertificates=no disables certificate checks. See the TLS and control-channel guides before changing these values.

[Server]¶

Per-process service switches and local paths. Master readers enable schedulers/repositories; Mover readers enable its own repositories/accounting. dontRecord* suppresses management-action history, not native telemetry globally. Globe cache settings belong to the Monitor. providerMBeanTimeout is milliseconds; topology probe timeout is a duration.

[SessionCache]¶

Shared session cache logging and minimum pause (milliseconds). Debug frequency is a diagnostic event-count interval.

[SocketConfig]¶

Defaults for sockets. Buffers are bytes (-1 leaves OS defaults), timeouts are milliseconds, backlog is pending connections, and keepAlive is TCP keepalive rather than application connection pooling. Named socket sections override these values; see the complete named-option list below.

[SshPlugin]¶

SSH/SFTP listener and server identity. keyPairsDir stores host keys, banner is a file path and softwareVersion is the advertised server string. maxConcurrentSessions=0 means unlimited. hidden controls virtual-domain visibility.

[StorageRepository]¶

Asynchronous persistence. *Size means storage-worker thread count, not queue capacity or batch size. *Delay, idle waits and ticket timeouts are milliseconds. *MaxAuthorisedSize caps repository entries (0 leaves the cap unset). Options only apply to repository types instantiated by this process.

[StreamPlug]¶

Shared stream-copy buffer size in bytes.

[TOTP]¶

OTP authentication provider configuration, including token endpoint, separate password/passcode client credentials, grant type and expected HTTP status. active=false disables this provider path; the login-form selector does not enable it.

[TestModule]¶

Synthetic transfer behaviour for non-operational Mover tests. Error frequency is a count interval, initial delays are milliseconds and rates are bytes/s. Template-only upload settings have no direct readers in the current Mover source.

[ThreadService]¶

Shared thread runtime. Virtual threads are opt-in; debugThreadLocal enables thread-local diagnostics.

Named socket sections¶

SocketConfig.load reads the following keys from a component's named section. Common section names for this service are CallBack, RMIServerSocketFactory, RMIClientSocketFactory, FtpPlugin, FTPPassiveSocket, FTPActiveSocket, ECproxyPlugin, ProxySocket, ECauthModule, FtpPASV, FtpPORT, FtpOPEN, HTTPSocketConfig, GCSSocketConfig, S3SocketConfig, WebDavModule, JSftp, TelnetWrapper and ECios. A transfer module or plugin must actually use that socket implementation for its settings to apply; Jetty HTTP listeners and third-party transfer clients do not automatically honour every low-level socket option.

Basic options inherit the values in [SocketConfig] shown above; the advanced TCP options from tcpCongestion onward default to unset and are read from the named section, not from the global [SocketConfig] defaults. Advanced options depend on OS/socket support. Caller-supplied addresses, ports and module/Host settings may override socket defaults.

Constructed HTTPS/TLS keys¶

The HTTPS helper constructs [Security] SSL<Key> names and otherwise consults the corresponding javax.net.ssl.<key> JVM property. Its actual keys are SSLKeyStore, SSLKeyStorePassword, SSLKeyStoreType, SSLTrustStorePath, SSLTrustStorePassword and SSLTrustStoreType.

SSLTrustStorePath is the HTTPS helper's spelling; the generic socket TLS reader separately uses SSLTrustStore. Do not assume these are aliases. The explicit [HttpPluginSSL] map takes precedence over the HTTPS helper's normal store-selection branch. MQTT uses [Security] SSLKeyStore and SSLKeyStorePassword through its own helper.

FTP mapping and per-user families¶

Streaming filter executables must be installed on the machine executing them. Adding a definition does not apply it to every transfer; the current stream manager selects the named lbzip2 or zstd command when using that format.

Plugin and module families¶

[PluginList] is an open map of plugin names to definitions:

[PluginList]
name=implementation.Class@/path/to/plugin,option=value

The library path and per-plugin options depend on the plugin. Server plugins support maxConnections and inverseResolution; deleting or adding a plugin definition changes which services start. These are options inside the value, not new top-level ecmwf.properties groups. See Plugin Infrastructure.

[ECtrans] is an open family of global transfer-option defaults, with keys module.option (for example, ftp.passive). Its option set is defined by ECtransOptions and the selected module, not a fixed list of literal Cnf keys. Entity/transfer setup values can override these defaults. The complete option/default references are:

• ECtrans common options and the protocol pages under Transfer Modules.
• Host options, Destination options, Web User options and Data User options.

These entity options are not interchangeable with service sections. Some options are only meaningful for a particular module or execution role.

WebDAV initialization parameters¶

The Mover's WebDAV servlet forwards initialization-parameter lookups to [HttpPlugin] by name. This is an extension family rather than a fixed OpenECPDS enum; unset values defer to the bundled Jackrabbit servlet's defaults. For example, Jackrabbit's missing-auth-mapping parameter is distinct from the OpenECPDS webdavPath listener setting. Only parameters actually read by the bundled servlet apply; arbitrary keys do not become new features.