Class MoverServer

java.lang.Object
java.rmi.server.RemoteObject
java.rmi.server.RemoteServer
java.rmi.server.UnicastRemoteObject
ecmwf.common.callback.CallBackObject
ecmwf.common.ecaccess.StarterServer
ecmwf.ecpds.mover.MoverServer
All Implemented Interfaces:
ClientInterface, RemoteServer, MBeanListener, MBeanService, ToBeStarted, MoverInterface, Serializable, Remote
public final class MoverServer extends StarterServer implements MoverInterface
The Class MoverServer.
See Also:

  • Constructor Details

  • Method Details

    • computeVolumeUsage

      public long[][] computeVolumeUsage(int volumeIndexMax) throws RemoteException
      Compute the used and total capacity for each volume in the data repository.

      Returns two parallel arrays of length volumeIndexMax + 1:

      • result[0][i] — bytes used in volume i
      • result[1][i] — total bytes available for volume i

      Dedicated-disk volumes (the volume directory resides on a filesystem shared with no other volume): raw statvfs stats are used — O(1) per volume.

      Shared-disk volumes (two or more volume directories reside on the same physical filesystem, e.g. a single-machine test setup or a small production environment where all volumes live on one disk): a virtual per-volume quota is applied to give the load balancer meaningful differentiation:

      • virtual total per volume = diskTotal / numberOfVolumesOnThisDisk
      • virtual used per volume = recursive byte sum of the volume directory (listSize())
      This makes the WeightedAllocator aware of which volumes actually hold more data even though the underlying filesystem reports identical free space for all of them.

      Specified by:
      computeVolumeUsage in interface MoverInterface
      Parameters:
      volumeIndexMax - highest volume index to include (inclusive); volumes 0..volumeIndexMax are reported
      Returns:
      a 2-element array: [usedPerVolume[], maxCapacityPerVolume[]] each of length volumeIndexMax + 1
      Throws:
      RemoteException - the remote exception

    • resetDataMover

      public void resetDataMover()
      Is called whenever the DataMover lost the connection with its MasterServer. This method can be called from the MasterManager if it is connected through RMI or from the TransferRepository if it is connected through the REST interface.

    • setRESTProvider

      public void setRESTProvider(RESTProvider restProvider)
      Sets the REST provider.
      Parameters:
      restProvider - the new REST provider

    • getRESTInterface

      public RESTInterface getRESTInterface(String httpProxy, String httpMover, int connectTimeout) throws MoverException
      Gets the REST interface.
      Parameters:
      httpProxy - the http proxy
      httpMover - the http mover
      connectTimeout - the connect timeout
      Returns:
      the REST interface
      Throws:
      MoverException - the mover exception

    • setMQTTInterface

      public void setMQTTInterface(MQTTInterface mqttInterface)
      Sets the MQTT interface.
      Parameters:
      mqttInterface - the new MQTT interface

    • getMQTTInterface

      public MQTTInterface getMQTTInterface() throws MoverException
      Gets the MQTT interface.
      Returns:
      the MQTT interface
      Throws:
      MoverException - the mover exception

    • getGenericFile

      public GenericFile getGenericFile(String path)
      Gets the generic file.
      Parameters:
      path - the path
      Returns:
      the generic file

    • getDataFileAccessInterface

      public DataAccessInterface getDataFileAccessInterface() throws MasterException
      Gets the data file access interface.
      Returns:
      the data file access interface
      Throws:
      MasterException - the master exception

    • getECproxyAddressAndPort

      public String getECproxyAddressAndPort()
      Gets the ECproxy plugging listen address and port. This is used by the Master Server to update the "TransferServer" entry in the database when the Mover subscribe. This address and port are used when the Master Server send the address of the allocated Mover to the ecpds command.
      Specified by:
      getECproxyAddressAndPort in interface MoverInterface
      Returns:
      the ECproxyPlugin address and port

    • deployHttpCertificate

      public void deployHttpCertificate(byte[] pkcs12Bytes, String keystorePassword) throws RemoteException
      Deploys a new TLS certificate to this Data Mover. The certificate is provided as a PKCS#12 keystore; it replaces the current keystore on disk and the HTTPS server is hot-reloaded without dropping active connections. Deploys a new TLS certificate to this Data Mover. The PKCS#12 bytes are written to the keystore path that the running HttpPlugin already uses, then the SslContextFactory is hot-reloaded without dropping active connections.
      Specified by:
      deployHttpCertificate in interface MoverInterface
      Parameters:
      pkcs12Bytes - the PKCS#12 keystore bytes
      keystorePassword - the password for the keystore and private key
      Throws:
      RemoteException - the remote exception

    • getHttpCertificateJson

      public String getHttpCertificateJson() throws RemoteException
      Returns a JSON-encoded snapshot of the certificate currently loaded in the Data Mover HTTPS server, or an empty JSON object {} if no certificate is loaded. Returns a JSON-encoded snapshot of the certificate currently loaded in the Data Mover HTTPS server.
      Specified by:
      getHttpCertificateJson in interface MoverInterface
      Returns:
      JSON string with certificate metadata fields
      Throws:
      RemoteException - the remote exception

    • getIncomingConnectionIds

      public String[] getIncomingConnectionIds()
      Gets the incoming connection ids.
      Specified by:
      getIncomingConnectionIds in interface MoverInterface
      Returns:
      the incoming connection ids

    • getIncomingConnections

      public List<IncomingConnection> getIncomingConnections()
      Gets the incoming connections.
      Specified by:
      getIncomingConnections in interface MoverInterface
      Returns:
      the incoming connections

    • closeIncomingConnection

      public boolean closeIncomingConnection(String id)
      Close incoming connection.
      Specified by:
      closeIncomingConnection in interface MoverInterface
      Parameters:
      id - the id
      Returns:
      true, if successful

    • invalidatePortalSessionsForUser

      public int invalidatePortalSessionsForUser(String user)
      Invalidate all portal HTTPS session tokens for the given user, forcing re-authentication on the next browser request. Called when the user's configuration changes (e.g. Portal Service mode).
      Specified by:
      invalidatePortalSessionsForUser in interface MoverInterface
      Parameters:
      user - the data user login name
      Returns:
      the number of sessions removed

    • invalidatePortalSession

      public void invalidatePortalSession(String token)
      Invalidate a single portal session token in this mover's local session cache, if present. Used by the Master Server to propagate a user-initiated logout to all other movers that may have imported the token via cross-mover session migration. Invalidate a single portal session token in the local cache. Called by the Master when the user logs out on another mover that previously exported the token to this one via cross-mover session migration.
      Specified by:
      invalidatePortalSession in interface MoverInterface
      Parameters:
      token - the portal session token to invalidate

    • resolvePortalSession

      public String resolvePortalSession(String token)
      Look up a portal session token in this mover's local session cache. Used by the Master Server to locate a token that was originally issued on a different mover (e.g. after a load-balancer failover).

      If the token is found and has not expired, the session entry is serialised as a single-line string in the format: 

      user TAB subscriberId TAB subscriberEmail TAB expiryEpochMs
      The caller is responsible for importing the entry into its own local cache.

      Serialise a local portal session entry so the Master can migrate it to another mover after a load-balancer failover.
      Specified by:
      resolvePortalSession in interface MoverInterface
      Parameters:
      token - the portal session token to resolve
      Returns:
      the serialised session entry, or null if not found or expired

    • closeAllIncomingConnections

      public void closeAllIncomingConnections()
      Close all incoming connections.
      Specified by:
      closeAllIncomingConnections in interface MoverInterface

    • publishToMQTTBroker

      public void publishToMQTTBroker(String topic, int qos, long expiryInterval, String contentType, String clientId, String payload, boolean retain)
      Publish a notification to the registered MQTT service (if any).
      Specified by:
      publishToMQTTBroker in interface MoverInterface
      Parameters:
      topic - the topic
      qos - the qos
      expiryInterval - the expiry interval
      contentType - the content type
      clientId - the client id
      payload - the payload
      retain - the retain

    • removeFromMQTTBroker

      public void removeFromMQTTBroker(String topic)
      Remove a retained notification from the MQTT broker.
      Specified by:
      removeFromMQTTBroker in interface MoverInterface
      Parameters:
      topic - the topic

    • getMQTTClientsCount

      public int getMQTTClientsCount()
      Get the number of client connected to the MQTT broker.
      Specified by:
      getMQTTClientsCount in interface MoverInterface
      Returns:
      the MQTT clients count

    • getRepository

      public static String getRepository()
      Gets the repository.
      Returns:
      the repository

    • getRoot

      public String getRoot()
      Gets the root.
      Specified by:
      getRoot in interface ClientInterface
      Returns:
      the root

    • getService

      public String getService()
      Gets the service.
      Specified by:
      getService in interface ClientInterface
      Returns:
      the service

    • getPassword

      public String getPassword()
      Gets the password.
      Specified by:
      getPassword in interface ClientInterface
      Returns:
      the password

    • getVersion

      public String getVersion()
      Gets the version.
      Specified by:
      getVersion in class StarterServer
      Returns:
      the version

    • getMBeanInfo

      public MBeanInfo getMBeanInfo()
      Gets the MBean info.
      Specified by:
      getMBeanInfo in interface MBeanService
      Overrides:
      getMBeanInfo in class StarterServer
      Returns:
      the MBean info

    • setAttribute

      public boolean setAttribute(String name, Object value) throws InvalidAttributeValueException, MBeanException
      Sets the attribute.
      Specified by:
      setAttribute in interface MBeanService
      Overrides:
      setAttribute in class StarterServer
      Parameters:
      name - the name
      value - the value
      Returns:
      true, if successful
      Throws:
      InvalidAttributeValueException - the invalid attribute value exception
      MBeanException - the MBean exception

    • getAttribute

      public Object getAttribute(String attributeName) throws AttributeNotFoundException, MBeanException
      Gets the attribute.
      Specified by:
      getAttribute in interface MBeanService
      Overrides:
      getAttribute in class StarterServer
      Parameters:
      attributeName - the attribute name
      Returns:
      the attribute
      Throws:
      AttributeNotFoundException - the attribute not found exception
      MBeanException - the MBean exception

    • invoke

      public Object invoke(String operationName, Object[] params, String[] signature) throws NoSuchMethodException, MBeanException
      Invoke.
      Specified by:
      invoke in interface MBeanService
      Overrides:
      invoke in class StarterServer
      Parameters:
      operationName - the operation name
      params - the params
      signature - the signature
      Returns:
      the object
      Throws:
      NoSuchMethodException - the no such method exception
      MBeanException - the MBean exception

    • check

      public void check(long ticket) throws RemoteException
      Check.
      Specified by:
      check in interface MoverInterface
      Parameters:
      ticket - the ticket
      Throws:
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • size

      public long size(DataTransfer transfer, String fileName) throws RemoteException
      Size.
      Specified by:
      size in interface MoverInterface
      Parameters:
      transfer - the transfer
      fileName - the file name
      Returns:
      the long
      Throws:
      ECtransException - the ectrans exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • del

      public void del(DataTransfer transfer, String fileName) throws RemoteException
      Del.
      Specified by:
      del in interface MoverInterface
      Parameters:
      transfer - the transfer
      fileName - the file name
      Throws:
      ECtransException - the ectrans exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • filter

      public DataFile filter(DataFile dataFile, boolean remove) throws RemoteException
      Filter.
      Specified by:
      filter in interface MoverInterface
      Parameters:
      dataFile - the data file
      remove - the remove
      Returns:
      the data file
      Throws:
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • replicate

      public DataFile replicate(DataFile dataFile, Host targetHost, Host[] hostsForSource) throws RemoteException
      Replicate.
      Specified by:
      replicate in interface MoverInterface
      Parameters:
      dataFile - the data file
      targetHost - the target host
      hostsForSource - the hosts for source
      Returns:
      the data file
      Throws:
      ECtransException - the ectrans exception
      SourceNotAvailableException - the source not available exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • getMoverReport

      public String getMoverReport(Host proxyHost) throws RemoteException
      Gets the mover report.
      Specified by:
      getMoverReport in interface MoverInterface
      Parameters:
      proxyHost - the proxy host
      Returns:
      the mover report
      Throws:
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • getHostReport

      public String getHostReport(Host proxyHost, Host host) throws RemoteException
      Gets the host report.
      Specified by:
      getHostReport in interface MoverInterface
      Parameters:
      proxyHost - the proxy host
      host - the host
      Returns:
      the host report
      Throws:
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • getReport

      public String getReport() throws RemoteException
      Gets the report.
      Specified by:
      getReport in interface MoverInterface
      Returns:
      the report
      Throws:
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • getReport

      public String getReport(Host host) throws RemoteException
      Gets the report.
      Specified by:
      getReport in interface MoverInterface
      Parameters:
      host - the host
      Returns:
      the report
      Throws:
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • download

      public DataFile download(DataFile dataFile, Host hostForSource) throws RemoteException
      Download.
      Specified by:
      download in interface MoverInterface
      Parameters:
      dataFile - the data file
      hostForSource - the host for source
      Returns:
      the data file
      Throws:
      SourceNotAvailableException - the source not available exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • purge

      public void purge(Host proxyHost, List<ExistingStorageDirectory> directories)
      Purge.
      Specified by:
      purge in interface MoverInterface
      Parameters:
      proxyHost - the proxy host
      directories - the directories
      Throws:
      RemoteException - the remote exception

    • purge

      public void purge(List<ExistingStorageDirectory> directories)
      Purge.
      Specified by:
      purge in interface MoverInterface
      Parameters:
      directories - the directories
      Throws:
      RemoteException - the remote exception

    • purge

      public void purge()
      Request from the JMX interface.

    • del

      public boolean del(Host proxyHost, DataFile dataFile) throws RemoteException
      Del.
      Specified by:
      del in interface MoverInterface
      Parameters:
      proxyHost - the proxy host
      dataFile - the data file
      Returns:
      true, if successful
      Throws:
      ECtransException - the ectrans exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • del

      public boolean del(DataFile dataFile) throws RemoteException
      Del.
      Specified by:
      del in interface MoverInterface
      Parameters:
      dataFile - the data file
      Returns:
      true, if successful
      Throws:
      ECtransException - the ectrans exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • put

      public DataTransfer put(Host[] hostsForSource, DataTransfer transfer, String targetName, long localPosn, long remotePosn) throws RemoteException
      Puts the.
      Specified by:
      put in interface MoverInterface
      Parameters:
      hostsForSource - the hosts for source
      transfer - the transfer
      targetName - the target name
      localPosn - the local posn
      remotePosn - the remote posn
      Returns:
      the data transfer
      Throws:
      ECtransException - the ectrans exception
      SourceNotAvailableException - the source not available exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • put

      public ProxySocket put(Host host, String target, long remotePosn, long size) throws RemoteException
      Allow getting a ProxySocket to upload a file on a target host. This method should be used only when the file is to be uploaded from a different data mover than the current one.
      Specified by:
      put in interface MoverInterface
      Parameters:
      host - the host
      target - the target
      remotePosn - the remote posn
      size - the size
      Returns:
      the proxy socket
      Throws:
      ECtransException - the ectrans exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • get

      public ProxySocket get(Host host, String source, long remotePosn, boolean removeOriginal) throws RemoteException
      Allow getting a ProxySocket to retrieve a file on a target host. This method should be used only when the file is to be retrieved from a different data mover than the current one.
      Specified by:
      get in interface MoverInterface
      Parameters:
      host - the host
      source - the source
      remotePosn - the remote posn
      removeOriginal - the remove original
      Returns:
      the proxy socket
      Throws:
      ECtransException - the ectrans exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • get

      public MoverServer.ECproxyCallback get(OutputStream out, Host host, String source, long remotePosn, DataFile dataFile) throws ECtransException
      Allow providing an output stream to retrieve a file. This method should only be used when it is called from the current data mover. The retrieval is done asynchronously. If the retrieval fail then the provided Output Stream is closed.
      Parameters:
      out - the output stream
      host - the host
      source - the source
      remotePosn - the remote posn
      dataFile - the data file
      Returns:
      the ecproxy callback for checking the outcome of the transmission
      Throws:
      ECtransException - the ectrans exception

    • get

      public ProxySocket get(DataFile dataFile, Host hostForSource, long remotePosn) throws RemoteException
      Gets the.
      Specified by:
      get in interface MoverInterface
      Parameters:
      dataFile - the data file
      hostForSource - the host for source
      remotePosn - the remote posn
      Returns:
      the proxy socket
      Throws:
      SourceNotAvailableException - the source not available exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • get

      public ProxySocket get(DataFile dataFile, Host[] hostsForSource, long remotePosn, long length) throws RemoteException
      Gets the.
      Specified by:
      get in interface MoverInterface
      Parameters:
      dataFile - the data file
      hostsForSource - the hosts for source
      remotePosn - the remote posn
      length - the length
      Returns:
      the proxy socket
      Throws:
      SourceNotAvailableException - the source not available exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • size

      public long size(Host host, String source) throws RemoteException
      Size.
      Specified by:
      size in interface MoverInterface
      Parameters:
      host - the host
      source - the source
      Returns:
      the long
      Throws:
      ECtransException - the ectrans exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • listAsStringArray

      public String[] listAsStringArray(Host host, String directory, String pattern) throws RemoteException
      List as string array.
      Specified by:
      listAsStringArray in interface MoverInterface
      Parameters:
      host - the host
      directory - the directory
      pattern - the pattern
      Returns:
      the string[]
      Throws:
      ECtransException - the ectrans exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • listAsByteArray

      public RemoteInputStream listAsByteArray(Host host, String directory, String pattern, boolean synchronous) throws RemoteException
      List as byte array.
      Specified by:
      listAsByteArray in interface MoverInterface
      Parameters:
      host - the host
      directory - the directory
      pattern - the pattern
      synchronous - the synchronous
      Returns:
      the remote input stream
      Throws:
      ECtransException - the ectrans exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • execute

      public RemoteInputStream execute(String script) throws RemoteException
      Execute.
      Specified by:
      execute in interface MoverInterface
      Parameters:
      script - the script
      Returns:
      the remote input stream
      Throws:
      ScriptException - the script exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • fetchUrlContent

      public String fetchUrlContent(Host host, String source, int maxBytes) throws RemoteException
      Fetch the raw content of a remote URL/path via the host's configured ECtrans module. Returns the content as a plain String (truncated to maxBytes if needed). Fetch the raw content of a remote URL/path via the host's configured ECtrans module. Uses the same ECtransContainer/StreamRepository path as production file retrieval.
      Specified by:
      fetchUrlContent in interface MoverInterface
      Parameters:
      host - the host whose ECtrans module is used for retrieval
      source - the remote URL or path to retrieve
      maxBytes - maximum number of bytes to return
      Returns:
      the content as a String
      Throws:
      RemoteException - the remote exception

    • del

      public void del(Host host, String source) throws RemoteException
      Del.
      Specified by:
      del in interface MoverInterface
      Parameters:
      host - the host
      source - the source
      Throws:
      ECtransException - the ectrans exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • mkdir

      public void mkdir(Host host, String dir) throws RemoteException
      Mkdir.
      Specified by:
      mkdir in interface MoverInterface
      Parameters:
      host - the host
      dir - the dir
      Throws:
      ECtransException - the ectrans exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • rmdir

      public void rmdir(Host host, String dir) throws RemoteException
      Rmdir.
      Specified by:
      rmdir in interface MoverInterface
      Parameters:
      host - the host
      dir - the dir
      Throws:
      ECtransException - the ectrans exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • move

      public void move(Host host, String source, String target) throws RemoteException
      Move.
      Specified by:
      move in interface MoverInterface
      Parameters:
      host - the host
      source - the source
      target - the target
      Throws:
      ECtransException - the ectrans exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • closeDataTransfer

      public boolean closeDataTransfer(DataTransfer transfer) throws IOException
      Close.
      Parameters:
      transfer - the transfer
      Returns:
      true, if successful
      Throws:
      ECtransException - the ectrans exception
      IOException - Signals that an I/O exception has occurred.

    • close

      public boolean close(DataTransfer transfer) throws RemoteException
      Close.
      Specified by:
      close in interface MoverInterface
      Parameters:
      transfer - the transfer
      Returns:
      true, if successful
      Throws:
      ECtransException - the ectrans exception
      IOException - Signals that an I/O exception has occurred.
      RemoteException

    • getMasterProxy

      public MasterProxy getMasterProxy() throws MoverException
      If the data mover is connected directly to its MasterServer then return a RMI interface to the Master, if the data mover is connected through a delegate data mover then return a REST interface, otherwise through an exception.
      Returns:
      the master proxy
      Throws:
      MoverException - the mover exception

    • getMasterInterface

      public MasterInterface getMasterInterface() throws ConnectionException
      Gets the master interface.
      Returns:
      the master interface
      Throws:
      ConnectionException - the connection exception

    • getECauthToken

      public ECauthToken getECauthToken(String user) throws IOException
      Gets the ecauth token.
      Parameters:
      user - the user
      Returns:
      the ecauth token
      Throws:
      IOException - Signals that an I/O exception has occurred.

    • getTicketRepository

      public TicketRepository getTicketRepository()
      Gets the ticket repository.
      Returns:
      the ticket repository

    • getDownloadRepository

      public MoverServer.DownloadRepository getDownloadRepository()
      Gets the download repository.
      Returns:
      the download repository

    • shutdown

      public void shutdown()
      Shutdown.
      Specified by:
      shutdown in interface ToBeStarted
      Overrides:
      shutdown in class StarterServer