Gast ::
| Hauptseite > Publikationsdatenbank > FairRootGroup/DDS: 3.8 [The Dynamic Deployment System (DDS)] |
| Software | GSI-2024-00918 |
; ; ; ; ; ;
2024
Please use a persistent id in citations: doi:10.5281/ZENODO.10532685
Abstract: The Dynamic Deployment System (DDS)DDS - is a tool-set that automates and significantly simplifies a deployment of user defined processes and their dependencies on any resource management system using a given topology.Basic conceptsDDS: implements a single-responsibility-principle command line tool-set and APIs, treats users’ tasks as black boxes, doesn’t depend on RMS (provides deployment via SSH, when no RMS is present), supports workers behind FireWalls (outgoing connection from WNs required), doesn’t require pre-installation on WNs, deploys private facilities on demand with isolated sandboxes, provides a key-value properties propagation service for tasks, provides a rules based execution of tasks.Release Notes ## v3.8 (2024-01-19) DDS general Fixed: On task done remove agents from the agent to tasks mapping. Fixed: Replace std::iterator as it's deprecated (C++17). Fixed: Tasks working directory is set to their slot directory instead of DDS_LOCATION. Fixed: Multiple stability issues. Modified: support C++20 standard (GH-477). Modified: Bump minimum version requirements for cmake (from 3.11.0 to 3.19) and boost (from 1.67 to 1.75). (GH-428) Modified: C++17 modernization of EnvProp.h/env_prop. (GH-368) Added: 3rd party dependency on Protobuf (min v3.15). Added: every DDS module logs now its pid, group id and parent pid. (GH-403) Added: Support for Task Assets. (GH-406) Added: Cancel running and pending SLURM jobs on DDS shutdown. (GH-429) Added: Support for Apple's arm64 architecture. (GH-393) Added: $DDS_CONFIG and /etc/dds/DDS.cfg are added to the DDS config search paths. (GH-458) Added: DDS libraries are now decorated with an ABI version. (GH-410) dds-agent Fixed: Address potential crash in the external process termination routines. Fixed: Revised handling of the slots container. Fixed: Ignore SIGTERM while performing cleaning procedures. (GH-459) dds_intercom_lib Fixed: Stability improvements. Modified: Temporary increase intercom message size to 2048. (GH-440) Modified: Set debug log severity on Custom command events. (GH-424) dds-session Fixed: skip bad or non-session directories/files when performing clean and list operations. Added: A data retention sanitization. Not running sessions older than the specified number of days ('server.data_retention') are auto deleted. (GH-435) dds-submit Added: Users can specify a GroupName tag for each submission. This tag will be assigned to agents and can be used as a requirement in topologies. (GH-407) Added: Users can provide a Submission Tag (--submission-tag). DDS RMS plug-ins will use this tag to name RMS jobs and directories. (GH-426) Added: The command learned a new argument --env-config/-e. It can be used to define a custom environment script for each agent. (GH-430) Added: The command learned a new argument --min-instances. It can be used to provide the minimum number of agents to spawn. (GH-434) Added: The command learned a new argument --enable-overbooking. The flag instructs DDS RMS plug-ing to not specify any CPU requirement for RMS jobs. (GH-442) Added: The command learned a new argument --inline-config. Content of this string will be added to the RMS job configuration file as is. It can be specified multiple times to add multiline options. (GH-449) Modified: WN package builder timeout interval was increased from 15 to 30 sec. (GH-468) Modified: Improve validation of the WN package builder. (GH-468) dds-topology Fixed: Stability improvements. Fixed: A bug which caused dds::topology_api::CTopoCreator to ignore task assets. (GH-452) Fixed: Activating topology takes too long when task assets are used. (GH-454) Fixed: a bug, which can cause a segfault when updating variables in topology. Added: A new groupName requirement. It can be used on task and collection. (GH-407) Added: Open API to read/update/add topology variable. The CTopoVars class. Added: Support for Task Assets. (GH-406) Added: Custom type of Task and Collection requirements. (GH-445) dds-ssh-plugin Fixed: ssh cfg parser is passing cfg files of all plug-ins. (GH-413) Added: Support for SubmissionID (GH-411) dds-slurm-plugin Fixed: Make sure that scancel's SIGTERM is properly handled by all job steps and their scripts. (GH-459) Added: Support for SubmissionID (GH-411) Added: Support of minimum number of agents to spawn. (GH-434) Modified: Replace array job submission with nodes requirement. (GH-430) Modified: Remove #SBATCH --ntasks-per-node=1. (GH-444) Modified: The #SBATCH --cpus-per-task=%DDS_NSLOTS% requirement is now can be disabled by providing the 'enable-overbooking' flag (ToolsAPI or dds-submit). (GH-442) Modified: Prevent job termination when downing a single node of the job allocation. (GH-450) dds-localhost-plugin Added: Support for SubmissionID (GH-411) dds-tools-api Modified: Logs of user processes which use Tools API are moved now to the DDS root log directory, instead of sessions directory. Modified: CSession::waitForNumAgents is renamed to CSession::waitForNumSlots. (GH-439) Added: An ability to unsubscribe from either individual events or all events of requests. (GH-382) Added: SAgentInfoResponseData provides the agent group name. (GH-415) Added: SSubmitRequestData supports flags. See SSubmitRequestData::setFlag and SSubmitRequestData::ESubmitRequestFlags. (GH-442) Added: Users can define additional job RMS configuration via SSubmitRequestData::m_inlineConfig. It will be inlined as is into the final job script. (GH-449) dds-user-defaults Fixed: a dangling reference to a temporary in User Defaults class. Modified: Bump the version to 0.5. Added: A server.data_retention configuration key. (GH-435) dds-info Fixed: wrong exit code when called with --help/--version. (GH-470) dds-agent-cmd Modified: getlog: now logs are tar'ed without their source directory structure - as a flat stack of files. (GH-369) Modified: getlog: the command outputs the destination directory where downloaded archives will be stored into. Also fixed command's description. (GH-369)
Note: LGPL-3.0 license
Contributing Institute(s): Research Program(s): Experiment(s):
Appears in the scientific report 2024
|
The record appears in these collections: |