[0KRunning with gitlab-runner 15.5.0 (0d4137b8)[0;m [0K on BB5 map runner pnPo3yJy[0;m section_start:1670427555:resolve_secrets [0K[0K[36;1mResolving secrets[0;m[0;m section_end:1670427555:resolve_secrets [0Ksection_start:1670427555:prepare_executor [0K[0K[36;1mPreparing the "custom" executor[0;m[0;m [0KUsing Custom executor with driver BB5 PROD runner v0.0.3...[0;m BB5 PROD runner running on bbpv1.epfl.ch, version 15.5.0, user TMPDIR is /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/tmp/custom-executor3423088440, slurm job id , CUSTOM_ENV_CI_RUNNER_TAGS is ["bb5_map"] uid=0(root) gid=0(root) groups=0(root) root UID is 0 Runner ID 29, project root hpc, project name coreneuron Pipeline ID 89228, build ref 8f27b92ff41054643c2d05571e9854847f1e3a46, job ID 463633 Build dir /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P89228, optional exclusive flag , optional cpus per task flag --cpus-per-task=8, optional qos flag CUSTOM_ENV_CI_RUNNER_TAGS is ["bb5_map"] uid=0(root) gid=0(root) groups=0(root) root UID is 0 A slurm job will be created with name GL_J463633_PROD_P112_CP4_C49 Job parameters: memory=76G, cpus_per_task=8, duration=1:00:00, constraint=cpu ntasks=2 account=proj9998 user=bbpcihpcproj12 partition=prod qos= Not executing the chown -R sbatch: INFO: Activating auto partition selection plugin, please report errors to HPC/CS sbatch: INFO: Job specifies cpu constraint, setting --constraint=[skl|clx] Submitted batch job 1055872 job state: PD job state: PD job state: PD job state: PD job state: PD job state: PD job state: PD job state: PD job state: PD job state: PD job state: PD job state: PD job state: PD job state: R sbatch: sbatch -p prod -A proj9998 --ntasks=2 --cpus-per-task=8 --mem=76G --job-name=GL_J463633_PROD_P112_CP4_C49 -C cpu --no-requeue -D /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P89228 --time=1:00:00 --wrap="sleep infinity" srun: srun --mpi=none --chdir=/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P89228 --ntasks=2 --jobid=1055872 --cpus-per-task=8 --mem=76G section_end:1670427570:prepare_executor [0Ksection_start:1670427570:prepare_script [0K[0K[36;1mPreparing environment[0;m[0;m Using git from spack modules Running on r1i4n28 via bbpv1.epfl.ch... section_end:1670427575:prepare_script [0Ksection_start:1670427575:get_sources [0K[0K[36;1mGetting source from Git repository[0;m[0;m Using git from spack modules [32;1mSkipping Git repository setup[0;m [32;1mSkipping Git checkout[0;m [32;1mSkipping Git submodules setup[0;m section_end:1670427575:get_sources [0Ksection_start:1670427575:restore_cache [0K[0K[36;1mRestoring cache[0;m[0;m Using git from spack modules [32;1mChecking cache for build:coreneuron:nmodl:intel:shared:debug-8-non_protected...[0;m Runtime platform [0;m arch[0;m=amd64 os[0;m=linux pid[0;m=69450 revision[0;m=58ba2b95 version[0;m=14.2.0 cache.zip is up to date [0;m [32;1mSuccessfully extracted cache[0;m section_end:1670427578:restore_cache [0Ksection_start:1670427578:download_artifacts [0K[0K[36;1mDownloading artifacts[0;m[0;m Using git from spack modules [32;1mDownloading artifacts for build:nmodl (463630)...[0;m Runtime platform [0;m arch[0;m=amd64 os[0;m=linux pid[0;m=69529 revision[0;m=58ba2b95 version[0;m=14.2.0 Downloading artifacts from coordinator... ok [0;m id[0;m=463630 responseStatus[0;m=200 OK token[0;m=NSqpX62a section_end:1670427579:download_artifacts [0Ksection_start:1670427579:step_script [0K[0K[36;1mExecuting "step_script" stage of the job script[0;m[0;m [0;33mWARNING: Starting with version 14.0 the 'build_script' stage will be replaced with 'step_script': https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26426[0;m Using git from spack modules [32;1m$ SPACK_BUILD="${PWD}/spack-build"[0;m [32;1m$ env -0 | sort -z | xargs -0 -L 1 echo > initial_environment.env[0;m [32;1m$ . ${SPACK_ROOT}/share/spack/setup-env.sh[0;m [32;1m$ export SPACK_USER_CONFIG_PATH=${PWD}/spack-config[0;m [32;1m$ mkdir ${SPACK_USER_CONFIG_PATH}[0;m [32;1m$ cat > ${SPACK_USER_CONFIG_PATH}/config.yaml << END_SCRIPT # collapsed multi-line command[0;m [32;1m$ spack config blame config[0;m --- config: /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463633/spack-config/config.yaml:2 build_stage: /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463633/spack-config/config.yaml:3 - /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463633/spack-build /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463633/spack-config/config.yaml:4 source_cache: /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463633/spack-source-cache /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/config.yaml:2 ccache: True /gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:2 install_tree: /gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:3 root: $user_cache_path/software /gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:4 projections: /gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:5 all: install_{compiler.name}-{compiler.version}-{target}/{name}-{version}-{hash:6} /gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:6 module_roots: /gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:7 tcl: $user_cache_path/modules /gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:9 environments: /gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:10 root: $user_cache_path/environments /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:88 concretization: separately /gpfs/bbp.cscs.ch/ssd/apps/bsd//config/config.yaml:12 build_jobs: 8 /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:18 extensions: /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:18 - $spack/bluebrain/spack-scripting /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:19 # This is the path to the root of the Spack install tree. /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:19 # You can use $spack here to refer to the root of the spack instance. /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:37 template_dirs: /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:37 - $spack/share/spack/templates /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # Temporary locations Spack can try to use for builds. /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # Recommended options are given below. /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # Builds can be faster in temporary directories on some (e.g., HPC) systems. /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # Specifying `$tempdir` will ensure use of the default temporary directory /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # (i.e., ``$TMP` or ``$TMPDIR``). /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # Another option that prevents conflicts and potential permission issues is /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # to specify `$user_cache_path/stage`, which ensures each user builds in their /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # home directory. /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # A more traditional path uses the value of `$spack/var/spack/stage`, which /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # builds directly inside Spack's instance without staging them in a /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # temporary space. Problems with specifying a path inside a Spack instance /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # are that it precludes its use as a system package and its ability to be /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # pip installable. /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # In any case, if the username is not already in the path, Spack will append /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # the value of `$user` in an attempt to avoid potential conflicts between /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # users in shared temporary spaces. /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # The build stage can be purged with `spack clean --stage` and /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # `spack clean -a`, so it is important that the specified directory uniquely /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:38 # identifies Spack staging to avoid accidentally wiping out non-Spack work. /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:73 test_stage: $user_cache_path/test /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:82 misc_cache: $user_cache_path/cache /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:94 connect_timeout: 10 /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:99 verify_ssl: True /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:107 suppress_gpg_warnings: False /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:113 install_missing_compilers: False /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:118 checksum: True /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:123 deprecated: False /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:128 dirty: False /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:136 build_language: C /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:145 locks: True /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:150 url_fetch_method: urllib /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:176 concretizer: clingo /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:183 db_lock_timeout: 300 /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:191 package_lock_timeout: null /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:197 shared_linking: rpath /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:202 allow_sgid: True /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/etc/spack/defaults/config.yaml:207 terminal_title: False _builtin debug: False [32;1m$ export XDG_CONFIG_HOME=${CI_BUILDS_DIR}/J${CI_JOB_ID}_local_config[0;m [32;1m$ echo "Configuring git to use CI_JOB_TOKEN to access git@bbpgitlab.epfl.ch (${XDG_CONFIG_HOME})"[0;m Configuring git to use CI_JOB_TOKEN to access git@bbpgitlab.epfl.ch (/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P89228/J463633_local_config) [32;1m$ mkdir -p "${XDG_CONFIG_HOME}/git"[0;m [32;1m$ echo -e "[url \"https://gitlab-ci-token:${CI_JOB_TOKEN}@bbpgitlab.epfl.ch/\"]\n insteadOf = git@bbpgitlab.epfl.ch:" > "${XDG_CONFIG_HOME}/git/config"[0;m [32;1m$ echo -e "[url \"https://gitlab-ci-token:${CI_JOB_TOKEN}@bbpgitlab.epfl.ch/\"]\n insteadOf = ssh://git@bbpgitlab.epfl.ch/" >> "${XDG_CONFIG_HOME}/git/config"[0;m [32;1m$ cat "${XDG_CONFIG_HOME}/git/config"[0;m [url "https://gitlab-ci-token:[MASKED]@bbpgitlab.epfl.ch/"] insteadOf = git@bbpgitlab.epfl.ch: [url "https://gitlab-ci-token:[MASKED]@bbpgitlab.epfl.ch/"] insteadOf = ssh://git@bbpgitlab.epfl.ch/ [32;1m$ SPACK_FULL_SPEC="${SPACK_PACKAGE}${SPACK_PACKAGE_COMPILER:+%}${SPACK_PACKAGE_COMPILER} ${SPACK_PACKAGE_SPEC} ${SPACK_PACKAGE_DEPENDENCIES} ${SPACK_PACKAGE_DEPENDENCY_ON_PREVIOUS_JOB}"[0;m [32;1m$ echo "Preparing to install ${SPACK_FULL_SPEC}"[0;m Preparing to install coreneuron%intel ~caliper~gpu~legacy-unit+nmodl~openmp+shared+sympy+tests~unified build_type=Debug ^hpe-mpi%gcc ^/4dzxcpsuksxgtuoesheax4sf76wrhkqb [32;1m$ JSON_SPEC=$(spack spec --json ${SPACK_FULL_SPEC})[0;m ==> Error: Package 'hdf5' not found in repository '/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P89228/J463628/spack/bluebrain/repo-bluebrain' srun: error: r1i4n28: task 0: Exited with exit code 1 section_end:1670427636:step_script [0Ksection_start:1670427636:upload_artifacts_on_failure [0K[0K[36;1mUploading artifacts for failed job[0;m[0;m Using git from spack modules [32;1mUploading artifacts...[0;m Runtime platform [0;m arch[0;m=amd64 os[0;m=linux pid[0;m=70863 revision[0;m=58ba2b95 version[0;m=14.2.0 install.xml: found 1 matching files and directories[0;m spack_build_info.env: found 1 matching files and directories[0;m spack_clone_variables.env: found 1 matching files and directories[0;m initial_environment.env: found 1 matching files and directories[0;m spack-build-env.txt: found 1 matching files and directories[0;m spack-build-out.txt: found 1 matching files and directories[0;m spack-configure-args.txt: found 1 matching files and directories[0;m Uploading artifacts as "archive" to coordinator... ok[0;m id[0;m=463633 responseStatus[0;m=201 Created token[0;m=NSqpX62a [32;1mUploading artifacts...[0;m Runtime platform [0;m arch[0;m=amd64 os[0;m=linux pid[0;m=70948 revision[0;m=58ba2b95 version[0;m=14.2.0 install.xml: found 1 matching files and directories[0;m Uploading artifacts as "junit" to coordinator... ok[0;m id[0;m=463633 responseStatus[0;m=201 Created token[0;m=NSqpX62a [32;1mUploading artifacts...[0;m Runtime platform [0;m arch[0;m=amd64 os[0;m=linux pid[0;m=71025 revision[0;m=58ba2b95 version[0;m=14.2.0 spack_build_info.env: found 1 matching files and directories[0;m spack_clone_variables.env: found 1 matching files and directories[0;m Uploading artifacts as "dotenv" to coordinator... ok[0;m id[0;m=463633 responseStatus[0;m=201 Created token[0;m=NSqpX62a section_end:1670427637:upload_artifacts_on_failure [0Ksection_start:1670427637:cleanup_file_variables [0K[0K[36;1mCleaning up project directory and file based variables[0;m[0;m Using git from spack modules section_end:1670427640:cleanup_file_variables [0K[31;1mERROR: Job failed: exit status 1 [0;m