[0KRunning with gitlab-runner 15.1.0 (76984217)[0;m
[0K on BB5 map runner pnPo3yJy[0;m
section_start:1665503258:resolve_secrets
[0K[0K[36;1mResolving secrets[0;m[0;m
section_end:1665503258:resolve_secrets
[0Ksection_start:1665503258:prepare_executor
[0K[0K[36;1mPreparing the "custom" executor[0;m[0;m
[0KUsing Custom executor with driver BB5 PROD runner v0.0.3...[0;m
BB5 PROD runner running on bbpv1.epfl.ch, version 15.1.0, user
TMPDIR is /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/tmp/custom-executor3335474938, slurm job id
Runner ID 29, project root hpc, project name coreneuron
Pipeline ID 78233, build ref e5b42505e38fac181676904a4899753ce9d85893, job ID 385808
Build dir /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P78233, optional exclusive flag , optional cpus per task flag --cpus-per-task=1, optional qos flag
A slurm job will be created with name GL_J385808_PROD_P112_CP18_C25
Job parameters: memory=30750M, cpus_per_task=1, duration=1:00:00, constraint=volta ntasks=8 account=proj9998 user=bbpcihpcproj12 partition=prod qos=
Not executing the chown -R
sbatch: INFO: Activating auto partition selection plugin, please report errors to HPC/CS
Submitted batch job 855061
job state: R
sbatch: sbatch -p prod -A proj9998 --ntasks=8 --cpus-per-task=1 --mem=30750M --job-name=GL_J385808_PROD_P112_CP18_C25 -C volta --no-requeue -D /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P78233 --time=1:00:00 --wrap="sleep infinity"
srun: srun --mpi=none --chdir=/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P78233 --ntasks=8 --jobid=855061 --cpus-per-task=1 --mem=30750M
section_end:1665503259:prepare_executor
[0Ksection_start:1665503259:prepare_script
[0K[0K[36;1mPreparing environment[0;m[0;m
Running on ldir01u09.bbp.epfl.ch via bbpv1.epfl.ch...
section_end:1665503266:prepare_script
[0Ksection_start:1665503266:get_sources
[0K[0K[36;1mGetting source from Git repository[0;m[0;m
[32;1mSkipping Git repository setup[0;m
[32;1mSkipping Git checkout[0;m
[32;1mSkipping Git submodules setup[0;m
section_end:1665503267:get_sources
[0Ksection_start:1665503267:download_artifacts
[0K[0K[36;1mDownloading artifacts[0;m[0;m
[32;1mDownloading artifacts for build:coreneuron:nmodl:nvhpc:omp:debug (385791)...[0;m
Runtime platform [0;m arch[0;m=amd64 os[0;m=linux pid[0;m=80362 revision[0;m=58ba2b95 version[0;m=14.2.0
Downloading artifacts from coordinator... ok [0;m id[0;m=385791 responseStatus[0;m=200 OK token[0;m=G8YB4nKg
section_end:1665503269:download_artifacts
[0Ksection_start:1665503269:step_script
[0K[0K[36;1mExecuting "step_script" stage of the job script[0;m[0;m
[0;33mWARNING: Starting with version 14.0 the 'build_script' stage will be replaced with 'step_script': https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26426[0;m
[32;1m$ env -0 | sort -z | xargs -0 -L 1 echo > initial_environment.env[0;m
[32;1m$ export CTEST_PARALLEL_LEVEL=${SLURM_TASKS_PER_NODE}[0;m
[32;1m$ . ${SPACK_ROOT}/share/spack/setup-env.sh[0;m
[32;1m$ cd ${SPACK_BUILD_DIR}[0;m
[32;1m$ export BOOST_TEST_COLOR_OUTPUT=no[0;m
[32;1m$ i_am_a_failure=0[0;m
[32;1m$ spack build-env ${SPACK_FULL_SPEC} -- ctest --output-on-failure -T Test || i_am_a_failure=1[0;m
Site: r1i5n21
Build name: Linux-nvc++
Create new tag: 20221011-1548 - Experimental
Test project /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P78233/J385791/spack-build/spack-stage-coreneuron-develop-cvp73izzgopbpcpcpjbysuhuaizams3t/spack-build-cvp73iz
Start 1: cmd_interface_test
Start 2: interleave_info_constructor_test
Start 3: alignment_test
Start 4: queuing_test
Start 5: test-solver
Start 6: lfp_test
Start 7: ring_TEST
1/20 Test #2: interleave_info_constructor_test ....... Passed 0.27 sec
Start 20: reporting_1
2/20 Test #3: alignment_test ......................... Passed 0.38 sec
3/20 Test #1: cmd_interface_test ..................... Passed 0.44 sec
Start 8: ring_binqueue_TEST
4/20 Test #4: queuing_test ........................... Passed 0.44 sec
5/20 Test #6: lfp_test ............................... Passed 0.42 sec
Start 9: ring_multisend_TEST
6/20 Test #5: test-solver ............................ Passed 3.12 sec
7/20 Test #20: reporting_1 ............................***Failed 4.02 sec
num_mpi=2
num_omp_thread=1
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2020
Version : 8.2.1 e5b42505 (2022-10-11 17:14:13 +0200)
Additional mechanisms from files
exp2syn.mod expsyn.mod halfgap.mod hh.mod netstim.mod passive.mod pattern.mod stim.mod svclmp.mod
Memory (MBs) : After mk_mech : Max 20.3164, Min 20.1289, Avg 20.2227
Memory (MBs) : After MPI_Init : Max 23.6172, Min 23.4258, Avg 23.5215
Memory (MBs) : Before nrn_setup : Max 23.8438, Min 23.6719, Avg 23.7578
../spack-src/coreneuron/io/nrn_filehandler.hpp:122: Assertion 'n_scan == 4' failed.
MPT ERROR: Rank 1(g:1) received signal SIGABRT/SIGIOT(6).
Process ID: 1209, Host: ldir01u09.bbp.epfl.ch, Program: /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P78233/J385791/spack-build/spack-stage-coreneuron-develop-cvp73izzgopbpcpcpjbysuhuaizams3t/spack-build-cvp73iz/bin/x86_64/special-core
MPT Version: HPE HMPT 2.25 10/22/21 03:18:39
MPT: --------stack traceback-------
../spack-src/coreneuron/io/nrn_filehandler.hpp:122: Assertion 'n_scan == 4' failed.
MPT: ImportError: No module named site
MPT: -----stack traceback ends-----
MPT: On host ldir01u09.bbp.epfl.ch, Program /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P78233/J385791/spack-build/spack-stage-coreneuron-develop-cvp73izzgopbpcpcpjbysuhuaizams3t/spack-build-cvp73iz/bin/x86_64/special-core, Rank 1, Process 1209: Dumping core on signal SIGABRT/SIGIOT(6) into directory /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P78233/J385791/spack-build/spack-stage-coreneuron-develop-cvp73izzgopbpcpcpjbysuhuaizams3t/spack-build-cvp73iz/tests/integration/reporting_1
srun: error: ldir01u09: task 1: Aborted (core dumped)
srun: launch/slurm: _step_signal: Terminating StepId=855061.5
slurmstepd: error: *** STEP 855061.5 ON ldir01u09 CANCELLED AT 2022-10-11T17:48:26 ***
srun: error: ldir01u09: task 0: Terminated
srun: Force Terminated StepId=855061.5
Start 10: ring_spike_buffer_TEST
8/20 Test #7: ring_TEST .............................. Passed 69.69 sec
Start 11: ring_gap_TEST
9/20 Test #9: ring_multisend_TEST .................... Passed 79.49 sec
Start 12: ring_gap_binqueue_TEST
10/20 Test #8: ring_binqueue_TEST ..................... Passed 79.73 sec
Start 13: ring_gap_multisend_TEST
11/20 Test #10: ring_spike_buffer_TEST ................. Passed 82.35 sec
Start 14: ring_permute1_TEST
12/20 Test #14: ring_permute1_TEST ..................... Passed 67.06 sec
Start 15: ring_gap_permute1_TEST
13/20 Test #11: ring_gap_TEST .......................... Passed 86.62 sec
Start 16: ring_permute2_TEST
14/20 Test #12: ring_gap_binqueue_TEST ................. Passed 86.07 sec
Start 17: ring_gap_permute2_TEST
15/20 Test #13: ring_gap_multisend_TEST ................ Passed 86.01 sec
Start 18: ring_permute2_cudaInterface_TEST
16/20 Test #16: ring_permute2_TEST ..................... Passed 95.30 sec
Start 19: ring_gap_permute2_cudaInterface_TEST
17/20 Test #18: ring_permute2_cudaInterface_TEST ....... Passed 92.96 sec
18/20 Test #15: ring_gap_permute1_TEST ................. Passed 112.82 sec
19/20 Test #17: ring_gap_permute2_TEST ................. Passed 110.64 sec
20/20 Test #19: ring_gap_permute2_cudaInterface_TEST ... Passed 44.58 sec
95% tests passed, 1 tests failed out of 20
Total Test time (real) = 296.26 sec
The following tests FAILED:
20 - reporting_1 (Failed)
Errors while running CTest
[32;1m$ cp -r Testing/ ${CI_PROJECT_DIR}/[0;m
[32;1m$ module load unstable unit-test-translator[0;m
Autoloading python/3.9.7
[32;1m$ cmake2junit > ${CI_PROJECT_DIR}/ctest.xml[0;m
[32;1m$ exit ${i_am_a_failure}[0;m
srun: error: ldir01u09: task 0: Exited with exit code 1
section_end:1665503599:step_script
[0Ksection_start:1665503599:upload_artifacts_on_failure
[0K[0K[36;1mUploading artifacts for failed job[0;m[0;m
[32;1mUploading artifacts...[0;m
Runtime platform [0;m arch[0;m=amd64 os[0;m=linux pid[0;m=5319 revision[0;m=58ba2b95 version[0;m=14.2.0
initial_environment.env: found 1 matching files and directories[0;m
Testing/: found 8 matching files and directories [0;m
Uploading artifacts as "archive" to coordinator... ok[0;m id[0;m=385808 responseStatus[0;m=201 Created token[0;m=G8YB4nKg
[32;1mUploading artifacts...[0;m
Runtime platform [0;m arch[0;m=amd64 os[0;m=linux pid[0;m=5349 revision[0;m=58ba2b95 version[0;m=14.2.0
ctest.xml: found 1 matching files and directories [0;m
Uploading artifacts as "junit" to coordinator... ok[0;m id[0;m=385808 responseStatus[0;m=201 Created token[0;m=G8YB4nKg
section_end:1665503600:upload_artifacts_on_failure
[0Ksection_start:1665503600:cleanup_file_variables
[0K[0K[36;1mCleaning up project directory and file based variables[0;m[0;m
section_end:1665503601:cleanup_file_variables
[0K[31;1mERROR: Job failed: exit status 1
[0;m