[0KRunning with gitlab-runner 15.1.0 (76984217)[0;m
[0K on BB5 map runner pnPo3yJy[0;m
section_start:1665502649:resolve_secrets
[0K[0K[36;1mResolving secrets[0;m[0;m
section_end:1665502649:resolve_secrets
[0Ksection_start:1665502649:prepare_executor
[0K[0K[36;1mPreparing the "custom" executor[0;m[0;m
[0KUsing Custom executor with driver BB5 PROD runner v0.0.3...[0;m
BB5 PROD runner running on bbpv1.epfl.ch, version 15.1.0, user
TMPDIR is /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/tmp/custom-executor2475959701, slurm job id
Runner ID 29, project root hpc, project name coreneuron
Pipeline ID 78233, build ref e5b42505e38fac181676904a4899753ce9d85893, job ID 385806
Build dir /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P78233, optional exclusive flag , optional cpus per task flag --cpus-per-task=1, optional qos flag
A slurm job will be created with name GL_J385806_PROD_P112_CP19_C24
Job parameters: memory=30750M, cpus_per_task=1, duration=1:00:00, constraint=volta ntasks=8 account=proj9998 user=bbpcihpcproj12 partition=prod qos=
Not executing the chown -R
sbatch: INFO: Activating auto partition selection plugin, please report errors to HPC/CS
Submitted batch job 855012
job state: PD
job state: PD
job state: PD
job state: R
sbatch: sbatch -p prod -A proj9998 --ntasks=8 --cpus-per-task=1 --mem=30750M --job-name=GL_J385806_PROD_P112_CP19_C24 -C volta --no-requeue -D /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P78233 --time=1:00:00 --wrap="sleep infinity"
srun: srun --mpi=none --chdir=/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P78233 --ntasks=8 --jobid=855012 --cpus-per-task=1 --mem=30750M
section_end:1665502653:prepare_executor
[0Ksection_start:1665502653:prepare_script
[0K[0K[36;1mPreparing environment[0;m[0;m
Running on ldir01u09.bbp.epfl.ch via bbpv1.epfl.ch...
section_end:1665502661:prepare_script
[0Ksection_start:1665502661:get_sources
[0K[0K[36;1mGetting source from Git repository[0;m[0;m
[32;1mSkipping Git repository setup[0;m
[32;1mSkipping Git checkout[0;m
[32;1mSkipping Git submodules setup[0;m
section_end:1665502662:get_sources
[0Ksection_start:1665502662:download_artifacts
[0K[0K[36;1mDownloading artifacts[0;m[0;m
[32;1mDownloading artifacts for build:coreneuron:nmodl:nvhpc:acc:shared (385789)...[0;m
Runtime platform [0;m arch[0;m=amd64 os[0;m=linux pid[0;m=63706 revision[0;m=58ba2b95 version[0;m=14.2.0
Downloading artifacts from coordinator... ok [0;m id[0;m=385789 responseStatus[0;m=200 OK token[0;m=XZT3Ekpb
section_end:1665502663:download_artifacts
[0Ksection_start:1665502663:step_script
[0K[0K[36;1mExecuting "step_script" stage of the job script[0;m[0;m
[0;33mWARNING: Starting with version 14.0 the 'build_script' stage will be replaced with 'step_script': https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26426[0;m
[32;1m$ env -0 | sort -z | xargs -0 -L 1 echo > initial_environment.env[0;m
[32;1m$ export CTEST_PARALLEL_LEVEL=${SLURM_TASKS_PER_NODE}[0;m
[32;1m$ . ${SPACK_ROOT}/share/spack/setup-env.sh[0;m
[32;1m$ cd ${SPACK_BUILD_DIR}[0;m
[32;1m$ export BOOST_TEST_COLOR_OUTPUT=no[0;m
[32;1m$ i_am_a_failure=0[0;m
[32;1m$ spack build-env ${SPACK_FULL_SPEC} -- ctest --output-on-failure -T Test || i_am_a_failure=1[0;m
Site: r1i6n17
Build name: Linux-nvc++
Create new tag: 20221011-1538 - Experimental
Test project /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P78233/J385789/spack-build/spack-stage-coreneuron-develop-5ic77t43uub7ftzfth3y7tno3vxw2oiv/spack-build-5ic77t4
Start 1: cmd_interface_test
Start 2: interleave_info_constructor_test
Start 3: alignment_test
Start 4: queuing_test
Start 5: test-solver
Start 6: lfp_test
Start 7: ring_TEST
1/20 Test #3: alignment_test ......................... Passed 0.24 sec
Start 20: reporting_1
2/20 Test #1: cmd_interface_test ..................... Passed 0.26 sec
3/20 Test #2: interleave_info_constructor_test ....... Passed 0.25 sec
Start 8: ring_binqueue_TEST
4/20 Test #4: queuing_test ........................... Passed 0.25 sec
5/20 Test #6: lfp_test ............................... Passed 0.24 sec
Start 9: ring_multisend_TEST
6/20 Test #5: test-solver ............................ Passed 4.60 sec
7/20 Test #20: reporting_1 ............................***Failed 6.57 sec
num_mpi=2
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2020
Version : 8.2.1 e5b42505 (2022-10-11 17:14:13 +0200)
Additional mechanisms from files
exp2syn.mod expsyn.mod halfgap.mod hh.mod netstim.mod passive.mod pattern.mod stim.mod svclmp.mod
Memory (MBs) : After mk_mech : Max 17.8555, Min 17.6641, Avg 17.7598
Memory (MBs) : After MPI_Init : Max 21.1250, Min 20.9453, Avg 21.0352
Memory (MBs) : Before nrn_setup : Max 21.3438, Min 21.1719, Avg 21.2578
../spack-src/coreneuron/io/nrn_filehandler.hpp:122: Assertion 'n_scan == 4' failed.
MPT ERROR: Rank 1(g:1) received signal SIGABRT/SIGIOT(6).
Process ID: 65398, Host: ldir01u09.bbp.epfl.ch, Program: /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P78233/J385789/spack-build/spack-stage-coreneuron-develop-5ic77t43uub7ftzfth3y7tno3vxw2oiv/spack-build-5ic77t4/bin/x86_64/special-core
MPT Version: HPE HMPT 2.25 10/22/21 03:18:39
MPT: --------stack traceback-------
../spack-src/coreneuron/io/nrn_filehandler.hpp:122: Assertion 'n_scan == 4' failed.
MPT: ImportError: No module named site
MPT: -----stack traceback ends-----
MPT: On host ldir01u09.bbp.epfl.ch, Program /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P78233/J385789/spack-build/spack-stage-coreneuron-develop-5ic77t43uub7ftzfth3y7tno3vxw2oiv/spack-build-5ic77t4/bin/x86_64/special-core, Rank 1, Process 65398: Dumping core on signal SIGABRT/SIGIOT(6) into directory /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P78233/J385789/spack-build/spack-stage-coreneuron-develop-5ic77t43uub7ftzfth3y7tno3vxw2oiv/spack-build-5ic77t4/tests/integration/reporting_1
srun: error: ldir01u09: task 1: Aborted (core dumped)
srun: launch/slurm: _step_signal: Terminating StepId=855012.6
slurmstepd: error: *** STEP 855012.6 ON ldir01u09 CANCELLED AT 2022-10-11T17:38:18 ***
srun: error: ldir01u09: task 0: Terminated
srun: Force Terminated StepId=855012.6
Start 10: ring_spike_buffer_TEST
8/20 Test #7: ring_TEST .............................. Passed 15.18 sec
Start 11: ring_gap_TEST
9/20 Test #9: ring_multisend_TEST .................... Passed 15.73 sec
Start 12: ring_gap_binqueue_TEST
10/20 Test #8: ring_binqueue_TEST ..................... Passed 15.88 sec
Start 13: ring_gap_multisend_TEST
11/20 Test #10: ring_spike_buffer_TEST ................. Passed 10.10 sec
Start 14: ring_permute1_TEST
12/20 Test #11: ring_gap_TEST .......................... Passed 8.06 sec
Start 15: ring_gap_permute1_TEST
13/20 Test #14: ring_permute1_TEST ..................... Passed 28.22 sec
Start 16: ring_permute2_TEST
14/20 Test #12: ring_gap_binqueue_TEST ................. Passed 40.66 sec
Start 17: ring_gap_permute2_TEST
15/20 Test #13: ring_gap_multisend_TEST ................ Passed 41.56 sec
Start 18: ring_permute2_cudaInterface_TEST
16/20 Test #15: ring_gap_permute1_TEST ................. Passed 56.83 sec
Start 19: ring_gap_permute2_cudaInterface_TEST
17/20 Test #16: ring_permute2_TEST ..................... Passed 38.47 sec
18/20 Test #18: ring_permute2_cudaInterface_TEST ....... Passed 37.83 sec
19/20 Test #17: ring_gap_permute2_TEST ................. Passed 56.06 sec
20/20 Test #19: ring_gap_permute2_cudaInterface_TEST ... Passed 59.38 sec
95% tests passed, 1 tests failed out of 20
Total Test time (real) = 139.50 sec
The following tests FAILED:
20 - reporting_1 (Failed)
Errors while running CTest
[32;1m$ cp -r Testing/ ${CI_PROJECT_DIR}/[0;m
[32;1m$ module load unstable unit-test-translator[0;m
Autoloading python/3.9.7
[32;1m$ cmake2junit > ${CI_PROJECT_DIR}/ctest.xml[0;m
[32;1m$ exit ${i_am_a_failure}[0;m
srun: error: ldir01u09: task 0: Exited with exit code 1
section_end:1665502831:step_script
[0Ksection_start:1665502831:upload_artifacts_on_failure
[0K[0K[36;1mUploading artifacts for failed job[0;m[0;m
[32;1mUploading artifacts...[0;m
Runtime platform [0;m arch[0;m=amd64 os[0;m=linux pid[0;m=70496 revision[0;m=58ba2b95 version[0;m=14.2.0
initial_environment.env: found 1 matching files and directories[0;m
Testing/: found 8 matching files and directories [0;m
Uploading artifacts as "archive" to coordinator... ok[0;m id[0;m=385806 responseStatus[0;m=201 Created token[0;m=XZT3Ekpb
[32;1mUploading artifacts...[0;m
Runtime platform [0;m arch[0;m=amd64 os[0;m=linux pid[0;m=70524 revision[0;m=58ba2b95 version[0;m=14.2.0
ctest.xml: found 1 matching files and directories [0;m
Uploading artifacts as "junit" to coordinator... ok[0;m id[0;m=385806 responseStatus[0;m=201 Created token[0;m=XZT3Ekpb
section_end:1665502832:upload_artifacts_on_failure
[0Ksection_start:1665502832:cleanup_file_variables
[0K[0K[36;1mCleaning up project directory and file based variables[0;m[0;m
section_end:1665502833:cleanup_file_variables
[0K[31;1mERROR: Job failed: exit status 1
[0;m