[0KRunning with gitlab-runner 15.1.0 (76984217)[0;m
[0K on BB5 map runner pnPo3yJy[0;m
section_start:1661773265:resolve_secrets
[0K[0K[36;1mResolving secrets[0;m[0;m
section_end:1661773265:resolve_secrets
[0Ksection_start:1661773265:prepare_executor
[0K[0K[36;1mPreparing the "custom" executor[0;m[0;m
[0KUsing Custom executor with driver BB5 PROD runner v0.0.3...[0;m
BB5 PROD runner running on bbpv1.epfl.ch, version 15.1.0, user
TMPDIR is /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/tmp/custom-executor4093803990, slurm job id
Runner ID 29, project root hpc, project name coreneuron
Pipeline ID 71711, build ref 54b4d266ceab6399db5a4cc652b84b23247d19c5, job ID 340050
Build dir /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P71711, optional exclusive flag , optional cpus per task flag --cpus-per-task=1, optional qos flag
A slurm job will be created with name GL_J340050_PROD_P112_CP16_C20
Job parameters: memory=30750M, cpus_per_task=1, duration=1:00:00, constraint=volta ntasks=8 account=proj9998 user=bbpcihpcproj12 partition=prod qos=
Not executing the chown -R
sbatch: INFO: Activating auto partition selection plugin, please report errors to HPC/CS
Submitted batch job 750853
job state: PD
job state: PD
job state: R
sbatch: sbatch -p prod -A proj9998 --ntasks=8 --cpus-per-task=1 --mem=30750M --job-name=GL_J340050_PROD_P112_CP16_C20 -C volta --no-requeue -D /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P71711 --time=1:00:00 --wrap="sleep infinity"
srun: srun --mpi=none --chdir=/gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs//bbpcihpcproj12/P71711 --ntasks=8 --jobid=750853 --cpus-per-task=1 --mem=30750M
section_end:1661773268:prepare_executor
[0Ksection_start:1661773268:prepare_script
[0K[0K[36;1mPreparing environment[0;m[0;m
Running on ldir01u05.bbp.epfl.ch via bbpv1.epfl.ch...
section_end:1661773276:prepare_script
[0Ksection_start:1661773276:get_sources
[0K[0K[36;1mGetting source from Git repository[0;m[0;m
[32;1mSkipping Git repository setup[0;m
[32;1mSkipping Git checkout[0;m
[32;1mSkipping Git submodules setup[0;m
section_end:1661773276:get_sources
[0Ksection_start:1661773276:download_artifacts
[0K[0K[36;1mDownloading artifacts[0;m[0;m
[32;1mDownloading artifacts for build:coreneuron:mod2c:nvhpc:acc:shared (340015)...[0;m
Runtime platform [0;m arch[0;m=amd64 os[0;m=linux pid[0;m=28977 revision[0;m=58ba2b95 version[0;m=14.2.0
Downloading artifacts from coordinator... ok [0;m id[0;m=340015 responseStatus[0;m=200 OK token[0;m=q1roGJfF
section_end:1661773277:download_artifacts
[0Ksection_start:1661773277:step_script
[0K[0K[36;1mExecuting "step_script" stage of the job script[0;m[0;m
[0;33mWARNING: Starting with version 14.0 the 'build_script' stage will be replaced with 'step_script': https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26426[0;m
[32;1m$ env -0 | sort -z | xargs -0 -L 1 echo > initial_environment.env[0;m
[32;1m$ export CTEST_PARALLEL_LEVEL=${SLURM_TASKS_PER_NODE}[0;m
[32;1m$ . ${SPACK_ROOT}/share/spack/setup-env.sh[0;m
[32;1m$ cd ${SPACK_BUILD_DIR}[0;m
[32;1m$ export BOOST_TEST_COLOR_OUTPUT=no[0;m
[32;1m$ i_am_a_failure=0[0;m
[32;1m$ spack build-env ${SPACK_FULL_SPEC} -- ctest --output-on-failure -T Test || i_am_a_failure=1[0;m
Site: r1i6n22
Build name: Linux-nvc++
Create new tag: 20220829-1141 - Experimental
Test project /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P71711/J340015/spack-build/spack-stage-coreneuron-develop-w2z6dfwdq6k5tkwudhpbj7owhrne5qzi/spack-build-w2z6dfw
Start 1: cmd_interface_test
Start 2: interleave_info_constructor_test
Start 3: alignment_test
Start 4: queuing_test
Start 5: test-solver
Start 6: lfp_test
Start 7: ring_TEST
1/20 Test #3: alignment_test ......................... Passed 0.16 sec
Start 20: reporting_1
2/20 Test #4: queuing_test ........................... Passed 0.17 sec
3/20 Test #1: cmd_interface_test ..................... Passed 0.28 sec
Start 8: ring_binqueue_TEST
4/20 Test #6: lfp_test ............................... Passed 0.26 sec
5/20 Test #2: interleave_info_constructor_test ....... Passed 0.30 sec
Start 9: ring_multisend_TEST
6/20 Test #20: reporting_1 ............................***Failed 4.71 sec
num_mpi=2
Duke, Yale, and the BlueBrain Project -- Copyright 1984-2020
Version : 9.0.0 54b4d266 (2022-08-29 13:31:34 +0200)
Additional mechanisms from files
exp2syn.mod expsyn.mod halfgap.mod hh.mod netstim.mod passive.mod pattern.mod stim.mod svclmp.mod
Memory (MBs) : After mk_mech : Max 15.8320, Min 15.6406, Avg 15.7363
Memory (MBs) : After MPI_Init : Max 19.0820, Min 18.9062, Avg 18.9941
Memory (MBs) : Before nrn_setup : Max 19.3008, Min 19.1328, Avg 19.2168
no chkpnt line for 31
../spack-src/coreneuron/io/nrn_filehandler.cpp:96: Assertion 'n_scan == 1' failed.
MPT ERROR: Rank 1(g:1) received signal SIGABRT/SIGIOT(6).
Process ID: 31580, Host: ldir01u05.bbp.epfl.ch, Program: /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P71711/J340015/spack-build/spack-stage-coreneuron-develop-w2z6dfwdq6k5tkwudhpbj7owhrne5qzi/spack-build-w2z6dfw/bin/x86_64/special-core
MPT Version: HPE HMPT 2.25 10/22/21 03:18:39
MPT: --------stack traceback-------
no chkpnt line for 33
../spack-src/coreneuron/io/nrn_filehandler.cpp:96: Assertion 'n_scan == 1' failed.
MPT: ImportError: No module named site
MPT: -----stack traceback ends-----
MPT: On host ldir01u05.bbp.epfl.ch, Program /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P71711/J340015/spack-build/spack-stage-coreneuron-develop-w2z6dfwdq6k5tkwudhpbj7owhrne5qzi/spack-build-w2z6dfw/bin/x86_64/special-core, Rank 1, Process 31580: Dumping core on signal SIGABRT/SIGIOT(6) into directory /gpfs/bbp.cscs.ch/ssd/gitlab_map_jobs/bbpcihpcproj12/P71711/J340015/spack-build/spack-stage-coreneuron-develop-w2z6dfwdq6k5tkwudhpbj7owhrne5qzi/spack-build-w2z6dfw/tests/integration/reporting_1
srun: error: ldir01u05: task 1: Aborted (core dumped)
srun: launch/slurm: _step_signal: Terminating StepId=750853.5
slurmstepd: error: *** STEP 750853.5 ON ldir01u05 CANCELLED AT 2022-08-29T13:41:54 ***
srun: error: ldir01u05: task 0: Terminated
srun: Force Terminated StepId=750853.5
7/20 Test #5: test-solver ............................ Passed 9.01 sec
Start 10: ring_spike_buffer_TEST
8/20 Test #7: ring_TEST .............................. Passed 205.63 sec
Start 11: ring_gap_TEST
9/20 Test #9: ring_multisend_TEST .................... Passed 205.97 sec
Start 12: ring_gap_binqueue_TEST
10/20 Test #8: ring_binqueue_TEST ..................... Passed 206.59 sec
Start 13: ring_gap_multisend_TEST
11/20 Test #10: ring_spike_buffer_TEST ................. Passed 198.89 sec
Start 14: ring_permute1_TEST
12/20 Test #14: ring_permute1_TEST ..................... Passed 205.33 sec
Start 15: ring_gap_permute1_TEST
13/20 Test #11: ring_gap_TEST .......................... Passed 289.07 sec
Start 16: ring_permute2_TEST
14/20 Test #12: ring_gap_binqueue_TEST ................. Passed 289.08 sec
Start 17: ring_gap_permute2_TEST
15/20 Test #13: ring_gap_multisend_TEST ................ Passed 289.63 sec
Start 18: ring_permute2_cudaInterface_TEST
16/20 Test #16: ring_permute2_TEST ..................... Passed 168.66 sec
Start 19: ring_gap_permute2_cudaInterface_TEST
17/20 Test #15: ring_gap_permute1_TEST ................. Passed 250.68 sec
18/20 Test #18: ring_permute2_cudaInterface_TEST ....... Passed 168.15 sec
19/20 Test #17: ring_gap_permute2_TEST ................. Passed 200.52 sec
20/20 Test #19: ring_gap_permute2_cudaInterface_TEST ... Passed 86.62 sec
95% tests passed, 1 tests failed out of 20
Total Test time (real) = 750.04 sec
The following tests FAILED:
20 - reporting_1 (Failed)
Errors while running CTest
[32;1m$ cp -r Testing/ ${CI_PROJECT_DIR}/[0;m
[32;1m$ module load unstable unit-test-translator[0;m
Autoloading python/3.9.7
[32;1m$ cmake2junit > ${CI_PROJECT_DIR}/ctest.xml[0;m
[32;1m$ exit ${i_am_a_failure}[0;m
srun: error: ldir01u05: task 0: Exited with exit code 1
section_end:1661774059:step_script
[0Ksection_start:1661774059:upload_artifacts_on_failure
[0K[0K[36;1mUploading artifacts for failed job[0;m[0;m
[32;1mUploading artifacts...[0;m
Runtime platform [0;m arch[0;m=amd64 os[0;m=linux pid[0;m=51797 revision[0;m=58ba2b95 version[0;m=14.2.0
initial_environment.env: found 1 matching files and directories[0;m
Testing/: found 8 matching files and directories [0;m
Uploading artifacts as "archive" to coordinator... ok[0;m id[0;m=340050 responseStatus[0;m=201 Created token[0;m=q1roGJfF
[32;1mUploading artifacts...[0;m
Runtime platform [0;m arch[0;m=amd64 os[0;m=linux pid[0;m=51830 revision[0;m=58ba2b95 version[0;m=14.2.0
ctest.xml: found 1 matching files and directories [0;m
Uploading artifacts as "junit" to coordinator... ok[0;m id[0;m=340050 responseStatus[0;m=201 Created token[0;m=q1roGJfF
section_end:1661774061:upload_artifacts_on_failure
[0Ksection_start:1661774061:cleanup_file_variables
[0K[0K[36;1mCleaning up project directory and file based variables[0;m[0;m
section_end:1661774062:cleanup_file_variables
[0K[31;1mERROR: Job failed: exit status 1
[0;m