Skip to content

Instantly share code, notes, and snippets.

@gvsmirnov

gvsmirnov/oom.killer.txt

Created May 12, 2017 21:11
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save gvsmirnov/78c0377865644a8b08157bd26239a923 to your computer and use it in GitHub Desktop.
Save gvsmirnov/78c0377865644a8b08157bd26239a923 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
oom.killer.txt
[7168160.156548] bash invoked oom-killer: gfp_mask=0x26000c0, order=2, oom_score_adj=0
[7168160.156551] bash cpuset=/ mems_allowed=0
[7168160.156556] CPU: 0 PID: 23277 Comm: bash Not tainted 4.4.0-59-generic #80-Ubuntu
[7168160.156558] Hardware name: Xen HVM domU, BIOS 4.2.amazon 09/19/2016
[7168160.156559] 0000000000000286 0000000021d7e43c ffff8800207dbaf0 ffffffff813f7583
[7168160.156562] ffff8800207dbcc8 ffff880033b60e00 ffff8800207dbb60 ffffffff8120ad5e
[7168160.156564] ffffffff81cd2dc7 0000000000000000 ffffffff81e67760 0000000000000206
[7168160.156567] Call Trace:
[7168160.156573] [<ffffffff813f7583>] dump_stack+0x63/0x90
[7168160.156577] [<ffffffff8120ad5e>] dump_header+0x5a/0x1c5
[7168160.156580] [<ffffffff81192722>] oom_kill_process+0x202/0x3c0
[7168160.156582] [<ffffffff81192b49>] out_of_memory+0x219/0x460
[7168160.156585] [<ffffffff81198abd>] __alloc_pages_slowpath.constprop.88+0x8fd/0xa70
[7168160.156587] [<ffffffff81198eb6>] __alloc_pages_nodemask+0x286/0x2a0
[7168160.156589] [<ffffffff81198f6b>] alloc_kmem_pages_node+0x4b/0xc0
[7168160.156593] [<ffffffff8107ea5e>] copy_process+0x1be/0x1b70
[7168160.156597] [<ffffffff811c1670>] ? handle_mm_fault+0xce0/0x1820
[7168160.156600] [<ffffffff81347bd3>] ? security_file_alloc+0x33/0x50
[7168160.156602] [<ffffffff810805a0>] _do_fork+0x80/0x360
[7168160.156604] [<ffffffff81080929>] SyS_clone+0x19/0x20
[7168160.156608] [<ffffffff818384f2>] entry_SYSCALL_64_fastpath+0x16/0x71
[7168160.156610] Mem-Info:
[7168160.156614] active_anon:1675477 inactive_anon:27690 isolated_anon:0
[7168160.156616] Node 0 DMA free:15904kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15988kB managed:15904kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[7168160.156621] lowmem_reserve[]: 0 1953 15274 15274 15274
[7168160.156624] Node 0 DMA32 free:62328kB min:2012kB low:2512kB high:3016kB active_anon:622028kB inactive_anon:18416kB active_file:543524kB inactive_file:537272kB unevictable:844kB isolated(anon):0kB isolated(file):0kB present:2080768kB managed:2000144kB mlocked:844kB dirty:404kB writeback:0kB mapped:7628kB shmem:23676kB slab_reclaimable:204172kB slab_unreclaimable:4484kB kernel_stack:2432kB pagetables:1240kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[7168160.156628] lowmem_reserve[]: 0 0 13321 13321 13321
[7168160.156630] Node 0 Normal free:89200kB min:13744kB low:17180kB high:20616kB active_anon:6079880kB inactive_anon:92344kB active_file:3532252kB inactive_file:3508364kB unevictable:2812kB isolated(anon):0kB isolated(file):0kB present:14024704kB managed:13641084kB mlocked:2812kB dirty:19152kB writeback:0kB mapped:38968kB shmem:122288kB slab_reclaimable:245508kB slab_unreclaimable:20280kB kernel_stack:3296kB pagetables:19804kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[7168160.156634] lowmem_reserve[]: 0 0 0 0 0
[7168160.156636] Node 0 DMA: 0*4kB 0*8kB 0*16kB 1*32kB (U) 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15904kB
[7168160.156645] Node 0 DMA32: 5657*4kB (UMEH) 4825*8kB (UMEH) 5*16kB (H) 4*32kB (H) 1*64kB (H) 1*128kB (H) 1*256kB (H) 1*512kB (H) 0*1024kB 0*2048kB 0*4096kB = 62396kB
[7168160.156654] Node 0 Normal: 22341*4kB (UM) 1*8kB (U) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 89372kB
[7168160.156661] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
[7168160.156662] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[7168160.156663] 2067517 total pagecache pages
[7168160.156664] 0 pages in swap cache
[7168160.156666] Swap cache stats: add 0, delete 0, find 0/0
[7168160.156666] Free swap = 0kB
[7168160.156667] Total swap = 0kB
[7168160.156668] 4030365 pages RAM
[7168160.156669] 0 pages HighMem/MovableOnly
[7168160.156669] 116082 pages reserved
[7168160.156670] 0 pages cma reserved
[7168160.156671] 0 pages hwpoisoned
[7168160.156672] [pid] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
[7168160.156676] [ 395] 0 395 11157 4288 27 3 0 0 systemd-journal
[7168160.156678] [ 437] 0 437 25742 53 17 3 0 0 lvmetad
[7168160.156680] [ 959] 0 959 4029 589 11 3 0 0 dhclient
[7168160.156682] [ 1111] 0 1111 206242 1004 34 4 0 0 lxcfs
[7168160.156683] [ 1115] 0 1115 1100 314 7 3 0 0 acpid
[7168160.156685] [ 1118] 0 1118 7248 710 18 3 0 0 systemd-logind
[7168160.156686] [ 1133] 0 1133 6517 492 18 3 0 0 cron
[7168160.156688] [ 1143] 107 1143 10763 674 25 3 0 -900 dbus-daemon
[7168160.156690] [ 1212] 0 1212 68466 532 35 3 0 0 accounts-daemon
[7168160.156691] [ 1216] 0 1216 6511 416 18 3 0 0 atd
[7168160.156693] [ 1218] 104 1218 65158 867 30 3 0 0 rsyslogd
[7168160.156694] [ 1233] 0 1233 3344 38 11 3 0 0 mdadm
[7168160.156696] [ 1235] 0 1235 1306 30 9 3 0 0 iscsid
[7168160.156697] [ 1236] 0 1236 1431 877 9 3 0 -17 iscsid
[7168160.156699] [ 1296] 0 1296 3211 391 12 3 0 0 agetty
[7168160.156700] [ 1297] 0 1297 3665 359 12 3 0 0 agetty
[7168160.156702] [ 1309] 0 1309 4868 371 14 3 0 0 irqbalance
[7168160.156703] [ 1314] 0 1314 69398 425 38 3 0 0 polkitd
[7168160.156705] [ 1339] 0 1339 16380 738 35 3 0 -1000 sshd
[7168160.156707] [23846] 100 23846 25081 387 19 3 0 0 systemd-timesyn
[7168160.156708] [31752] 0 31752 10685 563 23 3 0 -1000 systemd-udevd
[7168160.156710] [31876] 108 31876 6719 58 18 3 0 0 uuidd
[7168160.156711] [12353] 0 12353 11907 183 27 3 0 0 rpcbind
[7168160.156713] [23773] 113 23773 27509 504 24 3 0 0 ntpd
[7168160.156714] [ 4890] 0 4890 54986 2177 30 5 0 0 snapd
[7168160.156716] [18722] 1000 18722 11312 534 26 4 0 0 systemd
[7168160.156718] [18725] 1000 18725 35819 528 35 3 0 0 (sd-pam)
[7168160.156719] [25620] 1000 25620 747420 53429 183 6 0 0 java
[7168160.156721] [26108] 1000 26108 746649 55079 184 6 0 0 java
[7168160.156722] [ 2942] 112 2942 5992 211 16 3 0 0 nrpe
[7168160.156724] [ 7195] 1000 7195 747932 55099 189 6 0 0 java
[7168160.156726] [ 9822] 1000 9822 3603973 1481048 3607 18 0 0 java
[7168160.156727] [ 4518] 0 4518 23842 1564 51 3 0 0 sshd
[7168160.156728] [ 4592] 1000 4592 23842 967 49 3 0 0 sshd
[7168160.156730] [ 4593] 1000 4593 5377 1318 14 3 0 0 bash
[7168160.156732] [18327] 0 18327 23842 1562 49 3 0 0 sshd
[7168160.156733] [18359] 1000 18359 23842 943 47 3 0 0 sshd
[7168160.156734] [18360] 1000 18360 5375 1340 15 3 0 0 bash
[7168160.156736] [21581] 1000 21581 6513 695 16 3 0 0 screen
[7168160.156737] [21582] 1000 21582 5375 1334 15 3 0 0 bash
[7168160.156739] [23277] 1000 23277 3109 1028 10 3 0 0 bash
[7168160.156740] [10433] 112 10433 7050 621 20 3 0 0 nrpe
[7168160.156742] [10434] 112 10434 7050 146 17 3 0 0 nrpe
[7168160.156743] [10435] 112 10435 1127 177 8 3 0 0 sh
[7168160.156745] [10436] 112 10436 4929 792 14 3 0 0 check_kafka_lag
[7168160.156746] [10437] 112 10437 4929 431 12 3 0 0 check_kafka_lag
[7168160.156748] [10438] 112 10438 673371 17549 92 7 0 0 java
[7168160.156749] [10439] 112 10439 3236 237 12 3 0 0 grep
[7168160.156750] [10440] 112 10440 3756 243 12 3 0 0 sed
[7168160.156752] [10441] 112 10441 5869 337 15 3 0 0 awk
[7168160.156753] Out of memory: Kill process 9822 (java) score 379 or sacrifice child
[7168160.161789] Killed process 9822 (java) total-vm:14415892kB, anon-rss:5900116kB, file-rss:24076kB
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment