MoonPoint Support Weblog

TinyMUSH 3.1 Crashing After a Power Outage

A CentOS Linux system on which I was running TinyMUSH 3.1 patchlevel 5, the latest version of the MUSH available at the time I'm writing this entry, would not restart after I brought the system back online after a power outage. When I ran the Startmush script, I would see the following:

$ ./Startmush
Indexing help.txt
1004 topics indexed
Indexing mushman.txt
395 topics indexed
Indexing news.txt
line 5: line too long
line 7: line too long
line 9: line too long
line 11: line too long
line 13: line too long
line 17: line too long
line 19: line too long
line 21: line too long
line 23: line too long
line 25: line too long
line 27: line too long
line 41: line too long
line 43: line too long
line 45: line too long
line 47: line too long
line 49: line too long
line 120: line too long
line 122: line too long
line 124: line too long
line 126: line too long
line 128: line too long
line 130: line too long
line 132: line too long
line 134: line too long
line 136: line too long
line 138: line too long
line 140: line too long
line 142: line too long
line 144: line too long
line 146: line too long
line 148: line too long
line 150: line too long
line 152: line too long
line 244: line too long
line 247: line too long
line 298: line too long
line 300: line too long
line 305: line too long
line 350: line too long
line 357: line too long
line 359: line too long
line 362: line too long
line 364: line too long
line 621: line too long
line 623: line too long
line 625: line too long
line 627: line too long
line 629: line too long
line 631: line too long
line 633: line too long
line 839: line too long
line 841: line too long
line 843: line too long
line 845: line too long
line 847: line too long
line 853: line too long
line 862: line too long
line 874: line too long
line 876: line too long
line 878: line too long
line 880: line too long
line 882: line too long
line 884: line too long
line 886: line too long
line 888: line too long
line 890: line too long
line 907: line too long
line 912: line too long
62 topics indexed
Indexing plushelp.txt
line 4: line too long
line 9: line too long
line 142: line too long
line 229: line too long
line 254: line too long
line 282: line too long
line 655: line too long
line 680: line too long
line 880: line too long
42 topics indexed
Indexing qhelp.txt
18 topics indexed
Indexing wizhelp.txt
379 topics indexed
Indexing wiznews.txt
1 topics indexed
Checking for database files and creating backups of old files.
Saving old comsys module db.
Saving old mail module db.
Saving old game log moondreaming.log
Log cleanup done.
Process 18073
080729.193501 TinyMUSH INI/START: Starting: TinyMUSH version 3.1 patchlevel 5 #1 [01/27/2008]
080729.193501 TinyMUSH INI/START: Build date: Thu Apr 10 21:54:36 EDT 2008
080729.193501 TinyMUSH INI/START: Build info: ./configure
            gcc   -g  -I./gdbm-1.8.0
*** glibc detected *** ./bin/netmush: double free or corruption (top): 0x000000000c2076f0 ***
======= Backtrace: =========
/lib64/libc.so.6[0x2aaaab825684]
/lib64/libc.so.6(cfree+0x8c)[0x2aaaab828ccc]
/lib64/libc.so.6(fclose+0x14b)[0x2aaaab8148eb]
./bin/netmush[0x430a85]
./bin/netmush(tf_fclose+0x1d)[0x430cb0]
./bin/netmush[0x497e11]
./bin/netmush(main+0x74b)[0x499a72]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x2aaaab7d18b4]
./bin/netmush(sin+0xb1)[0x40f529]
======= Memory map: ========
00400000-0053d000 r-xp 00000000 fd:00 30377121                           /home/amy/tinymush/src/netmush
0073c000-0074c000 rw-p 0013c000 fd:00 30377121                           /home/amy/tinymush/src/netmush
0074c000-00770000 rw-p 0074c000 00:00 0 
0c052000-0c221000 rw-p 0c052000 00:00 0 
2aaaaaaab000-2aaaaaac5000 r-xp 00000000 fd:00 6651907                    /lib64/ld-2.5.so
2aaaaaac5000-2aaaaaac6000 rw-p 2aaaaaac5000 00:00 0 
2aaaaaae3000-2aaaaaae4000 rw-p 2aaaaaae3000 00:00 0 
2aaaaacc5000-2aaaaacc6000 r--p 0001a000 fd:00 6651907                    /lib64/ld-2.5.so
2aaaaacc6000-2aaaaacc7000 rw-p 0001b000 fd:00 6651907                    /lib64/ld-2.5.so
2aaaaacc7000-2aaaaad49000 r-xp 00000000 fd:00 6651947                    /lib64/libm-2.5.so
2aaaaad49000-2aaaaaf48000 ---p 00082000 fd:00 6651947                    /lib64/libm-2.5.so
2aaaaaf48000-2aaaaaf49000 r--p 00081000 fd:00 6651947                    /lib64/libm-2.5.so
2aaaaaf49000-2aaaaaf4a000 rw-p 00082000 fd:00 6651947                    /lib64/libm-2.5.so
2aaaaaf4a000-2aaaaaf5f000 r-xp 00000000 fd:00 6651956                    /lib64/libnsl-2.5.so
2aaaaaf5f000-2aaaab15e000 ---p 00015000 fd:00 6651956                    /lib64/libnsl-2.5.so
2aaaab15e000-2aaaab15f000 r--p 00014000 fd:00 6651956                    /lib64/libnsl-2.5.so
2aaaab15f000-2aaaab160000 rw-p 00015000 fd:00 6651956                    /lib64/libnsl-2.5.so
2aaaab160000-2aaaab162000 rw-p 2aaaab160000 00:00 0 
2aaaab162000-2aaaab173000 r-xp 00000000 fd:00 6651972                    /lib64/libresolv-2.5.so
2aaaab173000-2aaaab373000 ---p 00011000 fd:00 6651972                    /lib64/libresolv-2.5.so
2aaaab373000-2aaaab374000 r--p 00011000 fd:00 6651972                    /lib64/libresolv-2.5.so
2aaaab374000-2aaaab375000 rw-p 00012000 fd:00 6651972                    /lib64/libresolv-2.5.so
2aaaab375000-2aaaab378000 rw-p 2aaaab375000 00:00 0 
2aaaab378000-2aaaab381000 r-xp 00000000 fd:00 6651918                    /lib64/libcrypt-2.5.so
2aaaab381000-2aaaab580000 ---p 00009000 fd:00 6651918                    /lib64/libcrypt-2.5.so
2aaaab580000-2aaaab581000 r--p 00008000 fd:00 6651918                    /lib64/libcrypt-2.5.so
2aaaab581000-2aaaab582000 rw-p 00009000 fd:00 6651918                    /lib64/libcrypt-2.5.so
2aaaab582000-2aaaab5b0000 rw-p 2aaaab582000 00:00 0 
2aaaab5b0000-2aaaab5b2000 r-xp 00000000 fd:00 6651941                    /lib64/libdl-2.5.so
2aaaab5b2000-2aaaab7b2000 ---p 00002000 fd:00 6651941                    /lib64/libdl-2.5.so
2aaaab7b2000-2aaaab7b3000 r--p 00002000 fd:00 6651941                    /lib64/libdl-2.5.so
2aaaab7b3000-2aaaab7b4000 rw-p 00003000 fd:00 6651941                    /lib64/libdl-2.5.so
2aaaab7b4000-2aaaab8fe000 r-xp 00000000 fd:00 6651914                    /lib64/libc-2.5.so
2aaaab8fe000-2aaaabafd000 ---p 0014a000 fd:00 6651914                    /lib64/libc-2.5.so
2aaaabafd000-2aaaabb01000 r--p 00149000 fd:00 6651914                    /lib64/libc-2.5.so
2aaaabb01000-2aaaabb02000 rw-p 0014d000 fd:00 6651914                    /lib64/libc-2.5.so
2aaaabb02000-2aaaabb09000 rw-p 2aaaabb02000 00:00 0 
2aaaabb09000-2aaaabb18000 r-xp 00000000 fd:00 30377114                   /home/amy/tinymush/game/modules/comsys.so.0.0.0
2aaaabb18000-2aaaabd18000 ---p 0000f000 fd:00 30377114                   /home/amy/tinymush/game/modules/comsys.so.0.0.0
2aaaabd18000-2aaaabd19000 rw-p 0000f000 fd:00 30377114                   /home/amy/tinymush/game/modules/comsys.so.0.0.0
2aaaabd19000-2aaaabd2b000 r-xp 00000000 fd:00 30377105                   /home/amy/tinymush/game/modules/mail.so.0.0.0
2aaaabd2b000-2aaaabf2b000 ---p 00012000 fd:00 30377105                   /home/amy/tinymush/game/modules/mail.so.0.0.0
2aaaabf2b000-2aaaabf2c000 rw-p 00012000 fd:00 30377105                   /home/amy/tinymush/game/modules/mail.so.0.0.0
2aaaabf2c000-2aaaabf2e000 rw-p 2aaaabf2c000 00:00 0 
2aaaac000000-2aaaac021000 rw-p 2aaaac000000 00:00 0 
2aaaac021000-2aaab0000000 ---p 2aaaac021000 00:00 0 
2aaab0000000-2aaab000d000 r-xp 00000000 fd:00 6651906                    /lib64/libgcc_s-4.1.2-20080102.so.1
2aaab000d000-2aaab020d000 ---p 0000d000 fd:00 6651906                    /lib64/libgcc_s-4.1.2-20080102.so.1
2aaab020d000-2aaab020e000 rw-p 0000d000 fd:00 6651906                    /lib64/libgcc_s-4.1.2-20080102.so.1
7fffd4567000-7fffd457d000 rw-p 7fffd4567000 00:00 0                      [stack]
ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0                  [vdso]
080729.193501 TinyMUSH CNF/MOD  : Loaded module: comsys
080729.193501 TinyMUSH CNF/MOD  : Loaded module: mail
080729.193501 MoonDreamingMUSH INI/LOAD : Using gdbm file: moondreaming.gdbm
080729.193501 MoonDreamingMUSH INI/LOAD : Loading object structures.
080729.193501 MoonDreamingMUSH INI/LOAD : Loading db: data/mod_mail.db
Timeout - String 'Cleanup completed.' not found in 'moondreaming.log'. Giving up.
./Startmush: line 262: 18073 Aborted                 $BIN/netmush $make_db -c $GAMENAME.conf -l $LOGNAME -p $PIDFILE -t $TEXT -b $BIN -d $DATA -g $GDBM_DB -k $CRASH_DB >> $LOGNAME 2>&1

The "line too long" messages are normal for this MUSH and don't impact the functioning of the MUSH. The first unexpected message I see is the line below:

*** glibc detected *** ./bin/netmush: double free or corruption (top): 0x000000000c2076f0 ***

The line in the Startmush file that was producing the error message is shown below:

$BIN/netmush $make_db -c $GAMENAME.conf -l $LOGNAME -p $PIDFILE -t $TEXT -b $BIN -d $DATA -g $GDBM_DB -k $CRASH_DB >>$LOGNAME 2>&1 &

I suspected the problem might be related to the fact that I'm running the MUSH on a 64-bit version of CentOS now. It was previously installed on a 32-bit Red Hat 9 Linux system. I remembered encountering problems when I initially moved it to the CentOS system in April, related to the fact that the CentOS system was a 64-bit operating system. Unfortunately, I can't remember exactly how I resolved the problems I encountered then.

When I performed a Google search on "glibc double free or corruption 64-bit", one of the pages I found, glibc detected double free or corruption error? had a posting by RavenOfOdin that offered a solution:

Re: glibc detected double free or corruption error?

That's an old Linux C++ error. Not new news.
To fix it so you can run the program just type:

export MALLOC_CHECK_=0

before running the program.

So I entered the command export MALLOC_CHECK_=0 from a BASH shell prompt before the Startmush command. It solved the problem and the MUSH started as before the crash. I don't remember entering that command previously to get the MUSH to run, but since it did fix the problem, I thought I had better document it for the next time I encountered the problem.

$ export MALLOC_CHECK_=0
$ ./Startmush
Indexing help.txt
1004 topics indexed
Indexing mushman.txt
395 topics indexed
Indexing news.txt
line 5: line too long
line 7: line too long
line 9: line too long
line 11: line too long
line 13: line too long
line 17: line too long
line 19: line too long
line 21: line too long
line 23: line too long
line 25: line too long
line 27: line too long
line 41: line too long
line 43: line too long
line 45: line too long
line 47: line too long
line 49: line too long
line 120: line too long
line 122: line too long
line 124: line too long
line 126: line too long
line 128: line too long
line 130: line too long
line 132: line too long
line 134: line too long
line 136: line too long
line 138: line too long
line 140: line too long
line 142: line too long
line 144: line too long
line 146: line too long
line 148: line too long
line 150: line too long
line 152: line too long
line 244: line too long
line 247: line too long
line 298: line too long
line 300: line too long
line 305: line too long
line 350: line too long
line 357: line too long
line 359: line too long
line 362: line too long
line 364: line too long
line 621: line too long
line 623: line too long
line 625: line too long
line 627: line too long
line 629: line too long
line 631: line too long
line 633: line too long
line 839: line too long
line 841: line too long
line 843: line too long
line 845: line too long
line 847: line too long
line 853: line too long
line 862: line too long
line 874: line too long
line 876: line too long
line 878: line too long
line 880: line too long
line 882: line too long
line 884: line too long
line 886: line too long
line 888: line too long
line 890: line too long
line 907: line too long
line 912: line too long
62 topics indexed
Indexing plushelp.txt
line 4: line too long
line 9: line too long
line 142: line too long
line 229: line too long
line 254: line too long
line 282: line too long
line 655: line too long
line 680: line too long
line 880: line too long
42 topics indexed
Indexing qhelp.txt
18 topics indexed
Indexing wizhelp.txt
379 topics indexed
Indexing wiznews.txt
1 topics indexed
Checking for database files and creating backups of old files.
Saving old comsys module db.
Saving old mail module db.
Saving old game log moondreaming.log
Log cleanup done.
Process 20430
080729.202051 TinyMUSH INI/START: Starting: TinyMUSH version 3.1 patchlevel 5 #1 [01/27/2008]
080729.202051 TinyMUSH INI/START: Build date: Thu Apr 10 21:54:36 EDT 2008
080729.202051 TinyMUSH INI/START: Build info: ./configure
            gcc   -g  -I./gdbm-1.8.0
080729.202051 TinyMUSH CNF/MOD  : Loaded module: comsys
080729.202051 TinyMUSH CNF/MOD  : Loaded module: mail
080729.202051 MoonDreamingMUSH INI/LOAD : Using gdbm file: moondreaming.gdbm
080729.202051 MoonDreamingMUSH INI/LOAD : Loading object structures.
080729.202051 MoonDreamingMUSH INI/LOAD : Loading db: data/mod_mail.db
080729.202051 MoonDreamingMUSH INI/LOAD : Loading db: data/mod_comsys.db
080729.202051 MoonDreamingMUSH INI/COM  : Unrecognized comsys format.
080729.202051 MoonDreamingMUSH INI/LOAD : Load complete.
080729.202051 MoonDreamingMUSH CFG/UPDAT: God(#1) entered config directive: money_name_singular with args 'Moon Coin'. Status: Success.
080729.202051 MoonDreamingMUSH CFG/UPDAT: God(#1) entered config directive: money_name_plural with args 'Moon Coins'. Status: Success.
080729.202051 MoonDreamingMUSH INI/LOAD : Startup processing complete.
080729.202051 MoonDreamingMUSH NET/SLAVE: DNS lookup slave started on fd 1
080729.202051 MoonDreamingMUSH INI/LOAD : Cleanup completed.

References:

glibc detected double free or corruption error?
By: RavenOfOdin
Date: May 12th, 2006
Ubuntu Forums

[/gaming/tinymush] permanent link

Copying a MUSH

A family member wanted to create a new TinyMUSH using an existing one as a starting point. She felt it would be easier to modify the existing one than to start a new one from scratch. I took the steps detailed below to copy the existing MoonDreaming MUSH to a new one that she wanted to set up for SailorMoon roleplaying.

I shut down the original MUSH using the @shutdown command while logged into the MUSH as god. I then ran ./Backup from the command line.

$ ./Backup
Creating flatfile ./backups/moondreaming.1007-1558
Loaded module: comsys
Loaded module: mail
Using gdbm file: ./data/moondreaming.gdbm
Reading ........
Input: TinyMUSH-3 version 1: Zone Link GDBM AtrName AtrKey Parent AtrMoney ExtFlags MoreFlags Powers QuotedStr TypedQuotas Timestamps VisualAttrs
Output: TinyMUSH-3 version 1: Zone Link Parent ExtFlags MoreFlags Powers QuotedStr TypedQuotas Timestamps VisualAttrs
Writing ........
Cleaned 1600 attributes (now 1376): 222 deleted, 186 renumbered (96 objects and 463 individual attrs touched).
Creating database archive ./backups/moondreaming.1007-1558.tar.gz
./backups/moondreaming.1007-1558
./data/mod_comsys.db
./data/mod_mail.db

I changed the working directory to the one containing the existing MUSH directory and then copied that directory and its subdirectories recursively.

$ cp -p -r moondreaming sailormoon

I then made the current working directory the one for the new MUSH.

$ cd sailormoon

I then edited mush.config in that directory. I changed the GAMENAME line to match the name of the new MUSH. I left the OWNER line the same, since the owner was the same person in this case.

GAMENAME=sailormoon

I then renamed the old conf file to match the new MUSH name and deleted the log and pid files from the other MUSH. The pid file contains the process id for the other MUSH process. A new one will be assigned the first time the new MUSH is run. If the pid file is not deleted, when you run Startmush it will state that the MUSH is already running.

$ mv moondreaming.conf sailormoon.conf
$ rm moondreaming.log*
$ rm moondreaming.pid

I also deleted the db directories that came from the other MUSH's directory.

$ rm -f -r db-*

I edited the new sailormoon.conf file I renamed to set the port for the new MUSH and its new name, changing the lines below within it.

crash_database  moondreaming.db.CRASH
gdbm_database   moondreaming.gdbm

port 7676
mud_name MoonDreamingMUSH

The new MUSH must listen on a different port than the existing MUSH and one that is not in use by any other application. You can use netstat -a | grep 9999 substituting the port number you've picked instead of 9999 to see whether an application is listening on that port. If no other process is listening on that port, at least at the time you run the command, you see the command prompt returned, but nothing else. Only the root account can use a port less than 1024; ports below that number are considered to be "reserved".

I changed the working directory to the data directory and renamed the old gdbm file to use the name associated with the new MUSH. I deleted the FLAT and KILLED files associated with the other MUSH.

$ cd data
$ ls
mod_comsys.db      mod_mail.db      moondreaming.FLAT  moondreaming.KILLED
mod_comsys.db.old  mod_mail.db.old  moondreaming.gdbm
$ mv moondreaming.gdbm sailormoon.gdbm
$ rm moondreaming.FLAT
$ rm moondreaming.KILLED

I then moved up to the main diectory for the MUSH and removed all of the backup files from the other MUSH, which are in the backups directory.

$ cd ..
$ ls
moondreaming.0310-2131.tar.gz  moondreaming.0316-1729.tar.gz
moondreaming.0315-1900.tar.gz  moondreaming.1007-1416.tar.gz
$ rm backups/*

I then ran ./Startmush


$ ./Startmush
./Startmush: line 83: [: : integer expression expected
Indexing help.txt
1004 topics indexed
Indexing mushman.txt
395 topics indexed
Indexing news.txt
...
62 topics indexed
Indexing plushelp.txt
line 4: line too long
line 9: line too long
line 142: line too long
line 229: line too long
line 254: line too long
line 282: line too long
line 655: line too long
line 680: line too long
line 880: line too long
42 topics indexed
Indexing qhelp.txt
18 topics indexed
Indexing wizhelp.txt
379 topics indexed
Indexing wiznews.txt
1 topics indexed
Checking for database files and creating backups of old files.
tail: sailormoon.log: No such file or directory
Saving old comsys module db.
Saving old mail module db.
ls: sailormoon.log.*: No such file or directory
No previous game log.
Log cleanup done.
Process 19086
071007.164045 TinyMUSH INI/START: Starting: TinyMUSH version 3.1 patchlevel 4 #1 [10/11/2006]
071007.164045 TinyMUSH INI/START: Build date: Sat Mar 10 21:25:06 EST 2007
071007.164045 TinyMUSH INI/START: Build info: ./configure
            gcc   -g  -I./gdbm-1.8.0
071007.164045 TinyMUSH CNF/MOD  : Loaded module: comsys
071007.164045 TinyMUSH CNF/MOD  : Loaded module: mail
071007.164045 SailorMoonMUSH INI/LOAD : Using gdbm file: sailormoon.gdbm
071007.164045 SailorMoonMUSH INI/LOAD : Loading object structures.
071007.164045 SailorMoonMUSH INI/LOAD : Loading db: data/mod_mail.db
071007.164045 SailorMoonMUSH INI/LOAD : Loading db: data/mod_comsys.db
071007.164045 SailorMoonMUSH INI/LOAD : Load complete.
071007.164045 SailorMoonMUSH CFG/UPDAT: God(#1) entered config directive: money_name_singular with args 'Moon Coin'. Status: Success.
071007.164045 SailorMoonMUSH CFG/UPDAT: God(#1) entered config directive: money_name_plural with args 'Moon Coins'. Status: Success.
071007.164045 SailorMoonMUSH INI/LOAD : Startup processing complete.
071007.164045 SailorMoonMUSH NET/SLAVE: DNS lookup slave started on fd 1
071007.164045 SailorMoonMUSH INI/LOAD : Cleanup completed.

Don't worry about the "integer expression expected", the first time the Startmush is run for the MUSH; it won't appear the next time you use Startmush.

After the MUSH loaded, I logged in as god and changed the password.

@password oldpassword=newpassword
Password changed.

[/gaming/tinymush] permanent link

Tue, Jul 29, 2008 8:48 pm

Sun, Oct 07, 2007 4:52 pm

Wed, Mar 21, 2007 9:13 pm

Sat, Mar 10, 2007 9:52 pm