World Backup and Mapping Utility (windows only)

Greetz Dateranoth... FYI I am working on a fork to the script that eliminates the mapping tools, and handles the multiple worlds properly. I may also go a different route altogether and build it into a bukkit plugin but it may not be needed. Glad to see you are still supporting it.

The multiple worlds are supported as far as backing up unless you want them backed up to their own archives. The current revision can backup the entire Server directory and thus the worlds with it. The mapping stuff can all be disabled. It's pretty bloated when used simply, but it can be used just to backup. However, if you make a bukkit plugin that would be great. I would like to see a decent plugin for backups. The biggest downfall mine has is you have no access from minecraft and the saveall.vbs which has to be ran in order to force the world to save.

Anyhow, hope some of it can be useful to you Glad it's still floating out there and getting used.

@Dateranoth
I'm running into an issue with incremental backups failing. The full backup is working however. I noticed this recently when I unfortunately needed to restore from a backup only to discover that the most recent backup was 12 hours old. I watched the backup process fly by and noticed a system error msg flash by. So I logged the output of the backup process overnight.

It looks like it's running out of disk space, which is odd, because the target disk has 400+ gigs available. The server runs on a ramdisk though, which only has 256 megs, and 50 megs free. I think this might be the issue. I've stopped using incrementals for now. Is this something that can be fixed, or do I just need to reserve a certain amount of overhead room on the ramdisk?

Sorry to hear you're having trouble. The way the temporary backup works is it updates the IncrMaster in the cache directory and then compresses those for an incremental file in your backup folder. The cached IncrMaster has to be there so there is something to update against. 7zip is SUPPOSE to be creating a temporary IncrMaster in the same directory as your cache when it's updating, but the best I can figure is it's not in your case. I'm rolling out a patch with a fix in the next 15 or 20 minutes.

I will update here when I get done and you can let me know if it fixes the problem or not--- merged: Mar 5, 2011 8:22 PM ---
Ok. took longer than I thought. The latest version is up. Give it a try and let me know if you're still having problems.

::V2.9-Updated Incremental backup code for 7zip. It should have been placing the-------::
::-----temporary file when updating the backup in the same directory but it wasn't.-----::
::-----Added command line to force temp files to goto backup directory so people running::
::-----on RAMDisks or low disk space won't encounter errors.----------------------------::
::-----INFORMATION UPDATE: PLEASE READ: Regards Backuping up Entire Server and Python---::
::-----If backing up your entire server directory you CANNOT place the backup location--::
::-----inside your Server Directory. This creates a loop that xcopy won't allow and----::
::-----thus fails to create the backup.-------------------------------------------------::
::-----Second, if you use overviewer and Python you MUST have your Python Directory in--::
::-----your windows PATH environment. Instructions can be found here:------------------::
::-----http://docs.python.org/faq/windows.html------------------------------------------::--- merged: Mar 5, 2011 8:28 PM ---
Almost forgot, the saveall.vbs was updated as well. Just rearranged a little bit to make sure the save-all command gets through.

::V3.0-Now supporting retrieval and backup of a remote server via ftp. If you are------::
::-----using a server host such as xenonservers.com and you need a way to backup your---::
::-----server and run the mapping tools you now have a solution. As long as you have a-::
::-----machine that can run this script you can pull the remote server files, via ftp---::
::-----run the backup and mapping, and upload the map to the webserver. The process----::
::-----treats the remote files just like you were hosting it locally and supports all---::
::-----features EXCEPT save-all. You will need to run a plugin such as SimpleCronClone-::
::-----to force save-all ever X minutes ahead of when this script is going to run-------::
::-----My latest release will include the two .scc scripts needed to do this. You will-::
::-----need to download and install the plugin however.---------------------------------::
::-----The option to do remote retrieval can be found under FTP Options-----------------::
::-----The only other change is that the xcopy exclude file MUST have the full path-----::
::-----if being used. Example below.---------------------------------------------------::

V3.0 Supports Backing up Remote servers. Do you have a server host such as xenonservers.com with no way to backup your files or run mapping software? Now you can. Simply follow the directions below and the script will connect to your server via FTP, download the files and then run the backup and mapping process like you were running the server on your own machine. Not the best solution, but it's better than nothing. If using this for remote server backup, saveall should be disabled and an alternative method such as SimpleCronClone should be used.

If performing a REMOTE Server Backup Make sure the variable GetRemoteServerFiles is set to yes and all the Retrieve Server Files from FTP Options have been properly set. Also, make sure WinSCP is installed and the path is set in your variables.
Also, you will not use the VBS Script when backing up a remote server because it will have no effect. Instead you will want to install SimpleCronClone and using the two scripts I included have it run save-all every X minutes. You will want to force your server to save a few minutes more often then you backup. So, if you are backing up every 10 minutes, then force save every 7. It's not the best solution, but the only one available for remote backup atm.

::V3.0-Now supporting retrieval and backup of a remote server via ftp. If you are------::
::-----using a server host such as xenonservers.com and you need a way to backup your---::
::-----server and run the mapping tools you now have a solution. As long as you have a-::
::-----machine that can run this script you can pull the remote server files, via ftp---::
::-----run the backup and mapping, and upload the map to the webserver. The process----::
::-----treats the remote files just like you were hosting it locally and supports all---::
::-----features EXCEPT save-all. You will need to run a plugin such as SimpleCronClone-::
::-----to force save-all ever X minutes ahead of when this script is going to run-------::
::-----My latest release will include the two .scc scripts needed to do this. You will-::
::-----need to download and install the plugin however.---------------------------------::
::-----The option to do remote retrieval can be found under FTP Options-----------------::
::-----The only other change is that the xcopy exclude file MUST have the full path-----::
::-----if being used. Example below.---------------------------------------------------::

V3.0 Supports Backing up Remote servers. Do you have a server host such as xenonservers.com with no way to backup your files or run mapping software? Now you can. Simply follow the directions below and the script will connect to your server via FTP, download the files and then run the backup and mapping process like you were running the server on your own machine. Not the best solution, but it's better than nothing. If using this for remote server backup, saveall should be disabled and an alternative method such as SimpleCronClone should be used.

If performing a REMOTE Server Backup Make sure the variable GetRemoteServerFiles is set to yes and all the Retrieve Server Files from FTP Options have been properly set. Also, make sure WinSCP is installed and the path is set in your variables.
Also, you will not use the VBS Script when backing up a remote server because it will have no effect. Instead you will want to install SimpleCronClone and using the two scripts I included have it run save-all every X minutes. You will want to force your server to save a few minutes more often then you backup. So, if you are backing up every 10 minutes, then force save every 7. It's not the best solution, but the only one available for remote backup atm.

I actually just updated to the new DTT version of Overviewer today. I'll be updating the script tonight or tomorrow and I'll definitely add in a parameter for the changing modes ( and all the options ) and I'll see about doing more than one overviewer pass each run.

Ok, the latest version is out. Give it a try and let me know how it works for you. Overviewer DTT natively supports multiple pass rendering. In the never version just add this to the overviewer OPTIONS variable: --rendermodes=lighting,night etc.

V3.1 is out and has adopted the Direct to Tile version of Overviewer. Support for older versions have been dropped though they can still be used. If you wish to keep the older versions you will need to rename gmap.py or gmap.exe to overviewer.py or overviewer.exe . You will then need to change the overviewer options to comply with the older version. I highly suggest updating to the DTT version as it is much faster and doesn't require the first caching stage. It also allows you to render more than one mode at a time. Refer to the settings for more information. There is now an Overviewer OPTIONS setting allowing you to customize the commands. Finally, support for sessions in WinSCP has been added. This is only necessary if you are Backing up from a Remote Server who's timezone is different from yours.

::V3.1-The older versions of Overviewer have been dropped. Only the Direct to Tile-----::
::-----version is supported now. It can be found at the link below---------------------::
::-----https://github.com/brownan/Minecraft-Overviewer/tree/dtt-c-render----------------::
::-----If you are using an older version of Overviewer it will still work but you'll----::
::-----need to change gmap.py or gmap.exe to overviewer.py or overviewer.exe.-----------::
::-----You will also have to change the OVWROPTION sections below to comply with the old::
::-----Overviewer settings.-------------------------------------------------------------::
::-----Finally, I have added an option to the Remote Server Download. This option------::
::-----will allow you to open sessions saved in WinSCP, which is useful for syncing the-::
::-----server time zone with your local computer. If you do not then the files will not-::
::-----sync properly. Refer to the WinSCP website for instructions on the time offset.-::

If you Downloaded this in the last 35 minutes please redownload. I had implemented something unnecessary and removed it.

If you Downloaded this in the last 35 minutes please redownload. I had implemented something unnecessary and removed it.

Click to expand...

On first time it does the job backup/overviewer.
But then when it starts the 2nd time backup/viewer,
I get a error "file not found" or something. Its scrolls too fast to read.
it creates a 1kb.7z and then goes back to the countdown for next run.

Can you post your user variables section? Omit out usernames/passwords. Also, there should have been a file with it called MCDebug.bat, can you place that in the same folder as MCBackup.bat and run it. Let it get to the error section and then post the log file it creates.

Can you post your user variables section? Omit out usernames/passwords. Also, there should have been a file with it called MCDebug.bat, can you place that in the same folder as MCBackup.bat and run it. Let it get to the error section and then post the log file it creates.

Click to expand...

I fixed it, my "exclude.txt" was not in full path. just weird it works on first run :/

I think it was because I was lazy. When I initially wrote the batch I changed directories in the batch instead of opening the files using full path. So, on the first pass through the batch is running xcopy from the same location as your exclude.txt . On the next pass it's probably changed to a different directory and running xcopy from a location that is different from your exclude.txt

I think it was because I was lazy. When I initially wrote the batch I changed directories in the batch instead of opening the files using full path. So, on the first pass through the batch is running xcopy from the same location as your exclude.txt . On the next pass it's probably changed to a different directory and running xcopy from a location that is different from your exclude.txt

No current plans as of this moment. Everything has been working well so I haven't thought of any changes. I'm open to suggestions though.

Click to expand...

Thing's have been running really smoothly here. I've been running 4 instances of the batch file. One for each of the 3 worlds, and another for just the server and plugin files. The only thing I can think of adding might be some additional things to overviewer, such as the biome color extractor. It would be cool if it could be run before each render so any new chunks that are generated will have the proper biome colours.

I've also experimented with PNGOUT to reduce the map data size a bit. It works good, about 20% overall. I ended up saving a couple hundred megs, which is nothing to sneeze at, but takes forever cause it can only work on a single file at a time. I broke it down a bit to work on each of overviewers 4 folders in parallel, and that helped, but still takes a couple hours. So by the time it finished, overviewer has already generated some new tiles if something has changed. you'd have to run it almost constantly depeding on how big your world is and how often its updated, or have a lot more cores to work with. I'm using a Core i7 with 8 threads runnning at 4.8 ghz.

I used a starter bat to launch the 4 seperate pngout processes for each world.

I like what you have setup. I had thought about including the biome extractor, but the last time I had looked into it the only one I found specifically stated not to run the extractor while minecraft was running. That was a few revisions ago, and now everything is moved first anyhow. I'll have to look into adding it. I had also thought of adding multiple world support, but at the moment it is easier to run multiple instances.

I've got a bit of a problem. It seems to be working fine, and I see the 'world' folder copy into the directory I specified, but once the backup is completed and it starts counting down to the next backup, the world file it just copied disappears... D:

I've got a bit of a problem. It seems to be working fine, and I see the 'world' folder copy into the directory I specified, but once the backup is completed and it starts counting down to the next backup, the world file it just copied disappears... D:

Click to expand...

Do you mean the zip archive it creates?

Can you post some more details about how you have it set up? Maybe even post the relevent config section of the batch file.