Setting up a dedicated server
Setting up a dedicated server
There are no Windows binaries
This guide is written exclusively for Linux and covers Ubuntu/Debian/Debian-like and Red Hat RPM-based distros. Possible to run with other distros, but some of the commands won't work and you will need to know enough to figure out the software install of dependencies on your own.
If you are considering hosting from home, please read that section of the wiki first.
This guide assumes you know the basics of using a bash terminal: ssh'ing into a server, running bash commands, editing text files, etc.
If you have no experience using bash, maybe start here. Editing text files via commandline is often a challenge. Suggest using nano and reading guide here
If you are new to linux and accessing your server from Windows, highly suggest using MobaXterm as your SSH client as that can make required editing of text files easier using built in editor instead of the native linux command line editors.
And finally, ask yourself why you want to setup a server? If you look at the list, there are many, many more servers than players. If you don't have a team, discord community, or group of friends ready to play on your server, chances are it will go unused. Got one of these... great, let's go.
If you don't want to go through the hassle yourself and want to pay for a hosting service Arctic VR (PC and Shack) and Horde (Shack) provide a server hosting services and are trusted in the community. See links below.
Server Requirements
Pavlovserver can only be hosted under Linux.
This can be running as a VM under windows, but this will increase the complexity of the setup beyond this guide.
Ubuntu 18.04 x86_64, Ubuntu 19/04 amd64, Ubuntu 20.04, and Rocky Linux 8 are confirmed working. Ubuntu 15.04 x86 is known to fail to install using this guide. Recommend not running on "minimal" images as some tools are missing.
- 2GB RAM + 1GB RAM per each additional server with 10 players. More RAM is recommended for larger maps, higher player counts, or complex gamemodes. Not having enough RAM is a common cause of server crashes.
- A ~2.5Ghz CPU will comfortably support a 10 player server.
- A ~4Ghz CPU will comfortably support 24 players. This is a limit built into Pavlov itself, no server will allow players to join over 24. Lower player counts are recommended for complex gamemodes (E.g. CodZ) and/or highly detailed maps, so that maximum number will often be lower.
(Note that these are not hard limits for player counts, just suggestions for the best performance, based on experiences with many server providers on different grades of hardware.)
- Pavlovserver is functionally single threaded (one thread does vast majority of work). More CPUs only help if you are running more servers. More clockspeed = higher performance = more users per server.
The maximum number of servers you can run is dependent on how many CPUs you have. For example, if you have an 8 core CPU, the maximum number of servers would be 7. You should always have at least one CPU free as overhead, as Pavlov servers can use slightly more than one thread.
Hosting at home
When hosting at home there are additional requirements regarding networking: Your router has to support port-forwarding, hair-pinning/LoopbackNAT, and respond to ping requests. These techniques are necessary to allow your server to be found and to talk back to the client. please see https://en.wikipedia.org/wiki/Port_forwarding, https://en.wikipedia.org/wiki/Hairpinning, and https://www.speedguide.net/faq/how-to-become-pingable-behind-a-routerfirewall-376 for more info and google for these terms in addition to your router model to find guides on how to enable/configure it properly. If you have modern Router, it may have these features already enabled, in this case you just have to configure port-forwarding. If you can see your server but not join it while others can, this is indicative of a failure of LoopbackNAT.
A second challenge to hosting at home is CGNAT (Carrier grade NAT: https://www.a10networks.com/blog/carrier-grade-nat/). Just like NAT on your home network that has to be gotten around w/ port-forwarding, CGNAT breaks the ability of external connections to be made back to your server, but the controls are solely in your ISPs hands. Sometimes if you ask, you can be removed from CGNAT pools and/or assigned a static IP. CGNAT rollouts are becoming very common now, so if your server worked then doesn't allow connections now, your ISP may have made this change.
You can check whether you’re on a CGNAT network by visiting a Web site such as ipaddress.com or whatsmyip.com and comparing the IP address listed to the public/WAN IP address assigned to your Internet gateway. To find your gateway’s public IP address, check the status page in the administrative interface. If the two IP addresses differ, you’re likely on a CGNAT network. If so, this problem must be resolved before you go any further.
Not understanding these requirements and configuration is a frequent cause of failed setups and questions in discord. Don't skip this if you are trying to host from home
There are serious security implications of hosting this software at home. None of the devs are security experts and there has been no security review of the code. You are opening up your system to the world and publicity advertising this fact. Expect that any system hosting this software is open to compromise from the Internet and act accordingly. If your router offers a DMZ network, use it. Consider hosting on a cloud based server and think twice before running this software on any system that contains data you care about or has access to networks and computers you care about.
If you're hosting at home using Oracle's VirtualBox, there are steps you can take to prevent double NAT (which can cause issues later on)
1. Select your VM and then click Settings in VirtualBox's main window.
2. Click on Network and change Attached To from NAT to Bridged Adapter.
Once you've done this, your VM should be connected directly to your home router and double NAT should no longer be a issue.
Hosting Providers
Considering the suggested server specs, you may be interested in the following hosting providers. For providers promising dedicated "Pavlov hosting" as opposed to general VPS services please contact and ask how you change maps being hosted and how you get RCON access before you put your money down.
Provider | Base CPU | Details Arctic VR | 4.5 GHz | https://www.arctic-vr.com/server-hosting Amazon Web Services | 4.0 GHz | https://aws.amazon.com/ec2/instance-types/z1d/ VULTR | 4.0 GHz | https://www.vultr.com/products/high-frequency-compute/ packet.com | 3.5 GHz | https://www.packet.com/cloud/servers/c1-small/ Google Cloud Platform | 3.2 GHz | https://cloud.google.com/compute/docs/cpu-platforms
ARCTIC VR
Arctic VR is the largest and longest running Pavlov VR hosting provider. The In-Game Admin Menu is exclusive to the ArcticVR hosting service. They offer EU, NA East and NA West servers for both shack and PCVR. ArcticVR offers the fastest Pavlov servers as well as DDos Protection, RCON remote management, custom maps, low pings, unlimited slots, and expert support.
Website: https://www.arctic-vr.com/server-hosting
VULTR
Consider using a referral from the community:
Carlos: https://www.vultr.com/?ref=8615589-6G
Carrot: https://www.vultr.com/?ref=8689348
Archaeo: https://www.vultr.com/?ref=8653652-6G
Cheeto: https://www.vultr.com/?ref=8717358
For a Pavlov Shack server, I recommend minimum Cloud Compute, Intel High Performance, Ubuntu 20.04, and disabled automatic backups. At the time, this was $6/m. Some of the cheap options have 1TB bandwidth, and some have 2TB+. If it's a personal server, 1TB could be fine, but 2TB should be safe.
Google Cloud Platform (90 day free credit available)
The below configuration is successfully running 4 pavlov PC instances with ~15 total maps and up to 24 players per server. If you want more maps use larger boot disk. Both can dynamically be scaled based on popularity.
Menu -> compute engine -> VM instances
create instance
- region: wherever you like
- machine type: n1-highcpu-4 (4 vCPUs, 3.6 GB memory)
- boot disk: Ubuntu 20.04
- boot disk -> size: 25gb
- network tags: pavlov-server
- [create]
Menu -> Vpc network -> firewall rules
[Create firewall rule]
Name: pavlov server
Target Tags: pavlov-server
Source IP Ranges: 0.0.0.0/0
Protocols and Ports -> Specified protocols and ports -> UDP: 7777, 8177 (Shack and PC)
Protocols and Ports -> Specified protocols and ports -> TCP: 7777, 8177 (Shack-only)
Protocols and Ports -> Specified protocols and ports -> TCP: {RCON port - default 9100}
[save]
Amazon Web Services (30 day free credit available)
Login -> Change Region: Choose your region Services -> Compute -> EC2 -> Create Choose an AMI: Ubuntu 18.04 Instance Type: z1d.large Instance Details -> Tenancy: Dedicated (optional: this will more than triple the cost) Storage: 50gb disk (or whatever, you're in control of the map roster) Security Groups: Create a new Security Group Security group name: pavlov-server Description: pavlov server Rules: Type: Custom ICMP Rule - IPV4 / Protocol: Echo Request Type: Custom UDP Rule / Port Range: 7777 / Source: 0.0.0.0/0 Type: Custom UDP Rule / Port Range: 8177 / Source: 0.0.0.0/0 Type: SSH (optional, but I assume you wanna SSH in..) Launch -> Create new Keypair -> Download -> Launch Once it's started, SSH in like: $ ssh -i your-key.pem ubuntu@you-server-details.amazonaws.com (I use cmder on Windows, if you're using putty or mobaxterm or something, might be different)
Starting Server Install
Obtaining an ApiKey
For your server to show up in the server list, there is now a requirement to have an ApiKey issued by vankrupt. This is to prevent DOS attacks against the master server
Get the key by going here: https://pavlov-ms.vankrupt.com/servers/v1/key and using your mobile number to get a SMS (note that there have been reported failures of this SMS system for non-US mobiles. In this case DM davevillz your mobile and ask for a key)
Put this key in your Game.ini as follows:
ApiKey="ABC123FALSEKEYDONTUSEME"
or if you would like (personally untested..I would use game.ini) you can call it at server runtime with flag as follows:
-KEY="ABC123FALSEKEYDONTUSEME"
This key can be re-used for a reasonable number of servers (a limit of 10 has been mentioned). Contact the Devs if you need more servers per key.
System setup and software installation
Install dependencies and create the `steam` user. (You only need to do this once):
Confirm Linux version
lsb_release -a
Ubuntu 20.04
sudo apt update && sudo apt install -y gdb curl lib32gcc-s1 libc++-dev unzip sudo useradd -m steam
Ubuntu 20.10 or higher
sudo apt update && sudo apt install -y gdb curl lib32gcc-s1 libc++-dev unzip sudo useradd -m steam
Rocky Linux, CentOS 8, and RHEL 8
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/g' /etc/selinux/config sudo setenforce 0 sudo yum install gdb curl glibc.i686 libstdc++.i686 libstdc++-devel.i686 libstdc++-devel.x86_64 unzip wget -y wget https://download-ib01.fedoraproject.org/pub/epel/7/x86_64/Packages/l/libcxx-3.8.0-3.el7.x86_64.rpm sudo rpm -i libcxx-3.8.0-3.el7.x86_64.rpm sudo useradd -m steam
[Optional] Set a password for steam.
sudo passwd steam
Login as the steam user. (You will need to do this each time you connect to your server):
sudo su -l steam
Install Steam:
mkdir ~/Steam && cd ~/Steam && curl -sqL "https://steamcdn-a.akamaihd.net/client/installer/steamcmd_linux.tar.gz" | tar zxvf -
There are different dedicated server versions for Pavlov Server - choose which server version you want to host (PC non-beta, PC Beta, Shack Live, or Shack RC)
(You will need to run this command each time there is a Pavlov Server update).
To host PC Server (non-beta):
~/Steam/steamcmd.sh +force_install_dir /home/steam/pavlovserver +login anonymous +app_update 622970 +exit
To host PC BETA:
~/Steam/steamcmd.sh +force_install_dir /home/steam/pavlovserver +login anonymous +app_update 622970 -beta beta_server +exit
To host Shack Live (Quest):
~/Steam/steamcmd.sh +force_install_dir /home/steam/pavlovserver +login anonymous +app_update 622970 -beta shack +exit
To host Shack RC (Quest):
~/Steam/steamcmd.sh +force_install_dir /home/steam/pavlovserver +login anonymous +app_update 622970 -beta shack_beta +exit
Now that we've installed the server, all pavlov server versions should continue.
Install a fresh copy of steamclient.so (May have to do this after each update):
~/Steam/steamcmd.sh +login anonymous +app_update 1007 +quit mkdir -p ~/.steam/sdk64 cp ~/Steam/steamapps/common/Steamworks\ SDK\ Redist/linux64/steamclient.so ~/.steam/sdk64/steamclient.so cp ~/Steam/steamapps/common/Steamworks\ SDK\ Redist/linux64/steamclient.so ~/pavlovserver/Pavlov/Binaries/Linux/steamclient.so
Make the PavlovServer script executable. (You only need to do this once):
chmod +x ~/pavlovserver/PavlovServer.sh
Server Configuration
Before starting the server we need to configure it. Start by creating some directories:
mkdir -p /home/steam/pavlovserver/Pavlov/Saved/Logs mkdir -p /home/steam/pavlovserver/Pavlov/Saved/Config/LinuxServer mkdir -p /home/steam/pavlovserver/Pavlov/Saved/maps
Mods/Blacklist/Whitelist (optional)
The following three files allow you to specify mods, banned players, and whitelist players. These are optional, but go ahead and create them. Populating the files is described later.
touch /home/steam/pavlovserver/Pavlov/Saved/Config/mods.txt touch /home/steam/pavlovserver/Pavlov/Saved/Config/blacklist.txt touch /home/steam/pavlovserver/Pavlov/Saved/Config/whitelist.txt
Configuring Game.ini
Create a new Game.ini file using following command to open an editor and copy/paste the following block into it then save it. Note not all of these options are required. For instance remove or comment out the Password=0000 line to make a public server with no pin.
nano /home/steam/pavlovserver/Pavlov/Saved/Config/LinuxServer/Game.ini
[/Script/Pavlov.DedicatedServer] bEnabled=true ServerName="My_private_idaho" MaxPlayers=10 #Set this to 10 for Shack. 24 is the max for PC, setting it higher will not allow players to join. ApiKey="ABC123FALSEKEYDONTUSEME" bSecured=true bCustomServer=true bVerboseLogging=false bCompetitive=false #This only works for SND bWhitelist=false RefreshListTime=120 LimitedAmmoType=0 TickRate=90 TimeLimit=60 #Password=0000 #BalanceTableURL="vankruptgames/BalancingTable/main" MapRotation=(MapId="UGC1758245796", GameMode="GUN") MapRotation=(MapId="datacenter", GameMode="SND") MapRotation=(MapId="sand", GameMode="DM")
- bEnabled - whether the server appears in the server list (unconfirmed)
- ServerName - name it will be listed as. (approx 35 chars max)
- MaxPlayers - See suggested specs for limits. 10 standard (10 for shack), 20-24 is max recommended for large maps like Stalingrad or PUSH/RUSH modes, and servers allow a maximum of 24 players.
- ApiKey - Unique ApiKey required to register server with master server. Replace with your key
- bSecured - enabled Valve Anti-Cheat (VAC)
- bCustomServer - activates some new modding tools (save and http functions. Required to write to blacklist.txt for bans)
- bCompetitive - sets the server to competitive mode for SND. If removed the server will default to false
- bVerboseLogging - enables verbose logging, mostly detailed statistics. Be prepared for significant increase in logging.
- bWhitelist - Only allow users in whitelist.txt to join
- RefreshListTime - sets how often the running server checks for updates in ban/whitelist/mod files
- LimitedAmmoType - sets limited ammo mode
0 = Unlimited 1 = Limited Generic 2 = Limited Specific 3 = Custom (for modders) 4 = Limited Special 5 = Boxless Mode
- TimeLimit - sets time limit for each map
- TickRate - sets the server tickrate. (so far minimum stable seems around 50 and max around 120. Beware setting this changes weapon despawn time and breaks some game modes. Recommend not touching)
- Password - PIN number for server #FYI - this is now also changeable via rcon.
- BalanceTableURL - Points to a custom balance table from github. See https://github.com/vankruptgames/BalancingTable for instructions. This line isn't needed if you don't have one.
- MapRotation - you can add multiple of these. The MapId must be “UGC” followed by the map ID. you can find the map id in the URL of the steam workshop page. When a match ends, the server will load the next map in the rotation. You can also use RCON to switch to a map not included in rotation.
Available game modes as of 09/06/2022: SND, TDM, DM, GUN, ZWV, WW2GUN, TANKTDM, KOTH, TTT, OITC, INFECTION, HIDE, PUSH, PH
And for Shack only: CTF
For maps running a Custom gamemode (CodZ, BR, Duel, etc.) it doesn't matter which gamemode you choose, as the map will automatically override it with the custom mode.
UGC###### are for custom maps from Steam workshop (PC Only). Find the map you want in the workshop. This is the URL for Mirage (https://steamcommunity.com/sharedfiles/filedetails/?id=2670152485). The 10 numbers at the end of the URL are what you need, so UGC1661803933 would be the MapId for Mirage.
Firewall/Port forwarding
Using the default settings there are two UDP ports (7777, 8177) that need to be allowed to access the server either by firewall setup or port forwarding.
Ubuntu firewall commands to check firewall and open ports:
sudo ufw status sudo ufw allow 7777 sudo ufw allow 8177 sudo ufw allow 9100 sudo ufw status
Rocky Linux firewall commands to check firewall and open ports:
sudo firewall-cmd --zone=public --add-port=7777/tcp sudo firewall-cmd --zone=public --add-port=7777/udp sudo firewall-cmd --zone=public --add-port=8177/tcp sudo firewall-cmd --zone=public --add-port=8177/udp sudo firewall-cmd --zone=public --add-port=9100/udp sudo firewall-cmd --zone=public --add-port=9100/tcp sudo firewall-cmd --list-ports firewall-cmd --runtime-to-permanent
You may also need to allow Pings (ICMP) for the server's true latency to appear in the master list.
If you are running multiple servers and have set additional ports (see http://wiki.pavlov-vr.com/index.php?title=Dedicated_server#Running_multiple_servers_on_one_host) then you need to allow access to the defined port plus the port 400 higher. So if you use 7000 as your port, then UDP 7000 and 7400 need to be open
If you have setup rcon via RconSettings.txt (below) then that port also need to be allowed access via TCP
If you are running Shack/Quest and want to have maps that are downloadable from your server, your server ports (ex. 7777 & 8177) must also be open via TCP
Shack Maps (Quest)
If you try to use the steps above to add maps for Shack, youll run into issues very quickly. This is because Shack has no affiliation to Steam, and therefore cannot use Steam Workshop maps like above. In the future, these steps will apply to PC servers as well, allowing a server to hold the map itself, instead of offloading to Steam.
To get started, first create the directory to hold the maps
mkdir -p /home/steam/pavlovserver/Pavlov/Saved/maps
You can use any number of methods to move the map files onto the server, but ill go over the two simplest, starting with a GUI approach
Download and install CyberDuck onto your "home" PC (Whatever PC you use for access to the server, but not the server itsself): https://cyberduck.io/
Next, download and unzip your map of choice from https://www.pavlovquest.com/ , https://pavlovhorde.com/mapsList, or look in #shack-map-showcase channel in the discord server. I'm going to use Cheeto's WW2 Items test (http://www.mediafire.com/file/emyt9bs1z9u9ykw/SVR_Cheeto_Items.zip/file ) for this example, you do what you like
In the top left corner, open a connection, set the connection type to SFTP, and fill in the rest of the information. User will be root, password is whatever you set the root pw as (toor in Ubuntu by default)
In the directory dropdown (itll say /root) go back to /
Follow the directories until we arrive back at our "maps" folder
/home/steam/pavlovserver/Pavlov/Saved/maps
Drag and drop the unzipped map folder into the main window, in this case itll be SVR_Cheeto_Items
Hit Allow
Youll now see a folder named SVR_Cheeto_Items, and within it should be nothing but 3 files. If there is another folder in it, move this folder to the maps directory, otherwise the server wont work. The 3 files (2 .PAK one .JSON) should be in home/steam/pavlovserver/Pavlov/Saved/maps/SVR_Cheeto_Items, no more, no less.
Pavlov Shack servers load maps at pavlovserver start. YOU MUST RESTART THE GAME SERVER TO USE THE MAPS.
To play the map, you can either use RCON switch map command, or to add the map to your servers automatic map rotation list, add the map's folder name to your Game.ini. This replaces the MapID section, so for our example it would be
MapRotation=(MapId="SVR_Cheeto_Items", GameMode="TDM")
And thats all! Disconnect CyberDuck from your server before playing, otherwise youll have difficulty joining.
Adding Shack Maps over Command Line Interface
Pavlov Shack servers load maps at pavlovserver start. YOU MUST RESTART THE GAME SERVER TO USE THE MAPS.
A bit more complicated, but adding maps over CLI has its benefits
First, CD to the maps directory you made above
cd ~/pavlovserver/Pavlov/Saved/maps
Download the map files from whatever you use to upload them. To do this, you can either use curl or wget
curl -O insertURLhere
If you download your map from Google Drive, make sure that your .zip folder is shared to anyone.
Then, get the file's url from the shared link.
Example : https://drive.google.com/file/d/19bI8gUHaN5tUrDvnadAacUnchQf6_k/view
will be 19bI8gUHaN5tUrDvnadAacUnchQf6_k
Edit "fileId" and "fileName" before you execute the command.
fileId=19bI8gUHaN5tUrDvnadAacUnchQf6_k
fileName=SVR_USER_MAPNAME
curl -sc /tmp/cookie "https://drive.google.com/uc?export=download&id=${fileId}" > /dev/null
code="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
curl -Lb /tmp/cookie "https://drive.google.com/uc?export=download&confirm=${code}&id=${fileId}" -o ${fileName}
To keep with the Items example:
curl -O https://cdn.discordapp.com/attachments/744185280759857234/745027406695366788/SVR_Cheeto_Items.zip
Next, unzip the file. If this next step errors out, saying the command is unknown, switch back to the root user (su root), and install unzip (sudo apt install unzip), then return to the steam user, and continue.
unzip SVR_Cheeto_Items.zip
Now finish like before by adding the map to your Game.ini
nano /home/steam/pavlovserver/Pavlov/Saved/Config/LinuxServer/Game.ini
MapRotation=(MapId="SVR_Cheeto_Items", GameMode="TDM")
Admin/Moderator list (optional)
The only working in-game admin menu is through Arctic VR server hosting (see below).
If you add moderators to your server, they will not be subject to vote kicks.
nano ~/pavlovserver/Pavlov/Saved/Config/mods.txt
Example Mod List:
76541198397498572 #Optional Comment: Player name 76541198397498573 #Some Other Player
ArcticVR Admin Menu
Arctic-VR Offers an advanced In-Game Admin menu that is exclusive to their hosting service. Features include:
- Switch Maps+Modes
- Teleport Players to Players
- Change Player Skins
- Control Player Buying+Cash
- Spawn Items
- Clean up corpses + guns
- Spawn Bots
- Spawn Items for other players
- Control Teams + Balance/Shuffle Teams
- ResetSND
- And More
You can read more about the ArcticVR admin menu on their website: https://www.arctic-vr.com/server-admin-menu
Handy tools for finding steamIDs
Steam ID finder: https://steamidfinder.com/lookup/ or https://steamid.xyz/
You want to enter in the steamID64 format.
This tool can convert or lookup IDs: https://steamid.io/
If you are looking for a steamID of a player that has been on your server recently this is technique to find them:
grep -i playername ~/pavlovserver/Pavlov/Saved/Logs/* | grep TicketValidation
Configure Rcon
To enable Rcon access create config file:
nano /home/steam/pavlovserver/Pavlov/Saved/Config/RconSettings.txt
Copy in the following text - change password, and if desired the port:
Password=ChangeThisPassword Port=9100
You must specify a port for the Rcon server to use as well as a Password (use a different port per sever instance if running multiple server instances off the same hardware)
Starting the server
cd ~/pavlovserver && ./PavlovServer.sh
That's it, your server will be automatically broadcast to the master server! :)
Two handy tools to determine quickly if your server is broadcasting are Pablub (PC only) and Horde (Shack) and Horde (PC version)
Note that when you close your terminal, the server will go down. See the "supervisors" section below for how to prevent this.
Auto-updating the server
Assuming you have followed this guide and used the default paths, users, and install instructions, you can paste the following into your terminal to auto-update your Pavlov server:
cat << 'EOF' > $HOME/pavlov_daily_update_and_restart.sh #!/bin/bash USER="steam" SERVICENAME="pavlovserver.service" INSTALLDIRNAME="pavlovserver" USERHOME="/home/$USER" echo -e "Beginning Pavlov VR update run on $(date)\n\n" systemctl stop "$SERVICENAME" sleep 5 sudo -iu "$USER" "$USERHOME/Steam/steamcmd.sh" +login anonymous +force_install_dir "$USERHOME/$INSTALLDIRNAME" +app_update 622970 +exit sudo -iu "$USER" "$USERHOME/Steam/steamcmd.sh" +login anonymous +app_update 1007 +quit sudo -iu "$USER" cp "$USERHOME/Steam/steamapps/common/Steamworks SDK Redist/linux64/steamclient.so" "$USERHOME/.steam/sdk64/steamclient.so" sudo -iu "$USER" cp "$USERHOME/Steam/steamapps/common/Steamworks SDK Redist/linux64/steamclient.so" "$USERHOME/pavlovserver/Pavlov/Binaries/Linux/steamclient.so" systemctl start "$SERVICENAME" echo -e "Ending Pavlov VR update run on $(date)\n\n" EOF chmod +x $HOME/pavlov_daily_update_and_restart.sh mkdir $HOME/pavlov_update_logs && touch $HOME/pavlov_update_logs/pavlov_daily_update_and_restart.sh.log CRONLINE="00 2 * * * $HOME/pavlov_daily_update_and_restart.sh >>$HOME/pavlov_update_logs/pavlov_daily_update_and_restart.sh.log 2>&1" (sudo crontab -u root -l; echo "$CRONLINE" ) | sudo crontab -u root - unset CRONLINE
These commands will:
1. Create a shell script in the "steam" user's home directory that automatically stops the Pavlov server, pulls any updates via steam, pull updates to the Steamworks SDK Redist, and copies the latest version of steamclient.so to the Pavlov server directory
2. Make the shell script executable
3. Create a new directory called "pavlov_update_logs" in the "steam" user's directory, and create a logfile inside that directory called "pavlov_daily_update_and_restart.sh.log". You can check this logfile to see the output of the Pavlov and Steam SDK update commands.
4. Create a crontab entry for the root user that runs this shell script automatically every day at 2:00AM local time
If you want to change the time that this script runs, edit the numbers "00 2" in the line that starts with "CRONLINE=". In the example provided, 00 is the minutes column, and 2 is the hour column. For example, if you wanted the server to automatically update and restart at 4:17AM every day, change the line starting with "CRONLINE=" to:
CRONLINE="17 4 * * * $HOME/pavlov_daily_update_and_restart.sh >>$HOME/pavlov_update_logs/pavlov_daily_update_and_restart.sh.log 2>&1"
If the time is PM, use 24-hour time format, e.g. "32 18" for 6:32PM.
Supervising the server using systemd
Running ./PavlovServer.sh in your interactive shell is easy, but the server will shut down when you close your terminal. If you want to run the server in the background, you'll want to run it under a supervisor. There are many options, ranging from the simple (nohup, screen or tmux), to the more robust (supervisord, systemd).
Systemd is running by default on Ubuntu Linux, so we'll use that here.
To run the server as a systemd service, create the file /etc/systemd/system/pavlovserver.service (as root) by typing
sudo nano /etc/systemd/system/pavlovserver.service
Copy contents below into the file and save.
[Unit] Description=Pavlov VR dedicated server [Service] Type=simple WorkingDirectory=/home/steam/pavlovserver ExecStart=/home/steam/pavlovserver/PavlovServer.sh RestartSec=1 Restart=always User=steam Group=steam [Install] WantedBy = multi-user.target
If you'd like your server to start as competitive without editing the game.ini, change the line "ExecStart=/home/steam/pavlovserver/PavlovServer.sh" to look like this below. Though it's much easier to just use the game.ini to enable competitive instead.
ExecStart=/home/steam/pavlovserver/PavlovServer.sh -COMPETITIVE=1
Change the user or paths as appropriate. After saving the service file, to start, restart, stop, or monitor the server:
sudo systemctl start pavlovserver sudo systemctl restart pavlovserver sudo systemctl stop pavlovserver sudo systemctl status pavlovserver
To view the logs:
sudo journalctl -u pavlovserver # to live-tail the logs sudo journalctl -u pavlovserver -f
If you want to run the pavlov server as soon as your server boots (e.g. a VPS you turn on and off), add pavlov as a startup service:
sudo systemctl enable pavlovserver
Advanced Server Administration
Running multiple servers on one host
- Pavlovserver is functionally single threaded (there are multiple threads, but vast majority of work occurs in the primary one). Make sure you have # of pavlovservers + 1 CPU available. More CPUs= more servers. Faster CPU = more players.
- Make multiple server installs by defining a separate install dir for each server.
- Run each server with a unique port. eg: ./PavlovServer.sh -PORT="Desired Port"
- The Server will use the Desired Port and the Port 400 above (Example: You choose 8200 as Port. So Port 8600 will be used as well)
- If you are using RCON, each server will need a unique port defined in RconSettings.txt
- Ensure the ports are forwarded and opened in your firewall
To duplicate directories:
be in the steam home folder
cp -r pavlovserver pavlovserver1 /home/steam/pavlovserver1/PavlovServer.sh -PORT=[Desired Port]
Starting server in competitive mode
Competitive mode will prevent dead teammates from being able to see the match, to prevent ghosting. You can either enable this via Game.ini (described in Game.ini file), or using the -COMPETITIVE=1 parameter.
/home/steam/pavlovserver1/PavlovServer.sh -COMPETITIVE=1
Automated RCON examples
here are some examples of scripts that interact with Pavlov RCON
Inspect players every 5 minutes (INSTALL FROM GIT )
This BASH script connects to the server u set in the config with NETCAT 2 times every 5 min and it uses RCON commands RefreshList ServerInfo &.InspectPlayer {UniqueID} to build stats for the current round stats are then appended over the last one collected
as for the 5 min timer we use a crontask
pending reset (COMING SOON)
Useful Linux Commands
List files / folders in directory, print current working directory,
ls -la #list files/directories along with authorities and ownership pwd #print working directory sudo chown <owner> <file/directory> #change owner of file/directory. If root created something and you need to change it to steam sudo chgrp <group> <file/directory> #change group of file/directory.
Check if pavlov is running by checking process status. If the only row you see is the grep command, pavlov server is not started.
ps -ef | grep pav
Sample output:
steam 34044 1 0 16:52 ? 00:00:00 /bin/sh /home/steam/pavlovserver/PavlovServer.sh steam 34066 34044 80 16:52 ? 00:00:04 /home/steam/pavlovserver/Pavlov/Binaries/Linux/PavlovServer zombies+ 34101 34013 0 16:52 pts/0 00:00:00 grep --color=auto pav
Kill a process. Note this command uses 34044, which is a process id from the command output above.
kill -9 34044
Dump the log to the console so you can scroll through it:
cat /home/steam/pavlovserver/Pavlov/Saved/Logs/Pavlov.log
Search the log for a string - replace my_string with case-sensitive string
cat /home/steam/pavlovserver/Pavlov/Saved/Logs/Pavlov.log | grep my_string
Default Map ID's
datacenter
sand
bridge
containeryard
siberia (Prison Break)
hospital (Zombies map)
killhouse
range
tutorial
santorini
station
industry
ogcontainers (Shack only)
haguenau (Shack only)
foundation (Shack only)
stalingrad (PC only)
stalingrad_night (PC only)
santorini_night (PC only)
sand_night (PC only)
station_night (PC only)
industry_night (PC only)
bunker (PC only)
Automated server log backup
here is how to setup a bash script to 1x a day compress all log files and save them to protect from auto deletion it will also post a discord webhook to confirm its working and saved the days logs
https://github.com/JTWP-org/backUpLogs
Troubleshooting
To run the server without outputting the heartbeat:
./PavlovServer.sh | sed '/heartbeat/d'
To monitor the logs (without heartbeat) from another terminal:
tail -f -n10000 /home/steam/pavlovserver/Pavlov/Saved/Logs/Pavlov.log | grep -v 'Sending heartbeat to masterserver'
Check your CPU Clock Speed with:
lscpu | grep MHz
Server won't start
Error message:
ln: failed to create hard link '/home/steam/.steam/sdk64/steamclient.so': File exists ln: failed to create hard link '/home/steam/pavlovserver/Pavlov/Binaries/Linux/steamclient.so': File exists /home/steam/pavlovserver/Pavlov/Binaries/Linux/PavlovServer: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or directory
First two errors are harmless. The third one is because libc++ step was skipped or failed. Using sudo capable id (probably not steam id), [re]run this line:
sudo apt update && sudo apt install -y gdb curl lib32gcc1 libc++-dev unzip
Map wont load (kicks to datacenter)
try deleting all the downloaded workshop maps with:
sudo rm -rf /tmp/workshop/7777/content/555160/*
for all maps or
sudo rm -rf /tmp/workshop/7777/content/555160/whatever-UGC
for specific map
Map wont load when running in a container
The three typical container problems I have seen:
1) Steam can't write to /tmp for some reason
2) /tmp is actually tmpfs instead of real filesystem and a few gigs of maps runs you out of RAM
3) Made worse by fact that pavlovserver unpacks downloads to ram, then writes to /tmp/workshop. Not enough RAM to do this, no map for you.
Server stuck rotating forever
Run these 2 commands, then restart your server:
cd ~/Steam && ./steamcmd.sh +login anonymous +app_update 1007 +quit cp ~/Steam/steamapps/common/Steamworks\ SDK\ Redist/linux64/steamclient.so ~/.steam/sdk64/steamclient.so
If this doesn't fix it, double check that your server isn't running on the same port as another, by running this command:
ps -ef | grep pav
Then if 2 servers are on the same port, you can fix it by editing the pavlovserver.service file, and making sure the following line:
ExecStart=/home/steam/pavlovserver/PavlovServer.sh -PORT=Number
has a unique number after the port for each server you're running.
Server - ping 999
Google Cloud Platform - Make sure your compute engine has a network tag and your network rules are applied to the tag. Make sure you opened the specified ports.
Network testing tools: Test if your server will respond to ping from outside your network: https://www.wormly.com/test-remote-ping
Test if your ports are accessible via UDP (7777,8177,9100): https://www.ipvoid.com/udp-port-scan/
Can't connect to RCON
Check the file and group owner of RconSettings file. It should be steam, or the ID used to run the pavlov server.
cd /home/steam/pavlovserver/Pavlov/Saved/Config ls -la
If the owner/group are not steam, using root or an id with sudo authority, change them to steam, confirm they are now set properly, then restart the pavlov server. (Note - if Rcon has wrong permissions, after fixing you should also to check the other config files like Game.ini, mods/blacklist/whitelist for the same issue. If steam can't access these files, it could cause problems.).
cd /home/steam/pavlovserver/Pavlov/Saved/Config sudo chown steam RconSettings.txt sudo chgrp steam RconSettings.txt ls -la
PC ONLY - Map Download stops
When downloading a large map from the steam workshop make sure your VPS/Dedi or Shared hosting provider (Otherwise known as a GSP's) has enough RAM assigned to store the map files in its tempfs (stored in ram)!
This is less likely to happen on Dedicated servers and VPS with a decent amount of ram unless you are using docker with RAM limits!
Most GSP's will be using docker or some other type of jailing method to limit system resource allocations and direct access to the host machine, if they are and you know what map you want to use make sure you order a service with enough ram to store your map. If in doubt contact your GSP's support team.
Getting Help
There's a #pc-custom-servers channel on the official Pavlov Discord - https://discord.gg/Pavlov-VR. Lots of helpful people there!