Cosmovisor Installation Guide

cosmovisor is a process manager for Cosmos SDK application binaries that automates application binary switch at chain upgrades. It polls the upgrade-info.json file that is created by the x/upgrade module at upgrade height, and then can automatically download the new binary, stop the current binary, switch from the old binary to the new one, and finally restart the node with the new binary.

To install the latest version of cosmovisor, run the following command:

go install cosmossdk.io/tools/cosmovisor/cmd/cosmovisor@latest

To install a previous version, you can specify the version.

go install cosmossdk.io/tools/cosmovisor/cmd/cosmovisor@v1.2.0

Setting up environmental variables Cosmovisor relies on the following environmental variables to work properly:

IMPORTANT: If you don't have much free disk space, please set UNSAFE_SKIP_BACKUP=true to avoid your node failing the upgrade due to insufficient disk space when creating the backup.

Initialize the Cosmovisor directory following the specific structure outlined below:

<COSMOVISOR_HOME>
   ├── current -> genesis or upgrades/<name>
   ├── genesis
   │   └── bin
   │       └── $DAEMON_NAME
   └── upgrades
       ├── <name>
       │   ├── bin
       │   │   └── $DAEMON_NAME
       │   └── upgrade-info.json
       .....
       └── <name>
          ├── bin
          │   └── $DAEMON_NAME
          └── upgrade-info.json

Example of the cosmovisor setup

Lets assume, that we have to set up cosmosvisor for Initia node:

daemon=initiad;

Define daemon home directory.

daemon_home="$HOME/.initia"

Create the cosmovisor directory

mkdir -vp "${daemon_home}/cosmovisor"

Create the genesis directory and download (or build form source) genesis binary version to /cosmovisor/genesis/bin.

It is highly recommended to build binary from the source code, because some of them need special libraries, like libwasmvm or so.

mkdir -vp "${daemon_home}/cosmovisor/genesis/bin"
mkdir -vp "${HOME}/prebuilt_bin"

For Initia node link to genesis binary could be found here. Example for linux/amd64 Initiad:

url_for_download="https://initia.s3.ap-southeast-1.amazonaws.com/initiation-1/initia_v0.2.15_Linux_x86_64.tar.gz";
wget -O "${HOME}/prebuilt_bin/initia_v0.2.15_Linux_x86_64.tar.gz" ${url_for_download}

tar -xzf ${HOME}/prebuilt_bin/initia_v0.2.15_Linux_x86_64.tar.gz -C ${HOME}/prebuilt_bin/ && \
rm ${HOME}/prebuilt_bin/initia_v0.2.15_Linux_x86_64.tar.gz

chmod +x ${HOME}/prebuilt_bin/initiad

mv ${HOME}/prebuilt_bin/initiad ${daemon_home}/cosmovisor/genesis/bin/;
mv ${HOME}/prebuilt_bin/libmovevm.*.so /usr/lib;
mv ${HOME}/prebuilt_bin/libcompiler.*.so /usr/lib;

OR

Build binary from the source code. After building the binaries file move/copy it to the /genesis/bin directory.

cp $(which ${daemon}) ${daemon_home}/cosmovisor/genesis/bin/;

Now we have to find out upcoming upgrade name to create an upgrade directory in cosmovisor dir. this info could be extracted from the chain. The only information we need to know - upgrade proposal number.
For Initia the latest upgrade proposal number is:

upgrade_prop=159;

depends on cosmos sdk version, upcoming upgrade name could be found using public api of upgradable chain. For Initia:

api_url="https://api.initia.testnet.node75.org";

and query the upgrade_info from the proposal:

upgrade_name=$(curl -s ${api_url}/cosmos/gov/v1/proposals/${upgrade_prop} | jq -r '.proposal.messages[].plan.name')

or

upgrade_name=$(curl -s ${api_url}/cosmos/gov/v1/proposals/${upgrade_prop} | jq -r '.proposal.messages[].content.plan.name')

If upgrade proposal has passed, it is possible to get the upgrade_info from any cosmos-sdk-based chain:

upgrade_name=$(curl -s ${api_url}/cosmos/upgrade/v1beta1/current_plan | jq -r '.plan.name');

check the upcoming upgrade name:

echo ${upgrade_name}
> initiation-stage-2

Create the upgrade directory

mkdir -vp "${daemon_home}/cosmovisor/upgrades/${upgrade_name}/bin"

build binary from the source code (or download prebuilt binary).

move binaries to the "${daemon_home}/cosmovisor/upgrades/${upgrade_name}/bin" directory

mv "<PATH_TO_NEW_BIN>/initiad" "${daemon_home}/cosmovisor/upgrades/${upgrade_name}/bin"

checkout the bin version:

${daemon_home}/cosmovisor/upgrades/${upgrade_name}/bin/${daemon} version

Important Note!!!

Before creating the service file, please, find out your ${daemon} binary execution command: ${daemon} -h

Usually it is the start command. But sometimes it could be the run. If this is your case - change in following service file: cosmovisor run start to cosmovisor run run

Service file for the cosmovisor

creating new service with cosmovisor and setting up environtment (for autodownload new binaries) for binary with the start execution command:

tee $HOME/${daemon}_cosmovisor.service > /dev/null <<EOF  
[Unit]
Description=$daemon cosmovisor
After=network-online.target
[Service]
User=$USER
WorkingDirectory=${HOME}
ExecStart=$(which cosmovisor) run start
Restart=on-failure
RestartSec=10
LimitNOFILE=65535
Environment="DAEMON_HOME=${daemon_home}"
Environment="DAEMON_NAME=$daemon"
Environment="DAEMON_ALLOW_DOWNLOAD_BINARIES=true"
Environment="DAEMON_RESTART_AFTER_UPGRADE=true"
Environment="UNSAFE_SKIP_BACKUP=true"
[Install]
WantedBy=multi-user.target
EOF

Important Note!!!

If you are using a cosmovisor with prebuilt binaries, you better add extra "Environment" line with libraries path, to resolve the missing libs error when start.

For example: lib_path="/usr/lib";

Environment="LD_LIBRARY_PATH=${lib_path}"

Move service file from $HOME to systemd directory

sudo mv $HOME/${deamon}_cosmovisor.service /etc/systemd/system/${deamon}_cosmovisor.service

Reload daemon, enable and restart service file

sudo systemctl daemon-reload
sudo systemctl enable ${daemon}_cosmovisor
sudo systemctl restart ${daemon}_cosmovisor

check $daemon logs

journalctl -u ${daemon}_cosmovisor -f -o cat

For Example, for Initia daemon service control commands:

sudo systemctl restart initiad_cosmovisor && journalctl -u initiad_cosmovisor -f -o cat
sudo systemctl stop initiad_cosmovisor && systemctl status initiad_cosmovisor

Application' binary link for the cosmovisor

If you are syncing the chain from the scratch with cosmovisor help, then you should use the "genesis" binary and corresponding directory. Creating the link to the genesis binary:

cd ${daemon_home}/cosmovisor/
ln -sfn ${daemon_home}/cosmovisor/genesis/ current

if you are going to use cosmovisor for some arbitrary non genesis binary (the cosmovisor installation for "live" chain), you should set up cosmovisor' working directory to the right version of binary file and create a symlink to that new target:

ln -sfn ${daemon_home}/cosmovisor/upgrades/${upgrade_name}/ current

Check the сorrectness of symlink:

${daemon_home}/cosmovisor/current/bin/${daemon} version

Urgent upgrades

Sometimes there is a necessity to perform an urgent upgrade at random upgrade_height without the upgrade proposal. In this case you can manually use add-upgrade cosmovisor function: for example:

upgrade_height=322000;
upgrade_name="v0.12.3";
path_to_new_binary=$HOME/initia/build/initiad;

cosmovisor add-upgrade ${upgrade_name} ${path_to_new_binary} --force --upgrade-height ${upgrade_height}

check:

${daemon_home}/cosmovisor/upgrades/${upgrade_name}/bin/${daemon} version
>v0.12.3

Important Note!!!

Don't restart your service until the upgrade happens. In that case cosmovisor will perform the upgrade prior to the planned upgrade_height and you can get the "AppHash error" !!!

References

More information you can find in the official cosmovisor repository.