Clustering Setup (Windows 5xx)

From Project Skyfire
Jump to navigation Jump to search

This page describes a Windows setup for the SkyFire 5.4.8 clustering branch. It builds on Installation (Windows 5xx) and uses the same client build, extracted data and native build requirements.

Revision reference: Written against clustering commit e1a492634d. Clustering is under development. Keep binaries, Python services, web files, configuration templates and SQL from the same revision. Validate the complete installation before production deployment.

What the cluster currently does

  • Hubserver registers nodes, routes authentication connections, coordinates session handoffs and supervises local auth/world processes. Its web interface provides server health, commands, account administration, backups and coordinated restarts.
  • Authserver authenticates players against the shared authentication database. It can register a legacy authentication endpoint or an Authnet endpoint.
  • Worldserver runs players, creatures, scripts, movement, combat and gameplay simulation. Clients connect directly to the world endpoint returned during authentication.
  • Mapserver supplies extracted game files over authenticated HTTPS. Multiple providers can own different map IDs. Worldserver verifies and caches the files before startup, then loads nearby terrain, collision and navigation tiles into memory as players move.
  • The optional backup worker executes backup and recovery jobs requested through the hub.
Game client ---> Hub authentication ingress ---> Authserver
Game client ----------------------------------> Worldserver

Hubserver <--- mutual-TLS registration ---> Auth / World / Map nodes
Worldserver --- discovery ---> Hubserver
Worldserver --- mutual-TLS HTTPS ---> Mapservers ---> local world cache
Operator ------ local web interface -----------> Hubserver

Map bytes travel directly from mapserver to worldserver. They do not pass through the hub. Normal gameplay uses the local world cache and does not stream terrain from mapserver for every movement.

Current limits: Use one active hub and one active world owner per realm. Multiple worlds for different realms are supported; two live worlds claiming the same realm make that realm unavailable. Do not run two gameplay processes against the same character database as interchangeable replicas. Automatic gameplay migration, seamless world fallback, automatic map-provider failover and multi-active hub failover are not implemented.

Example layout and ports

This guide starts with a single-host installation and uses generic example paths: C:\SkyFire\Source for the source checkout, C:\SkyFire\Server for installed services, C:\SkyFire\Database for database downloads, and C:\SkyFire\PKI for certificate issuance. Replace paths, credentials and addresses for your installation. PowerShell commands assume the server directory below; configuration examples use forward slashes to avoid backslash escaping. 

C:\SkyFire\Server\
    hubserver.exe
    authserver.exe
    worldserver.exe
    hubserver.conf
    authserver.conf
    worldserver.conf
    web\
    mapserver\
    backup\
    control\
    sql\
    cluster-tls\
    data\
        maps\
        vmaps\
        mmaps\
        dbc\
        db2\
        cameras\
    map-cache\
    backups\
    .venv\
Service Example endpoint Access
Hub web interface 127.0.0.1:54880 Local operator browser
Hub cluster listener 127.0.0.1:9100 Cluster nodes; mutual TLS
Hub legacy login ingress 127.0.0.1:3723 Game clients using legacy login
Authserver legacy backend 127.0.0.1:3724 Hub only
Optional hub Authnet ingress 127.0.0.1:1118 Authnet clients
Optional Authnet backend 127.0.0.1:1119 Hub only
Worldserver 127.0.0.1:8085 Game clients
Map provider A / B 127.0.0.1:54910 / 54911 Worldserver; mutual-TLS HTTPS

Loopback addresses work only on this computer. For LAN or Internet clients, use reachable ingress and world addresses, adjust bind addresses, certificates and firewall rules, and restrict backend/cluster/map ports to their intended peers. Registry TLS does not encrypt the forwarded authentication transport itself; backend links belong on a trusted network.

Requirements and installation

  1. Follow Installation (Windows 5xx) for supported Visual Studio, CMake, Boost, OpenSSL, MySQL and client extraction requirements. Use matching 5.4.8 build-18414 data.
  2. Check out the clustering branch before configuring and compiling. Keep a record of the commit you deploy.
  3. Configure the CMake installation destination, compile the matching hubserver, authserver and worldserver, and run the INSTALL target for the selected build configuration.
  4. Install Python 3.11 or newer for mapserver and the optional backup worker. The local hub web interface itself needs neither Python, PHP nor Node.js.
  5. Install compatible MySQL command-line tools if using backups. The backup worker needs both mysql.exe and mysqldump.exe.

For an already configured build directory, an installation command takes this form: 

cmake --install "C:\SkyFire\Source\build" --config Release --prefix "C:\SkyFire\Server"

Use the actual source/build paths and the configuration you compiled. INSTALL supplies the accompanying web, mapserver, control and backup assets. Copying only executables from a Release directory can leave an older web interface or Python protocol behind. Existing operator configuration files are preserved: compare them with the updated .dist files and merge new settings deliberately.

From PowerShell: 

$ServerRoot = 'C:\SkyFire\Server'
Set-Location $ServerRoot
py -3.11 -m venv .venv
& '.\.venv\Scripts\python.exe' -m pip install -r mapserver/requirements.txt

Substitute the installed Python launcher version if newer. Use the resulting virtual environment explicitly in commands and configuration. The world download helper uses the Python standard library; the map daemon additionally needs the packages in its requirements file.

Copy hubserver.conf.dist, authserver.conf.dist and worldserver.conf.dist to their corresponding .conf names only when creating a new installation. Keep their current ConfVersion values. The snippets below are changes to those complete files, not replacements for them.

Database preparation

Use four distinct databases, for example sf_cluster_hub, sf_cluster_auth, sf_cluster_world and sf_cluster_characters. Configure a MySQL account with the schema privileges required by setup and updates. Automatic database creation additionally requires permission to create those databases; alternatively create the empty schemas yourself and leave AutoCreate disabled.

All participating auth/world nodes and the hub must reference the same authentication database. Each independently running realm needs its own character database. A hub administrator is separate from a game account and its GM/RBAC permissions.

Point the SQL settings at the complete SQL directory from the deployed source revision. INSTALL supplies hub SQL, but do not assume it supplies the complete auth/world/characters update tree. For example, if the matching checkout remains available, use C:/SkyFire/Source/sql. On a separate deployment machine, copy the matching sql/base, required sql/updates domain directories and any deliberately selected pending updates into a deployment SQL directory instead.

The examples below use C:/SkyFire/Source/sql. Replace this everywhere if your SQL package lives elsewhere. Obtain the world base database release required by that revision and set WorldDatabase.BaseSqlFile to the actual extracted SQL file, not a ZIP archive or a guessed filename.

Hub database

In hubserver.conf: 

HubDatabaseInfo = "127.0.0.1;3306;skyfire;REPLACE_PASSWORD;sf_cluster_hub"
LoginDatabaseInfo = "127.0.0.1;3306;skyfire;REPLACE_PASSWORD;sf_cluster_auth"
HubDatabase.SqlPath = "C:/SkyFire/Source/sql"
HubDatabase.AutoSetup = 1
HubDatabase.AutoCreate = 1
HubDatabase.AutoBaseline = 0
HubDatabase.AllowUpdateHashMismatch = 0
HubDatabase.ImportPendingUpdates = 0
Hub.Handoff.Store = "memory"

The updater runs before hub listeners and managed processes start. AutoSetup imports the base only into an empty schema; it is not a repair tool for a partially populated database. An empty SqlPath disables these setup/update checks. Once initialized, a tracked database still receives eligible updates when AutoSetup is disabled.

Pending-update caveat for this revision: The fresh hub base already contains hub_backup_worker.restart_owner, also added by sql/pending_updates/hub/002_node_restarts.sql. Do not blindly enable all pending imports on a fresh base or execute that ALTER twice. Existing databases lacking the column need that migration before coordinated restarts. Keep pending imports disabled for this fresh-install example.

The optional Hub.Handoff.Store = "database" requires the separate sql/pending_updates/hub/001_durable_handoffs.sql schema. This guide uses the default memory store. Follow the matching repository's migration instructions before enabling database storage; it does not enable multi-hub failover.

Never enable AutoBaseline merely to silence missing-schema errors: it records updates as applied without executing them. Do not edit already applied SQL or bypass hash mismatches to conceal a mixed SQL package. The update ledger is skyfire_db_updates.

Authentication database

In authserver.conf: 

LoginDatabaseInfo = "127.0.0.1;3306;skyfire;REPLACE_PASSWORD;sf_cluster_auth"
LoginDatabase.SqlPath = "C:/SkyFire/Source/sql"
LoginDatabase.AutoSetup = 1
LoginDatabase.AutoCreate = 1
LoginDatabase.AutoBaseline = 0
LoginDatabase.AllowUpdateHashMismatch = 0
LoginDatabase.ImportPendingUpdates = 0

The current authentication updates must be applied before enabling cluster handoffs on any participating auth/world node.

World and character databases

In worldserver.conf: 

LoginDatabaseInfo = "127.0.0.1;3306;skyfire;REPLACE_PASSWORD;sf_cluster_auth"
WorldDatabaseInfo = "127.0.0.1;3306;skyfire;REPLACE_PASSWORD;sf_cluster_world"
CharacterDatabaseInfo = "127.0.0.1;3306;skyfire;REPLACE_PASSWORD;sf_cluster_characters"
WorldDatabase.SqlPath = "C:/SkyFire/Source/sql"
CharacterDatabase.SqlPath = "C:/SkyFire/Source/sql"
WorldDatabase.BaseSqlFile = "C:/SkyFire/Database/SFDB_REPLACE_WITH_ACTUAL_RELEASE.sql"
WorldDatabase.AutoSetup = 1
WorldDatabase.AutoCreate = 1
WorldDatabase.AutoBaseline = 0
WorldDatabase.AllowUpdateHashMismatch = 0
WorldDatabase.ImportPendingUpdates = 0
CharacterDatabase.AutoSetup = 1
CharacterDatabase.AutoCreate = 1
CharacterDatabase.AutoBaseline = 0
CharacterDatabase.AllowUpdateHashMismatch = 0
CharacterDatabase.ImportPendingUpdates = 0
RealmID = 1
WorldServerPort = 8085
DataDir = "C:/SkyFire/Server/data"

Ensure the auth database's realmlist contains realm ID 1 with the correct name and client build 18414. In cluster realm-directory mode the configured realm identity/access rules still come from this table, but the live world endpoint comes from worldserver's cluster registration.

Cluster certificates

Use a dedicated cluster certificate authority and a different leaf certificate/private key for every node. Certificates must be signed by the same trusted cluster CA, not be separate unrelated CA certificates.

Identity Certificate requirement
Hub Certificate SAN matches the name nodes use as Cluster.HubHost, such as localhost
auth-1 Certificate common name equals Cluster.NodeKey
world-1 Certificate common name equals Cluster.NodeKey; also used to authenticate map downloads
maps-eastern / maps-kalimdor Common name equals provider node_key; SAN contains its advertised numeric IP

Issue cluster leaves with both serverAuth and clientAuth extended key usages. Keep the CA private key outside the running server directory; deploy only the CA certificate and each process's required leaf/key. Restrict private-key access to the service identity and administrators.

For a new single-host PKI, the following PowerShell example creates leaves valid for localhost and 127.0.0.1. Run it only in a new PKI directory, with an OpenSSL CLI available on PATH. The CA key is encrypted; OpenSSL asks for its passphrase during creation and signing. Do not overwrite an existing cluster CA. 

$PkiRoot = 'C:\SkyFire\PKI'
if (Test-Path $PkiRoot) { throw 'Choose a new PKI directory.' }
New-Item -ItemType Directory -Path $PkiRoot | Out-Null
Set-Location $PkiRoot
function Invoke-ClusterOpenSSL {
    & openssl @args
    if ($LASTEXITCODE -ne 0) { throw 'OpenSSL failed; stop and inspect its output.' }
}
@'
[req]
distinguished_name = dn
[dn]
[ca_ext]
basicConstraints = critical,CA:TRUE
keyUsage = critical,keyCertSign,cRLSign
subjectKeyIdentifier = hash
[leaf_ext]
basicConstraints = critical,CA:FALSE
keyUsage = critical,digitalSignature,keyEncipherment
extendedKeyUsage = serverAuth,clientAuth
subjectAltName = DNS:localhost,IP:127.0.0.1
'@ | Set-Content -Encoding ascii cluster.cnf
Invoke-ClusterOpenSSL req -new -x509 -newkey rsa:3072 -sha256 -days 3650 -config cluster.cnf -extensions ca_ext -subj '/CN=SkyFire Cluster CA' -keyout ca-key.pem -out ca.pem
foreach ($NodeKey in 'hub','auth-1','world-1','maps-eastern','maps-kalimdor') {
    Invoke-ClusterOpenSSL req -new -newkey rsa:3072 -nodes -sha256 -config cluster.cnf -subj "/CN=$NodeKey" -keyout "$NodeKey-key.pem" -out "$NodeKey.csr"
    Invoke-ClusterOpenSSL x509 -req -in "$NodeKey.csr" -CA ca.pem -CAkey ca-key.pem -CAcreateserial -days 365 -sha256 -extfile cluster.cnf -extensions leaf_ext -out "$NodeKey.pem"
}
$TlsRoot = 'C:\SkyFire\Server\cluster-tls'
New-Item -ItemType Directory -Force -Path $TlsRoot | Out-Null
Copy-Item ca.pem $TlsRoot
foreach ($NodeKey in 'hub','auth-1','world-1','maps-eastern','maps-kalimdor') {
    Copy-Item "$NodeKey.pem","$NodeKey-key.pem" $TlsRoot
}

For multiple machines, issue each leaf with its actual DNS/IP SANs instead of copying these localhost identities unchanged. Track certificate expiry and renew before it interrupts registration or downloads.

Hub and node configuration

Hubserver

Add to the complete hubserver.conf: 

Hub.BindIP = "127.0.0.1"
Hub.Port = 9100
Hub.Cluster.Enable = 1
Hub.Cluster.Certificate = "cluster-tls/hub.pem"
Hub.Cluster.PrivateKey = "cluster-tls/hub-key.pem"
Hub.Cluster.CA = "cluster-tls/ca.pem"
Hub.MapData.Readers = "world-1"
Hub.Restart.Countdown = 60
Web.Enable = 1
Web.BindIP = "127.0.0.1"
Web.Port = 54880
Web.Root = "web"
Web.AllowRemote = 0
Hub.LegacyIngress.Enable = 1
Hub.LegacyIngress.BindIP = "127.0.0.1"
Hub.LegacyIngress.Port = 3723
Hub.LegacyIngress.PreserveClientIP = 1

This example uses the legacy authentication profile below. Use the separate Authnet profile if that is how your client installation authenticates.

Authserver: legacy authentication profile

Cluster.Enable = 1
Cluster.Handoff.Enable = 1
Cluster.RealmDirectory.Enable = 1
Cluster.HubHost = "localhost"
Cluster.HubPort = 9100
Cluster.NodeKey = "auth-1"
Cluster.NodeName = "Primary authentication"
Cluster.AdvertiseAddress = "127.0.0.1"
Cluster.AuthEndpoint = "legacy"
Cluster.Certificate = "cluster-tls/auth-1.pem"
Cluster.PrivateKey = "cluster-tls/auth-1-key.pem"
Cluster.CA = "cluster-tls/ca.pem"
BindIP = "127.0.0.1"
RealmServerPort = 3724
Realm.ProxyProtocol.Enable = 1
Realm.ProxyProtocol.TrustedPeers = "127.0.0.1"

Point the client's authentication endpoint at the hub's port 3723, using the host/port configuration supported by your client setup. The backend remains on 3724. With PROXY protocol enabled, direct client connections to the backend are rejected because they lack the trusted hub header.

Alternative: Authnet authentication profile

Start with a working Authnet setup, including its required account/module/proof configuration for your client. The cluster settings do not create those Authnet prerequisites. Keep the common certificate, handoff and realm-directory settings above, then select: 

# authserver.conf
Authnet.Enabled = 1
Authnet.BindIP = "127.0.0.1"
Authnet.Port = 1119
Cluster.AuthEndpoint = "authnet"
Authnet.ProxyProtocol.Enable = 1
Authnet.ProxyProtocol.TrustedPeers = "127.0.0.1"

# hubserver.conf
Hub.LegacyIngress.Enable = 0
Hub.AuthnetIngress.Enable = 1
Hub.AuthnetIngress.BindIP = "127.0.0.1"
Hub.AuthnetIngress.Port = 1118
Hub.AuthnetIngress.PreserveClientIP = 1

Point the Authnet client endpoint at hub port 1118. An auth node advertises one selected endpoint; enabling both ingress listeners does not make one node advertise both protocols. Serving both requires separately registered suitable backend endpoints.

Worldserver

Cluster.Enable = 1
Cluster.Handoff.Enable = 1
Cluster.HubHost = "localhost"
Cluster.HubPort = 9100
Cluster.NodeKey = "world-1"
Cluster.NodeName = "Primary world"
Cluster.AdvertiseAddress = "127.0.0.1"
Cluster.Certificate = "cluster-tls/world-1.pem"
Cluster.PrivateKey = "cluster-tls/world-1-key.pem"
Cluster.CA = "cluster-tls/ca.pem"
BindIP = "127.0.0.1"

For remote players, Cluster.AdvertiseAddress must be a client-reachable address and the world listener must accept those connections. Do not advertise localhost to clients on another computer.

Mapservers and full game-data delivery

Place the complete extraction under C:/SkyFire/Server/data. Full-data mode serves maps, vmaps, mmaps, dbc, db2 and cameras. Do not put configuration files, credentials or logs under this data root.

Every extracted map ID must have exactly one owner. Assigning only 0 and 1 covers the two main continents, not instances, transport maps or other continents. Both the provider TOMLs and worldserver's MapData.Sources must contain matching assignments.

The following PowerShell example derives IDs from the actual terrain/collision/navigation filenames. For a simple two-provider installation it gives map 1 to maps-kalimdor and every remaining ID to maps-eastern. This is an example allocation, not a claim that all remaining maps geographically belong to Eastern Kingdoms. Rebalance the lists later without overlaps or gaps. 

$ServerRoot = 'C:\SkyFire\Server'
$MapIds = @(foreach ($Folder in 'maps','vmaps','mmaps') {
    Get-ChildItem -LiteralPath "$ServerRoot\data\$Folder" -File | ForEach-Object {
        if ($_.Name -match '^([0-9]{4})(?:_[0-9]{2}_[0-9]{2}\.(?:map|vmtile|mmtile)|\.(?:vmtree|mmap))$') {
            [int]$Matches[1]
        }
    }
}) | Sort-Object -Unique
if ($MapIds.Count -eq 0 -or 1 -notin $MapIds) { throw 'Check the extraction and data path.' }
$EastIds = @($MapIds | Where-Object { $_ -ne 1 }) -join ','
$WestIds = '1'
"Eastern TOML: maps = [$EastIds]"
"Kalimdor TOML: maps = [$WestIds]"
'MapData.Sources = "maps-eastern=' + $EastIds + ' maps-kalimdor=' + $WestIds + '"'

Save the printed assignments into the corresponding files. Do not paste PowerShell variable names into TOML or worldserver.conf.

Create mapserver/maps-eastern.toml from the installed template, with: 

node_key = "maps-eastern"
node_name = "Eastern and remaining map assets"
maps = [REPLACE_WITH_PRINTED_EASTERN_IDS]
full_data = true
data_root = "../data"
bind_address = "127.0.0.1"
advertise_address = "127.0.0.1"
port = 54910
hub_host = "localhost"
hub_port = 9100
ca = "../cluster-tls/ca.pem"
certificate = "../cluster-tls/maps-eastern.pem"
private_key = "../cluster-tls/maps-eastern-key.pem"
allowed_world_nodes = ["world-1"]
max_transfers = 4

Create mapserver/maps-kalimdor.toml with the same shared paths and settings, changing: 

node_key = "maps-kalimdor"
node_name = "Kalimdor map assets"
maps = [1]
port = 54911
certificate = "../cluster-tls/maps-kalimdor.pem"
private_key = "../cluster-tls/maps-kalimdor-key.pem"

Each provider includes shared DBC/DB2/camera/collision assets. All providers must use the same extraction set; conflicting shared-file hashes stop bootstrap. On separate machines, copy that same extraction to each provider and update data_root and endpoint settings.

In worldserver.conf, add the generated MapData.Sources line and: 

MapData.Enable = 1
MapData.FullData = 1
MapData.Python = "C:/SkyFire/Server/.venv/Scripts/python.exe"
MapData.Bootstrap = "C:/SkyFire/Server/mapserver/fetch_maps.py"
MapData.CachePath = "C:/SkyFire/Server/map-cache"
MapData.StartupTimeout = 1800
MapData.VerifyCache = 0
MapData.PrefetchRadius = 1
MapData.PrefetchTilesPerSecond = 1

Both provider allowed_world_nodes and hub Hub.MapData.Readers must allow world-1. These names match the world certificate common name. TOML relative paths resolve from the TOML's directory; map bootstrap paths resolve from the world configuration directory.

Full-data bootstrap makes the verified cache the world's effective data directory, including DBC/DB2 and cameras. Missing unassigned files do not silently fall back to the original data folder. The source extraction and cache both consume disk space; old snapshots are not automatically pruned.

VerifyCache=0 reuses successful verification when file identity, size and timestamps are unchanged. New/changed files are hashed. Use VerifyCache=1 for a complete hash pass after suspected corruption; metadata reuse cannot detect silent disk corruption. Never modify an active cache snapshot. Stop providers before replacing their source extraction, then restart them to publish the new manifest.

PrefetchRadius=1 warms nearby tiles in a 3-by-3 area, subject to the configured work limit. Whole continents are cached on disk, not kept entirely resident in memory. A running world keeps reading its selected local snapshot if a provider disconnects; starting or restarting a world requires its providers to be available again.

First startup

1. Initialize authentication before starting the hub

The hub opens the authentication database, so initialize that schema first. From a visible PowerShell terminal: 

Set-Location 'C:\SkyFire\Server'
& '.\authserver.exe' -c '.\authserver.conf'

Allow setup/updates to complete. Cluster connection retries are expected while the hub is offline. Once authentication startup succeeds, stop this temporary standalone instance normally with Ctrl+C. Do not leave it competing for ports with the hub-managed instance.

2. Start hubserver and create an administrator

Set-Location 'C:\SkyFire\Server'
& '.\hubserver.exe' -c '.\hubserver.conf'

Confirm the hub database setup finishes and the cluster/web listeners open. In the hub console, create an administrator: 

admin create hubadmin REPLACE_WITH_A_LONG_UNIQUE_PASSWORD_WITHOUT_SPACES 15
admins
status

Use your own password. Access flags 15 grant local administration; they do not enable remote login. Open the local hub web interface and sign in. Game account creation, editing, bans, mutes, GM and RBAC operations are separate; account creation/editing needs an online worldserver to execute those changes through the hub.

3. Configure managed native services

The hub base creates managed service keys authnet and world. The authnet key is a supervisor label for authserver; the chosen authentication protocol still comes from authserver.conf.

For this example, update the two existing rows in the hub database using your MySQL administration tool: 

UPDATE hub_managed_services
SET executable_path='C:/SkyFire/Server/authserver.exe',
    config_path='C:/SkyFire/Server/authserver.conf',
    working_directory='C:/SkyFire/Server', enabled=1
WHERE service_key='authnet';

UPDATE hub_managed_services
SET executable_path='C:/SkyFire/Server/worldserver.exe',
    config_path='C:/SkyFire/Server/worldserver.conf',
    working_directory='C:/SkyFire/Server', enabled=1
WHERE service_key='world';

Then run .reload db_records in the hub console. Absolute paths avoid launching a different installation accidentally. A managed-service record controls a local process; it does not register a live cluster route by itself.

4. Start both map providers

Open a separate PowerShell terminal for each command: 

Set-Location 'C:\SkyFire\Server'
& '.\.venv\Scripts\python.exe' mapserver/mapserver.py --config mapserver/maps-eastern.toml

Set-Location 'C:\SkyFire\Server'
& '.\.venv\Scripts\python.exe' mapserver/mapserver.py --config mapserver/maps-kalimdor.toml

Wait for both providers to finish cataloging and register ready. Use registry in the hub console and inspect Server Health. The initial manifest build may take time on a large extraction.

Mapserver is an independent Python daemon. The hub's native managed-service Start/Stop mechanism does not launch it. For unattended operation, configure these exact commands, their working directory and service identity in your Windows service manager or Task Scheduler. Local helper scripts are not supplied merely by running INSTALL.

5. Prewarm the world cache

With the hub and providers running, use another terminal: 

Set-Location 'C:\SkyFire\Server'
& '.\.venv\Scripts\python.exe' mapserver/fetch_maps.py --world-config worldserver.conf

Wait for successful completion. The first transfer/hash pass can be lengthy. Prewarming avoids putting that entire operation inside the hub's native process-start deadline. Subsequent startups still authenticate providers and verify manifests, but unchanged verified files use the faster metadata checks.

6. Start authentication and the world through the hub

In the hub console: 

start authnet
start world
registry
realms
routing
status

The first world launch may import a large world base and apply SQL updates. Allow it to complete and inspect its own log before treating an unresponsive startup indicator as a crash. Do not launch another copy while it initializes. For a slow initial database import, initialize the world separately while the hub and providers are available, shut it down normally once ready, and then start it through the hub for supervision.

Before spawning a managed world, the hub checks that all assigned map providers have fresh, ready registrations and acceptable policy/capacity. Missing providers block the launch with a dependency error. World bootstrap performs additional discovery, TLS and file checks before gameplay readiness.

Verify the installation

  1. Confirm the registry contains the expected auth, world and two map identities with no duplicate realm owner.
  2. Confirm realms shows the configured realm with a ready, reachable world endpoint.
  3. Log in through the hub authentication ingress. Check routing counters to distinguish a hub login from a direct backend connection.
  4. Enter the world and check movement, collision, pathfinding and an inter-continent transfer. Watch world logs as well as map/server health.
  5. Confirm the world sidebar reports uptime, players, CPU and update time. Map health should report uptime, memory, CPU and transfer statistics. No map network traffic with a warm cache is normal.
  6. Resize or collapse the global server-status sidebar as needed. Status refreshes independently so ordinary page interaction continues.
  7. Test graceful restart during a planned test window and confirm saved character state after reconnecting.

Graceful shutdown and coordinated restart

World commands can be sent from the hub console or its web console. For the default world, examples in the hub console are: 

world server shutdown 300
world server restart 300
.server shutdown time 23:00
.server restart time 23:00
.server shutdown cancel
.server restart cancel

Countdowns use seconds; the time form uses server local time. The world saves characters during normal shutdown. Avoid killing the process as a routine restart procedure.

Restart all in the web interface refreshes the running managed auth/world services and registered restart-capable mapservers while keeping the hub and web interface online. It performs a graceful world countdown, refreshes map daemons, waits for readiness, then starts the participating native services again. Previously stopped native services remain stopped. The backup worker is outside this restart set.

After updating Python files, externally restart an older map daemon once so it loads the restart-capable script. Otherwise Restart all can refuse with a mapserver readiness/capability error. A failed restart retains maintenance for operator review; inspect the failed stage before releasing it. Drain withdraws new routes/handoffs and does not restart a process or migrate existing players.

Backups and schedules

Install the backup worker dependencies: 

Set-Location 'C:\SkyFire\Server'
& '.\.venv\Scripts\python.exe' -m pip install -r backup/requirements.txt
Copy-Item backup/backup.toml.dist backup/backup.toml

Copy the template only for a new worker configuration. Set the actual MySQL executable paths and a writable archive volume in backup/backup.toml. For example: 

hub_config = "../hubserver.conf"
world_config = "../worldserver.conf"
mysql = "C:/Program Files/MySQL/REPLACE_WITH_INSTALLED_VERSION/bin/mysql.exe"
mysqldump = "C:/Program Files/MySQL/REPLACE_WITH_INSTALLED_VERSION/bin/mysqldump.exe"
output_directory = "../backups"
shutdown_warning_seconds = 3600
server_timezone = "America/Chicago"
allow_live_restore = false

Use server_timezone = "system" to follow the worker computer's configured timezone. America/Chicago explicitly follows Central time, including daylight-saving changes. In the web Backups menu, choose Server time for daily/weekly schedules; existing UTC schedules remain UTC until edited.

Start one worker for this hub: 

& '.\.venv\Scripts\python.exe' backup/backupserver.py --config backup/backup.toml

Confirm the worker is online before using Backup now. Review archive/storage information and job results in the Backups menu. The worker targets auth/hub from hubserver.conf and world/characters from one configured worldserver.conf; it does not automatically cover every additional realm's databases.

Scheduled sets issue the normal world shutdown countdown one hour ahead with the example setting above. The participating managed services stop gracefully, the set runs, and the recorded services restart afterward. Failures retain maintenance for review. Manual Backup now is an immediate request and does not initiate that scheduled countdown/restart cycle.

InnoDB supports transactional dumps. For MyISAM or other nontransactional tables, gracefully stop all game services before a manual backup; the worker uses maintenance and read locks. Hub control tables must remain InnoDB. Do not apply schema migrations during a backup. Separate database dumps are not a single coordinated cluster recovery point.

Live replacement of auth/world/characters is an explicit recovery operation, disabled by default. Enable it only when the hub controls all database writers, including external servers and scripts. Test isolated restore verification first. Live restore requires services offline, a typed confirmation, staging validation and a fresh rollback archive; review completion before ending recovery maintenance and restarting. Hub-database recovery is an offline procedure. Follow the repository's backup/recovery guide for exact prerequisites and commands.

Adding machines or nodes

  • Give every node a unique key, matching certificate common name, descriptive name and non-conflicting endpoint.
  • Replace loopback addresses with private cluster addresses and client-reachable world/ingress addresses as appropriate. Issue matching certificate SANs and update allowlists/firewalls.
  • Keep every participant on the same compatible revision, client-data extraction and database schema.
  • Add new world identities to Hub.MapData.Readers and every relevant provider's allowed_world_nodes.
  • A second active world should own a different realm ID and character database. A same-realm fallback world must remain stopped until the original owner is safely stopped; this is a manual recovery exercise, not seamless failover.
  • The local hub supervisor launches local executables. Remote registration alone does not grant remote process supervision, and coordinated Restart all may refuse unmanaged remote native nodes.
  • Reassign map IDs in both provider configurations and MapData.Sources during maintenance. The current design allows one provider per map, up to 16 providers and 512 assigned map IDs.

Upgrading an existing cluster

  1. Record the deployed revision and take a verified backup of databases and operator configuration.
  2. Stop scheduled work and the backup worker before schema changes. Gracefully stop worlds and authentication, wait for leases to expire, and resolve any active maintenance operation.
  3. Build matching native binaries and run INSTALL. Merge configuration changes from the new templates; keep secrets and local paths out of source control.
  4. Deploy the matching SQL package. Review released and pending migrations against the existing schema and ledger; do not import the destructive base SQL over an existing installation.
  5. Update Python requirements where changed and restart Python daemons so they load the new scripts.
  6. Start the hub, providers and worker as appropriate, prewarm changed cache snapshots, then start native services through the hub.
  7. Repeat login, character-save, map transfer and restart checks. If the browser still shows an old menu, verify Web.Root and the installed web assets before clearing/reloading the browser cache.

MySQL DDL is not an all-or-nothing transaction for a migration file. After a failed migration, inspect what already ran before retrying; do not mark the entire file applied without checking its schema changes.

Troubleshooting

Symptom Check
Hub will not start Read its startup log. Check SQL path, database credentials/schema, certificates and whether another instance already owns ports 9100 or 54880.
Authentication starts but no route is available Confirm the selected Cluster.AuthEndpoint matches the enabled hub ingress, the node is ready, and PROXY protocol/PreserveClientIP settings agree.
Realm unavailable Check live world readiness, RealmID, the auth realmlist identity, duplicate realm owners and the client-reachable advertised address.
World start blocked by map dependency Start the named provider, check its live registry entry and policy, and wait for fresh readiness. Do not bypass the dependency check.
Map TLS/download rejected Check shared CA trust, certificate expiry, CN/node-key match, provider numeric-IP SAN and both world allowlists.
Vmaps/cache startup takes a long time Prewarm once, keep data immutable and use VerifyCache=0 for normal reuse. A missing verification record or changed data requires hashing again.
Missing map/DBC/DB2/camera data Check extraction completeness, full_data and MapData.FullData, every map-ID assignment, and matching shared files across providers.
Restart all requires a restart-capable mapserver Install the matching Python assets and externally restart that daemon once; wait for its new ready registration.
Backup worker offline Start backupserver.py with the correct TOML, Python environment and MySQL tool paths; inspect its output and hub database access.
Nontransactional backup refused Gracefully stop all writers before a manual maintenance backup, or use the scheduled shutdown cycle. Do not force-kill worlds.
Duplicate restart_owner column during update Check whether the fresh base already includes pending migration 002. Do not rerun its ALTER or use broad baselining as a workaround.

Further reference

For remote administration, follow the gateway guide rather than exposing the local web listener directly. The optional gateway adds its own Python/PHP requirements, HTTPS certificate, shared control token and explicit remote administrator permission; these are not required for the local interface described here.

Retrieved from "https://wiki.projectskyfire.org/index.php?title=Clustering_Setup_(Windows_5xx)&oldid=1287"