← Back to Open Source

rtcstats-server: Requirements & Operations Guide

The production hub for rtcstats-server: what to check before go-live, how to pick a deployment shape, security and privacy, monitoring, upgrades, symptom routing, and what to send when you ask for help.

Last updated Applies tortcstats-server

On this page21 sections

How to use this guide

If you are here, then what you are looking for is to set up the open source rtcStats WebRTC observability solution: rtcstats-server. Here's how to approach it:

  1. Start at the before-production checklist
  2. Keep the rest as the operating reference for the running service

The two quickstart guides (AWS App Runner, DigitalOcean Apps Platform) get you to a server that boots. They say plainly that what they produce is not production ready. This page will get you to production ready.

If you are reviewing this deployment rather than performing it, start instead with IT specifications and requirements. It answers an infrastructure or security review directly: ports and firewall rules, external dependencies, high availability, serviceability, data protection, and the known limitations.

Scope:

  • Covered: rtcstats-server only.
  • Requires: a PostgreSQL-compatible database for session metadata, and S3-compatible object storage for dump files.
  • Out of scope: provisioning, sizing, backup, and high availability of those two services, plus rtcstats-features (offline dump processing).

Before-production checklist

Follow this checklist. Be sure you've got the first four nailed down or else they'll bite you later on.

# Check Why it matters
1 NODE_ENV is set to production This isn't cosmetic. See below.
2 storage.s3 is configured and a real dump file has landed in the bucket Without storage, dump files stay on the instance disk and are deleted on the next restart.
3 database.postgres.connectionString is configured Without it, nothing points at the stored dumps.
4 authorization.jwtSecret is set and clients send tokens Unset means no authorization is performed at all.
5 TLS terminates at the load balancer, and the leg behind it is a private network The server does not handle TLS itself.
6 The load balancer health check is HTTP on /healthcheck, not TCP TCP only proves the port is open.
7 Session affinity is on at the load balancer One session holds one WebSocket to one instance for its whole life.
8 Disk is sized for in-flight sessions, and deleteAfterUpload is left at its default Both default to cleaning up. Turning either off is a development convenience that fills a production disk.
9 You have decided what, if anything, forwards to rtcstats.com Forwarding is disabled by default.

Why NODE_ENV=production is item 1

Both quickstart guides deliberately set NODE_ENV to not-yet-production, so promoting a quickstart deploy means changing it. Two behaviors are gated on the literal string production:

  • Multi-process operation. In production the primary process forks one worker per available CPU. Outside production it runs a single process, so a 4 vCPU instance uses one core. server.numberOfProcesses caps the worker count when you want fewer than the machine has; 0 (the default) means use them all.
  • Top-level error handling. The uncaughtException and unhandledRejection handlers are only installed in production. Async event handlers return promises the emitter never awaits, so without those handlers a single rejection can terminate the process, and every session it was collecting with it.

A production deploy that still says not-yet-production is a single-core server one unhandled rejection away from dropping its connections.


Choose your deployment and scaling shape

Two decisions, in this order.

Managed platform or your own machines

Managed platform Your own VMs or containers
Examples AWS App Runner, DigitalOcean Apps Platform EC2, Droplets, ECS, Kubernetes, anything that runs the Dockerfile
Validated guides exist Yes, see AWS App Runner and DigitalOcean No platform-specific guide
TLS and load balancing Handled by the platform You provision it
Watch out for Platform request idle timeouts. App Runner enforces one, and long WebRTC sessions can hit it and reconnect mid-call. Validate against your real session lengths. Nothing platform-specific, but the health check, affinity, and private-network work in the checklist above is yours to do

Managed platforms are the shorter path to a first production deployment. Neither one removes any checklist item.

One instance or a cluster

Start at two instances behind a load balancer if you want high availability at all. A single instance is a single point of failure and there is no way to make it otherwise: when an instance goes down, the sessions it was collecting are lost.

Scale out, not up. The preference is more, smaller machines, limiting blast radius. Plan to add instances when an instance approaches 1,000+ concurrent connections, or when CPU or memory utilization sits consistently above 70 to 80 percent.

Treat ~1,000 concurrent sessions per production node as a planning ceiling and confirm the real number with a load test on your own hardware and traffic. Session weight varies widely and there is no universal sessions-per-vCPU constant.

The full requirements are in the horizontal scaling guide. The database and the object storage must scale along the same dimension: every instance in the cluster writes to the same two services.

Machine requirements

Operating system. Linux (x86_64 or arm64), any distribution able to run a current LTS Node.js runtime. Container deployment is supported.

Node.js. Node.js 22 is the baseline. The project Dockerfile builds on node:22-alpine, and both validated managed deployments use the platform's Node.js 22 runtime. Keep your instances current with the Node.js release lifecycle.

Baseline sizing.

Tier vCPU Memory Local disk Intended use
Pilot / staging 1 2 GB 20 GB Validation, low traffic.
Production node 2 4 GB 50 GB Standard production instance in a cluster of 2 or more.

The 1 vCPU / 2 GB tier is the documented reference configuration used in both quickstart guides. The 2 vCPU / 4 GB tier is a recommended production starting point. Validate both against your own traffic.

Sizing local disk. The server uses two directories on the instance disk: server.workDirectory for sessions still in flight, and server.uploadDirectory for finished dumps waiting to upload. Size for the sum of both: concurrent sessions x average dump size x safety factor. Example with assumptions to adjust: 1,000 concurrent sessions x 5 MB average in-flight dump x 4 = 20 GB. Compressed dumps are often much smaller, hundreds of KB for short calls. Long sessions with many peer connections are larger.

Network. One inbound port (default 8080) receiving plain WebSocket traffic from the load balancer. Instances sit in a private subnet, only the load balancer is internet-facing. Outbound access is required to the object storage endpoint and the database, plus rtcstats.com if you enable forwarding.

Capacity drivers. Per instance, capacity is bounded by CPU (message parsing, delta decompression of getStats payloads, dump compression), disk (temporary files for every active session), and concurrent WebSocket connections (file descriptors and memory per socket).

Architecture

flowchart TD JS["rtcstats-js
(browser / app)"] LB["Load balancer
(TLS terminates here, session affinity on)"] Server["rtcstats-server
1..N instances, N processes each"] Disk["Local disk
work/ during the session
upload/ after it"] Storage[("S3-compatible storage
dump files, on session end")] DB[("PostgreSQL
session metadata + storage pointer")] Cloud["rtcstats.com
(optional, off by default)"] JS -->|"WSS (TLS)"| LB LB -->|"WS (plain, private network)"| Server Server --> Disk Disk --> Storage Server --> DB Server -.-> Cloud

Each rtcstats-js client holds one persistent WebSocket connection to one instance for the duration of its session. Events are written to a temporary file on local disk. When the connection ends, the file is uploaded to storage and the database is updated with a pointer to the storage URL. Instances are stateless between sessions: all durable state lives in the database and the object storage, so a new instance can start at any time.


Security and privacy checklist

Transport

  • rtcstats-js connects over WSS to the load balancer.
  • The load balancer terminates TLS. rtcstats-server does not handle TLS or certificates itself.
  • The leg from load balancer to instances is plain WebSocket on port 8080 and must stay inside a private network: VPC or private subnet, with firewall rules restricting ingress to the load balancer.
  • Certificate issuance and rotation therefore live entirely at the load balancer layer.

Authorization

  • Set authorization.jwtSecret. If it is not set, no authorization is performed and anyone who finds your URL can post data to it.
  • Each client presents a JWT carrying the claims user, session, and conference, passed as the rtcstats-token query parameter during the WebSocket connect phase.
  • The token is signed with an HMAC shared secret and validated by the server. An invalid token closes the WebSocket with a policy-violation (code 1008).
  • Use at least 256 bits of entropy for the secret, have a rotation process that does not require downtime, and set expiration long enough for a session but not indefinite. Six to 24 hours is a good range.
  • Putting session, conference, and user in the claims is also what stops a client impersonating another session, and it is how those identifiers reach the database at all.
  • Full setup: How to authenticate clients with rtcstats-server. For ad-hoc testing, bin/generate-token.js in the repo signs a token from the configured secret.

The HTTP upload path

server.httpUploadPath enables a POST upload as an alternative to WebSockets. It is disabled by default and only active when you set the path. It goes through the same JWT authorization as the WebSocket path. If you are not using it, leave it unset rather than set to something obscure.

Privacy

  • server.obfuscateIpAddresses defaults to true. All IP addresses the server encounters are masked before dumps are written to storage, so stored files carry no IP-level PII. Files forwarded to rtcstats.com are anonymized for the same reason.
  • Do not put PII in the user ID. No names, no email addresses. Hash it, or better, use a random UUID you can associate back to the user yourself. The same applies to conference and session IDs, at lower risk.
  • Geolocation, if you enable GeoIP enrichment, runs before anonymization and keeps coarse granularity by design.
  • Because rtcstats-server runs in your infrastructure, you are the data controller and the self-hosted mediation layer is what satisfies data residency requirements.
  • Verify rather than assume: inspect a stored dump for IP addresses and URLs, then upload a test session and confirm no PII appears in rtcstats.com.
  • Details: How to configure rtcstats-server for privacy and Data privacy and compliance.

Secrets handling

The JWT secret, the S3 credentials, the database connection string, and the rtcstats.com token all arrive through NODE_CONFIG or a production.yaml. Inject them as encrypted environment variables rather than committing them. On AWS, an instance role removes the S3 credentials from the config entirely, which is one fewer secret to rotate.


Monitoring and failure handling

What the server exposes

GET /healthcheck returns 200 OK, with no authentication required. Every other GET path returns 404.

That is the entire monitoring surface. There are no metrics, no stats endpoint, and no structured telemetry. Monitor the rest from the outside:

Signal Where it comes from
Instance health /healthcheck via the load balancer, HTTP not TCP
CPU, memory, free disk Your cloud provider's standard instance monitoring
Concurrent connections Load balancer connection counts
Upload failures, S3 and database errors The runtime logs. The server logs to stdout and stderr
Dumps actually landing Object counts in the bucket, row counts in the database

Run --host-identifier <string> on each instance. The value is stored with every dump, which is the only way to attribute a dump back to the instance that received it once you are running a cluster.

Failure handling

  • An instance goes down. The sessions it was collecting are lost. There is no recovery path for the in-flight data. A cluster limits the blast radius to one instance's worth of sessions.
  • A WebSocket is severed mid-call. The rest of that session is not collected. The client does not resume into a different instance.
  • Planned maintenance. Remove the instance from the load balancer targets, wait for its existing WebSocket connections to drain, then stop it. The server installs no SIGTERM handler, so it does not drain on its own. A hard stop is the same event as an instance going down.
  • Storage or database unreachable. Uploads and inserts fail and are logged. Check the runtime logs before assuming the collection side is at fault.

Upgrades

rtcstats-server lives in the open source monorepo. There are no numbered server releases: the current source on main is the current version, and bugs are fixed there.

  • Cadence. Update at least every 6 months, and never less than once a year.
  • Turn off autodeploy. Both quickstart guides set the deployment trigger to manual on purpose. You do not want a push to main to silently redeploy production.
  • Rolling upgrade. With two or more instances, upgrade the way you do maintenance: drain one instance out of the load balancer, replace it, return it, then move to the next. Sessions in flight on the drained instance are still lost, so drain rather than cut.
  • Dump file format. The current format is version 3. Versions 1 and 2 came from the legacy server and are not supported. Breaking format changes require an upgrade plan and are not made silently.
  • Node.js. Track the Node.js release lifecycle alongside the server.
  • GeoIP data. If you use it, download a fresh GeoLite2 database on every deployment. IP blocks get reassigned.

Symptom routing

Start here when something is wrong with the service. For symptoms in the calls themselves rather than the collector, go to WebRTC troubleshooting.

Symptom Most likely cause Where to go
Health check is green, but the bucket is empty Storage not configured at all, or wrong region / bucket / endpoint, or the instance role does not grant s3:PutObject on that bucket ARN AWS or DigitalOcean Stage 2
Dumps existed, then vanished after a restart Storage not configured. Files are kept locally and removed on the next restart Checklist item 2
Clients cannot connect at all JWT. jwt malformed, rtcstats-token is missing, or TokenExpiredError, all visible in the server logs. The close code is 1008 Authenticate clients
Anyone can post data to your server authorization.jwtSecret is unset, so no authorization runs Checklist item 4
Sessions are truncated or cut mid-call Load balancer is not sticky, an instance restarted, or a platform idle timeout fired Horizontal scaling, and the App Runner idle timeout note above
Disk fills up deleteAtStart or deleteAfterUpload turned off, or upload failures leaving files in uploadDirectory Sizing above, then the runtime logs
One core busy, others idle NODE_ENV is not production, so the server is running a single process Checklist item 1
The process dies under load Same cause. The error handlers are only installed in production Checklist item 1
No country or city on sessions maxmind.path unset, or the .mmdb file is not where the config says GeoIP enrichment
Sessions have no user, conference, or session ID Those fields come from the JWT claims. No token means no identifiers Session, conference and user identifiers
Nothing appears in rtcstats.com Forwarding is off by default and needs a token, or randomPercentage is sampling below 1.0 Integration guide
Database rows appear but dumps do not, or the reverse Two different services, two different failures. Check the runtime logs for which one errored Monitoring above

Requesting support

Send these with the first message. Every one of them is something we would otherwise have to ask for, and each round trip costs a day.

About the deployment

  • Commit SHA or approximate date of the rtcstats/rtcstats checkout you are running
  • Node.js version and how you deploy: managed platform and which one, container, or VM
  • Instance count and size, and whether the problem shows on one instance or all of them

About the configuration

  • Your NODE_CONFIG or config file, with the JWT secret, S3 credentials, database connection string, and rtcstats.com token removed. Send the shape, not the secrets
  • The value of NODE_ENV
  • Load balancer type, and whether session affinity and the HTTP health check are on

About the symptom

  • What you expected, what happened, and when it started
  • Relevant lines from the runtime logs, including the surrounding lines and not only the error
  • Whether it is intermittent or constant, and roughly what fraction of sessions it affects
  • Concurrent session counts and CPU, memory, and disk at the time

Evidence

  • A dump file that shows the problem, anonymized first
  • The session, conference, and user identifiers for an affected session
  • The --host-identifier of the instance, if you run a cluster

Where to send it depends on your arrangement. Open source users without a support contract use the GitHub repository. If you want a ticket system, an SLA, or a shared Slack channel, see premium support for rtcStats open source. Anything else, contact us.


Sources

Was this page helpful?