Incidents | sota staircase

Immich recovered

Wed, 03 Jun 2026 21:18:17 +0000

Immich recovered

Immich went down

Wed, 03 Jun 2026 21:15:54 +0000

Immich went down

Immich recovered

Wed, 03 Jun 2026 21:03:24 +0000

Immich recovered

Immich went down

Wed, 03 Jun 2026 21:00:49 +0000

Immich went down

Immich recovered

Wed, 03 Jun 2026 01:36:09 +0000

Immich recovered

Immich went down

Wed, 03 Jun 2026 01:33:52 +0000

Immich went down

Object Store recovered

Thu, 28 May 2026 11:24:39 +0000

Object Store recovered

Immich recovered

Thu, 28 May 2026 11:24:15 +0000

Immich recovered

Immich went down

Thu, 28 May 2026 11:21:43 +0000

Immich went down

Object Store went down

Thu, 28 May 2026 11:19:29 +0000

Object Store went down

Immich recovered

Mon, 18 May 2026 19:33:11 +0000

Immich recovered

Git Forge recovered

Mon, 18 May 2026 19:31:52 +0000

Git Forge recovered

Object Store Health recovered

Mon, 18 May 2026 19:31:06 +0000

Object Store Health recovered

Object Store recovered

Mon, 18 May 2026 19:30:49 +0000

Object Store recovered

Git Forge went down

Mon, 18 May 2026 17:31:54 +0000

Git Forge went down

Object Store Health went down

Mon, 18 May 2026 17:31:05 +0000

Object Store Health went down

Object Store went down

Mon, 18 May 2026 17:30:39 +0000

Object Store went down

Immich went down

Mon, 18 May 2026 17:30:20 +0000

Immich went down

minecraft killed the ssd

Sun, 17 May 2026 16:35:00 -0000

disk saturation from emergency recovery of /mnt/ssd is done, we up

minecraft killed the ssd

Sun, 17 May 2026 12:46:00 -0000

disks are saturated; server performance will be slow till est noon tomorrow, 1200 18/5/2026 UTC+8 (SGT)

Object Store recovered

Sun, 17 May 2026 12:18:33 +0000

Object Store recovered

Object Store went down

Sun, 17 May 2026 12:12:37 +0000

Object Store went down

Object Store Health recovered

Sun, 17 May 2026 10:13:13 +0000

Object Store Health recovered

Object Store recovered

Sun, 17 May 2026 10:12:40 +0000

Object Store recovered

Immich recovered

Sun, 17 May 2026 10:12:15 +0000

Immich recovered

minecraft killed the ssd

Sun, 17 May 2026 10:12:00 -0000

should be up (except mc); monitoring stuff

Git Forge recovered

Sun, 17 May 2026 10:10:52 +0000

Git Forge recovered

minecraft killed the ssd

Fri, 15 May 2026 18:42:00 -0000

due to ryan and me having zero time, the restoration will be done on sunday

Object Store Health went down

Sun, 10 May 2026 15:07:03 +0000

Object Store Health went down

Object Store went down

Sun, 10 May 2026 15:06:29 +0000

Object Store went down

Immich went down

Sun, 10 May 2026 15:06:19 +0000

Immich went down

Git Forge went down

Sun, 10 May 2026 15:05:22 +0000

Git Forge went down

minecraft killed the ssd

Sun, 10 May 2026 15:00:00 -0000

server boots off an nvme, core services like the forge, immich, and object store are on a hdd raid array this only affects a personal minecraft instance but we're working on data recovery

Git Forge recovered

Fri, 01 May 2026 17:47:00 +0000

Git Forge recovered

Immich recovered

Fri, 01 May 2026 17:45:14 +0000

Immich recovered

Immich went down

Fri, 01 May 2026 17:42:51 +0000

Immich went down

Git Forge went down

Fri, 01 May 2026 17:40:58 +0000

Git Forge went down

Immich recovered

Fri, 24 Apr 2026 02:51:22 +0000

Immich recovered

Git Forge recovered

Fri, 24 Apr 2026 02:49:50 +0000

Git Forge recovered

Immich went down

Fri, 24 Apr 2026 02:48:09 +0000

Immich went down

Git Forge went down

Fri, 24 Apr 2026 02:47:07 +0000

Git Forge went down

Git Forge recovered

Sun, 29 Mar 2026 22:31:51 +0000

Git Forge recovered

Object Store Health recovered

Sun, 29 Mar 2026 22:30:59 +0000

Object Store Health recovered

Object Store recovered

Sun, 29 Mar 2026 22:30:33 +0000

Object Store recovered

Immich recovered

Sun, 29 Mar 2026 22:30:16 +0000

Immich recovered

Object Store Health went down

Sun, 29 Mar 2026 22:16:57 +0000

Object Store Health went down

Object Store went down

Sun, 29 Mar 2026 22:16:31 +0000

Object Store went down

Immich went down

Sun, 29 Mar 2026 22:15:55 +0000

Immich went down

Git Forge went down

Sun, 29 Mar 2026 22:14:50 +0000

Git Forge went down

Git Forge recovered

Sun, 29 Mar 2026 14:01:57 +0000

Git Forge recovered

Git Forge went down

Sun, 29 Mar 2026 13:59:45 +0000

Git Forge went down

Git Forge recovered

Mon, 09 Mar 2026 21:04:51 +0000

Git Forge recovered

Object Store Health recovered

Mon, 09 Mar 2026 21:04:08 +0000

Object Store Health recovered

Object Store recovered

Mon, 09 Mar 2026 21:03:33 +0000

Object Store recovered

Git Forge went down

Mon, 09 Mar 2026 20:17:50 +0000

Git Forge went down

Object Store went down

Mon, 09 Mar 2026 20:16:33 +0000

Object Store went down

Object Store Health went down

Mon, 09 Mar 2026 20:13:59 +0000

Object Store Health went down

Immich recovered

Fri, 27 Feb 2026 14:39:24 +0000

Immich recovered

Immich went down

Fri, 27 Feb 2026 14:36:55 +0000

Immich went down

Object Store Health recovered

Mon, 09 Feb 2026 23:25:13 +0000

Object Store Health recovered

Object Store recovered

Mon, 09 Feb 2026 23:24:37 +0000

Object Store recovered

Immich recovered

Mon, 09 Feb 2026 23:24:24 +0000

Immich recovered

Git Forge recovered

Mon, 09 Feb 2026 23:22:57 +0000

Git Forge recovered

Git Forge went down

Mon, 09 Feb 2026 14:20:35 +0000

Git Forge went down

Object Store Health went down

Mon, 09 Feb 2026 14:19:55 +0000

Object Store Health went down

Object Store went down

Mon, 09 Feb 2026 14:19:29 +0000

Object Store went down

Immich went down

Mon, 09 Feb 2026 14:18:43 +0000

Immich went down

Object Store Health recovered

Mon, 09 Feb 2026 14:01:08 +0000

Object Store Health recovered

Immich recovered

Mon, 09 Feb 2026 14:00:22 +0000

Immich recovered

Git Forge recovered

Mon, 09 Feb 2026 13:58:52 +0000

Git Forge recovered

Immich went down

Mon, 09 Feb 2026 13:57:14 +0000

Immich went down

Git Forge went down

Mon, 09 Feb 2026 13:56:35 +0000

Git Forge went down

Object Store Health went down

Mon, 09 Feb 2026 13:55:50 +0000

Object Store Health went down

Immich recovered

Mon, 09 Feb 2026 11:57:34 +0000

Immich recovered

Git Forge recovered

Mon, 09 Feb 2026 11:55:55 +0000

Git Forge recovered

Git Forge went down

Mon, 09 Feb 2026 11:53:22 +0000

Git Forge went down

Immich went down

Mon, 09 Feb 2026 11:51:58 +0000

Immich went down

Object Store Health recovered

Sun, 18 Jan 2026 23:07:14 +0000

Object Store Health recovered

Object Store Health went down

Sun, 18 Jan 2026 23:04:02 +0000

Object Store Health went down

Immich recovered

Tue, 13 Jan 2026 14:42:36 +0000

Immich recovered

Immich went down

Tue, 13 Jan 2026 06:09:23 +0000

Immich went down

Immich recovered

Sun, 11 Jan 2026 15:51:12 +0000

Immich recovered

Immich went down

Sun, 11 Jan 2026 15:48:12 +0000

Immich went down

lets slime on mama

Sat, 10 Jan 2026 18:32:00 -0000

we up

Immich recovered

Sat, 10 Jan 2026 17:33:16 +0000

Immich recovered

Immich went down

Sat, 10 Jan 2026 17:30:27 +0000

Immich went down

Git Forge recovered

Sat, 10 Jan 2026 17:22:59 +0000

Git Forge recovered

Object Store recovered

Sat, 10 Jan 2026 17:22:03 +0000

Object Store recovered

Immich recovered

Sat, 10 Jan 2026 17:21:19 +0000

Immich recovered

Git Forge went down

Sat, 10 Jan 2026 17:20:01 +0000

Git Forge went down

Object Store went down

Sat, 10 Jan 2026 17:18:39 +0000

Object Store went down

Immich went down

Sat, 10 Jan 2026 17:18:38 +0000

Immich went down

Immich recovered

Sat, 10 Jan 2026 17:12:22 +0000

Immich recovered

Git Forge recovered

Sat, 10 Jan 2026 17:11:07 +0000

Git Forge recovered

Object Store recovered

Sat, 10 Jan 2026 17:09:37 +0000

Object Store recovered

Object Store Health recovered

Sat, 10 Jan 2026 17:07:25 +0000

Object Store Health recovered

Git Forge went down

Tue, 06 Jan 2026 12:17:44 +0000

Git Forge went down

Object Store Health went down

Tue, 06 Jan 2026 12:17:00 +0000

Object Store Health went down

Object Store went down

Tue, 06 Jan 2026 12:16:49 +0000

Object Store went down

Immich went down

Tue, 06 Jan 2026 12:15:50 +0000

Immich went down

lets slime on mama

Tue, 06 Jan 2026 11:45:00 -0000

ocis is GONE buh bye there will be planned migration of data this is the reckoning thanks bye

everything is DOWN

Tue, 06 Jan 2026 11:30:52 -0000

we are going to migrate our main raid array from RAID5 to RAID10 (1+0) because ryan forgot the drives were SMR instead of CMR retort by ryan: "i did not know there was this distinction" retort-retort by mark: i forgot tbh

Object Store recovered

Tue, 06 Jan 2026 04:15:40 +0000

Object Store recovered

Object Store went down

Tue, 06 Jan 2026 04:12:31 +0000

Object Store went down

Object Store recovered

Mon, 05 Jan 2026 23:54:35 +0000

Object Store recovered

Object Store went down

Mon, 05 Jan 2026 23:45:30 +0000

Object Store went down

Object Store recovered

Mon, 05 Jan 2026 23:39:36 +0000

Object Store recovered

Object Store went down

Mon, 05 Jan 2026 23:33:36 +0000

Object Store went down

Object Store recovered

Mon, 05 Jan 2026 23:28:03 +0000

Object Store recovered

Object Store went down

Mon, 05 Jan 2026 23:24:32 +0000

Object Store went down

Object Store recovered

Mon, 05 Jan 2026 23:18:32 +0000

Object Store recovered

Object Store went down

Mon, 05 Jan 2026 23:15:30 +0000

Object Store went down

Object Store recovered

Mon, 05 Jan 2026 22:30:32 +0000

Object Store recovered

Object Store went down

Mon, 05 Jan 2026 22:24:35 +0000

Object Store went down

Object Store recovered

Mon, 05 Jan 2026 19:15:37 +0000

Object Store recovered

Object Store went down

Mon, 05 Jan 2026 19:09:40 +0000

Object Store went down

Object Store recovered

Mon, 05 Jan 2026 15:34:14 +0000

Object Store recovered

Object Store went down

Mon, 05 Jan 2026 15:24:29 +0000

Object Store went down

Object Store recovered

Mon, 05 Jan 2026 15:12:54 +0000

Object Store recovered

Object Store went down

Mon, 05 Jan 2026 15:06:36 +0000

Object Store went down

Object Store recovered

Mon, 05 Jan 2026 11:39:40 +0000

Object Store recovered

Object Store went down

Mon, 05 Jan 2026 11:36:33 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 17:45:33 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 17:30:38 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 17:03:37 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 17:00:34 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 10:30:44 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 10:09:32 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 09:54:37 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 09:48:36 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 09:33:35 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 08:48:32 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 08:09:36 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 07:54:39 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 05:57:51 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 05:54:34 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 05:48:42 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 05:39:38 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 05:33:52 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 05:18:37 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 05:12:36 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 05:06:35 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 04:54:37 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 04:48:30 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 04:39:36 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 04:21:39 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 04:12:40 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 04:03:39 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 03:45:31 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 03:30:30 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 03:24:46 +0000

Object Store recovered

Git Forge recovered

Sun, 04 Jan 2026 03:13:56 +0000

Git Forge recovered

Git Forge went down

Sun, 04 Jan 2026 03:11:54 +0000

Git Forge went down

Object Store went down

Sun, 04 Jan 2026 03:00:34 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 02:36:42 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 02:27:31 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 02:12:54 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 02:09:30 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 01:42:29 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 01:21:32 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 00:33:30 +0000

Object Store recovered

Object Store went down

Sun, 04 Jan 2026 00:27:52 +0000

Object Store went down

Object Store recovered

Sun, 04 Jan 2026 00:24:35 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 23:45:33 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 23:21:37 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 23:16:06 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 21:39:37 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 21:33:35 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 21:21:37 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 21:06:34 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 21:00:38 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 20:51:29 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 20:45:34 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 20:36:39 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 20:09:59 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 20:03:42 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 20:00:34 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 19:51:34 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 19:24:44 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 19:21:36 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 18:42:36 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 18:36:28 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 18:24:49 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 18:18:38 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 18:09:32 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 17:51:31 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 17:36:38 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 17:30:38 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 16:48:34 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 16:36:34 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 16:30:35 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 16:03:34 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 15:36:29 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 15:24:51 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 15:21:35 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 15:13:03 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 15:00:38 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 14:54:37 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 14:12:37 +0000

Object Store recovered

Git Forge recovered

Sat, 03 Jan 2026 13:07:58 +0000

Git Forge recovered

Git Forge went down

Sat, 03 Jan 2026 13:02:54 +0000

Git Forge went down

Object Store went down

Sat, 03 Jan 2026 13:00:36 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 12:42:45 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 12:21:39 +0000

Object Store went down

Git Forge recovered

Sat, 03 Jan 2026 12:04:57 +0000

Git Forge recovered

Immich recovered

Sat, 03 Jan 2026 12:00:21 +0000

Immich recovered

Object Store Health recovered

Sat, 03 Jan 2026 11:58:08 +0000

Object Store Health recovered

Object Store recovered

Sat, 03 Jan 2026 11:54:39 +0000

Object Store recovered

Object Store Health went down

Sat, 03 Jan 2026 11:50:06 +0000

Object Store Health went down

Object Store went down

Sat, 03 Jan 2026 11:49:01 +0000

Object Store went down

Git Forge went down

Sat, 03 Jan 2026 11:47:29 +0000

Git Forge went down

Git Forge recovered

Sat, 03 Jan 2026 11:43:53 +0000

Git Forge recovered

Object Store recovered

Sat, 03 Jan 2026 11:40:13 +0000

Object Store recovered

Git Forge went down

Sat, 03 Jan 2026 11:35:04 +0000

Git Forge went down

Object Store went down

Sat, 03 Jan 2026 11:34:33 +0000

Object Store went down

Immich went down

Sat, 03 Jan 2026 11:33:30 +0000

Immich went down

system upgrade

Sat, 03 Jan 2026 11:30:07 -0000

moving from NixOS 25.05 to NixOS 25.11, and will reboot afterwards

Object Store recovered

Sat, 03 Jan 2026 10:45:40 +0000

Object Store recovered

Git Forge recovered

Sat, 03 Jan 2026 10:43:55 +0000

Git Forge recovered

Git Forge went down

Sat, 03 Jan 2026 10:38:51 +0000

Git Forge went down

Object Store went down

Sat, 03 Jan 2026 10:18:36 +0000

Object Store went down

Object Store recovered

Sat, 03 Jan 2026 10:12:34 +0000

Object Store recovered

Object Store went down

Sat, 03 Jan 2026 10:00:38 +0000

Object Store went down

lets slime on mama

Sat, 03 Jan 2026 09:40:00 -0000

disks are saturated

Immich recovered

Wed, 31 Dec 2025 16:09:16 +0000

Immich recovered

Object Store recovered

Wed, 31 Dec 2025 16:06:33 +0000

Object Store recovered

Git Forge recovered

Wed, 31 Dec 2025 16:04:53 +0000

Git Forge recovered

Object Store Health recovered

Wed, 31 Dec 2025 16:04:07 +0000

Object Store Health recovered

Git Forge went down

Tue, 30 Dec 2025 10:46:54 +0000

Git Forge went down

Object Store Health went down

Tue, 30 Dec 2025 10:46:34 +0000

Object Store Health went down

Object Store went down

Tue, 30 Dec 2025 10:46:30 +0000

Object Store went down

Immich went down

Tue, 30 Dec 2025 10:45:52 +0000

Immich went down

Object Store Health recovered

Sun, 28 Dec 2025 06:46:07 +0000

Object Store Health recovered

Object Store Health went down

Sun, 28 Dec 2025 06:40:57 +0000

Object Store Health went down

Object Store Health recovered

Sat, 20 Dec 2025 03:28:01 +0000

Object Store Health recovered

Object Store recovered

Sat, 20 Dec 2025 03:27:39 +0000

Object Store recovered

Immich recovered

Sat, 20 Dec 2025 03:27:31 +0000

Immich recovered

Object Store Health went down

Sat, 20 Dec 2025 03:23:06 +0000

Object Store Health went down

Object Store went down

Sat, 20 Dec 2025 03:22:35 +0000

Object Store went down

Immich went down

Sat, 20 Dec 2025 03:21:54 +0000

Immich went down

the server is down, and now we know

Tue, 02 Dec 2025 15:59:00 -0000

resolved

the server is down, and now we know

Sat, 29 Nov 2025 08:05:00 -0000

all bare metal services are currently unresponsive and time out; we are aware and will resolve it in the following hours. any affected users if at all may consider reaching out to mark at joshwel dot co for urgent data retrieval or likewise

object store is being redeployed!!

Fri, 05 Sep 2025 11:27:25 -0000

was previously a monitored preprod/test instance but i like it!!! so i'm redeploying it with slightly different configs!!!

IMMICH IS DOWN, MIGRATING DATA

Tue, 02 Sep 2025 16:00:22 -0000

data is currently being migrated, disks are probably saturated

EVERYTHING IS DOWN

Tue, 02 Sep 2025 16:00:00 -0000

Maintenance completed

EVERYTHING IS DOWN

Thu, 28 Aug 2025 03:00:00 -0000

we are fully upgrading the server because it will explode soon blah blah to serve you better yall dont have school bro but like if needed just tele us or email with (some) of our hearts, Mark Joshwel Ryan Lin

the server is down, and we knew

Tue, 24 Jun 2025 15:17:00 -0000

an unexpected router restart had messed with port forwarding, after a twiddle around router settings (turning port forwarding on and off), we're back up.

the server is down, and we know

Fri, 13 Jun 2025 07:48:00 -0000

it was a bootloop, server restored around 1230h SGT

the server is down, and we know

Thu, 12 Jun 2025 22:20:00 -0000

server is down but ryan is sleeping; it could be a: - heater-related power trip (in which downtime up to six hours may be expected) - or, a router failure (in which downtime up to an hour is expected)

the server is down, and we know

Wed, 04 Jun 2025 01:15:00 -0000

aight we up should be good

the server is down, and we know

Wed, 04 Jun 2025 01:09:00 -0000

a power trip has caused shit to fly, and services are expected to resume around noon for any qualms contact ryan (at) scorchthinks (dot) dev

la poisson du forge is down, and we know

Tue, 18 Mar 2025 12:31:00 -0000

the server restoration post-power fault did not properly start the backend before the frontend. resulting in service disruption although the status monitor remained up. the monitored url has been updated and the container stack was fully restarted in proper. no data was lost.

la poisson du forge is down, and we know

Sun, 16 Mar 2025 05:42:00 -0000

due to a server power fault, the forge's data is currently inaccessible. an investigation will be launched in the following days. any requests for data can be sent to ryan (at) scorchthinks (dot) dev