DB Overhaul Trauma: Running Lemmy on Postgres, Failing Drives, and Null Logos

Post date: January 27, 2025 · Discovered: April 17, 2026 · 4 posts, 87 comments

The platform recently endured major technical trauma, involving moves from OVH to dedicated hardware and critical upgrades like migrating the pict-rs database from sled-db to Postgres. Furthermore, an unscheduled failure forced the replacement of two Raid 1 Samsung MZVL2512HCJQ-00B07 drives during one maintenance window.

The core arguments are dominated by infrastructure risk. 'Illecors' flagged that the pict-rs migration is dangerously complex because the sled database is 'not stateless.' More critically, 'Illecors' warned admins must set the site logo to *NULL* to prevent the lemmy-ui from crashing with a 500 error during image handling. The controversy centers on risk management: do you accept temporary data loss or system instability—like 'split brain' conditions—to avoid a total service blackout?

The clear takeaway is that the platform requires sweeping, difficult infrastructure replacement. The community understands the scope includes object storage migrations and database shifts. The fault lines are drawn between those who document the necessary patches and those who fear the inherent instability of the ongoing migration process.

Key Points

SUPPORT

The database must move from sled-db to Postgres.

This is a necessary, complex data structure upgrade point highlighted by multiple records.

OPPOSE

System instability risks during migration are paramount.

The core debate focuses on whether accepting temporary data loss is preferable to risking a complete service downtime.

SUPPORT

Setting the site logo to NULL prevents application crashes.

'Illecors' specified this technical fix is required to keep the lemmy-ui from throwing a 500 error.

SUPPORT

The pict-rs migration is inherently risky.

'Illecors' noted the process is highly complex because the sled database state cannot be ignored.

SUPPORT

Hardware failures necessitate unscheduled downtime.

A specific incident required a two-hour window due to failing Raid 1 drives.

SUPPORT

Some users reported noticeable speed improvements.

'wise_pancake' reported browsing and loading speeds were significantly snappier post-maintenance.

Source Discussions (4)

This report was synthesized from the following Lemmy discussions, ranked by community score.

163
points
Server maintenance on Sunday - All done!
[email protected]·21 comments·9/7/2023·by Shadow
125
points
Server Maintenance - Jan 29th and Jan 30th at 9AM PT
[email protected]·34 comments·1/27/2025·by Shadow
110
points
Lemmy.ca scheduled maintenance Saturday July 6th - Complete
[email protected]·19 comments·7/4/2024·by Shadow
75
points
2 hour maintenance window on Tuesday Nov 26th, 8am - 10am PST
[email protected]·13 comments·11/24/2024·by Shadow