Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limited post size on MySQL backend #987

Open
blacklight opened this issue Apr 8, 2024 · 0 comments
Open

Limited post size on MySQL backend #987

blacklight opened this issue Apr 8, 2024 · 0 comments

Comments

@blacklight
Copy link

blacklight commented Apr 8, 2024

Describe the bug

posts.content is modelled as a text column on schema.sql.

The TEXT data type has a limit of 2^16=64KB on MySQL.

This means that longer posts (longer than ~8000/9000 words in a language like English) can't be stored without having the database to throw an error.

(Note that you don't have this limitation on SQLite, where the maximum size of a data type isn't enforced and it only obeys the laws of SQLITE_MAX_LENGTH, which is usually ~1GB).

Steps to reproduce (if necessary)

Steps to reproduce the behavior:

  1. Create a post with more than 64K characters on a MySQL backend
  2. Submit
  3. Submission will fail - a database error will be logged

Expected behavior

Posts up to a reasonable length for a blog article should be allowed on all the supported databases.

Application configuration

  • Single mode or Multi-user mode?
  • Database? mysql
  • Open registration? N/A
  • Federation enabled? N/A

Version or last commit:
N/A

Workarounds

  1. If you are planning to initialize your db through : change the data type of the content column on the posts table - from text to mediumtext.
  2. If you have an existing database, change the data type of posts.content:
ALTER TABLE posts MODIFY COLUMN content MEDIUMTEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL;

Proposal

Change posts.content data type from TEXT (size: 2^16 on MySQL) to MEDIUMTEXT (size: 2^24 on MySQL).

It's quite straightforward how to do it from schema.sql or via alter commands, but I'm not familiar enough with the SQL generation logic in db/create.go (I could take a look into it if there's interest in a PR though). At a first glance it seems that a new ColumnTypeMediumText needs to be added and then handled (as MEDIUMTEXT for MySQL, and simple TEXT for SQLite), but I'm not sure if this will cover everything (plus I guess that there would also be a migration logic to add).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant