Created: 2021-02-15. Updated: 2021-02-15
Problems with search speed are resolved, now searching is much faster.
I decided to change the way I publish site data. First, it is difficult to publish database dumps incrementally. Second, I change database structure from time to time, it would be better if users don't need to adjust to the new schema each time. Third, most popular programming languages cannot load data from sql files directly, users need to create a database, import data there, it's rather long process.
So, I will publish site data in a different format. Each month there will be an archive, which will contain torrent files, obtained during that month. To make their size smaller, only "info" section will be present ("pieces" field from it will be removed as well). Data for January and February is coming soon.
It turned out, that there are quite a lot of dumps of existing torrent sites on the internet (both current and past). In most cases they don't contain the full metadata, but they do contain infohashes, that's enough for my purposes (code can obtain the full metadata using BEP9). It means, that I will be able to obtain more torrents, than planned (around a few million each month starting from March). Feel free to send me links to the dumps you are aware of here.
I plan to add "Stats" page, so you can see the site statistics and the monthly progress (visitors, found infohashes, obtained metadata, etc).