Pomoc Excessive HDD Activity on EXT 4 file system when downloading some torrents

[JayB33]

Hello, the symptoms are as follows: Excessive HDD activity (fully saturated) when downloading a torrent from some sources.

As you can see in the screenshot, the read activity of HDD 1 and 2 (RAID 1) is maxed out with just 1 torrent active. The torrent itself is only being read from and written to at a slow speed. If I pause this torrent, the HDD activity returns to normal. Even when the torrent is finished and seeding, the HDD activity remains at maximum, even if I limit the max upload rate. The file system is verified EXT4 using the command line suggested in your latest rTorrent update post.

 

[Usunięty użytkownik pigers]

65 MB/s ? that is half of top speed of HDD

 

[JayB33]

65 MB/s ? that is half of top speed of HDD

Perhaps not when it is randomly accessing some of the data from the platters. I can tell it's saturated because QTS is very slow to respond, rTorrent Pro is slow to log into and movies playing from the NAS freeze up.

 

[Silas Mariusz]

Interesting...
Could you please using SSH execute following commands:

sed -i -e "s:system\.file\.allocate.*:system\.file\.allocate\.set = no:" /usr/bin/rtorrent/etc/rtorrent.conf
/etc/init.d/rtorrent.sh reload

And try again, but make sure to add torrent again (DONT continue previous torrent download). Add new torrent again.

Does this issue happen again?

 

[Silas Mariusz]

One more question:
How much free disk space do you have on that disk volume? Please tell me value in percent %.

 

[JayB33]

I copy and pasted your commands into puTTY and I believe your SSH command may have fixed the issue. I did exactly the same as before except using a new torrent as you suggested: I grabbed a magnet link - metadata was fetched - I used the "Save to..." option in rTorrent Pro to set the correct directory with 'Move Data' ticked, and changed the label to 'Game' & 'Set Ratio Group' to 0 which is set to 3.0. Immediately I noticed download speeds ramping up to much higher rate than they were previously, and the resource manager in QTS showed no abnormal read activity taking place.

14.59% disk space remaining.

The new torrent I downloaded was only 16.85 GB, but the previous torrent was 95.88 GB, that's the only difference.

Thanks for your help. I'll notify you here if another torrent goes bad.

Połączono posty: 5 Wrzesień 2023

5.31 TB total capacity.

 

[Silas Mariusz]

14.59% disk space remaining.

Okay.
This seems to be an issue.
Please always keep for any EXT4/ZFS at least 20% of disk free space. Otherewise your data always be fragmented incorrectly.

The only one solution to fix that, is to copy your data somewhere else, delete content on HDD and restore it from copy. Because I said your data is fragmented. ...or still not use allocation which.

In this case I have to release new version of rtorrent with additional item in settings to enable/disable files allocation.

Thanks for help

 

[Silas Mariusz]

Rtorrent-Pro [x86_64] - Aktualizacje
Please find new update.
Make sure to read note.

 

[JayB33]

Rtorrent-Pro [x86_64] - Aktualizacje
Please find new update.
Make sure to read note.

Okay, thanks for adding the new option. I didn't know the EXT file system became fragmented, I haven't seen any defrag tools in Linux before and I think I read somewhere at the time I first bought a NAS about 7 years ago that it's not necessary to defrag EXT because it has built-in resilience against fragmentation. I suppose it's in the last 15% capacity of the disk is where the limitations of this resilience come into play.

When I upgrade my NAS and disks I'll have to take into consideration ~5% reserved space for the OS and another ~20% to make sure files aren't being fragmented.